Scientists at the European Molecular Biology Laboratory (EMBL) have combined AI algorithms with light-field microscopy and light-sheet microscopy. The AI-enhanced hybrid approach reduced the time it took to process images – from days to seconds – and ensured the resulting images were crisp and clear.
Light-field microscopy captures large 3D images that allow researchers to track and measure fine movements at high speed, like the beating heart of a fish larva. However, the technique produces massive amounts of data that can take days to process. Although the final images are complete, they may lack resolution.
Light sheet microscopy examines a single 2D plane of a sample at a time so researchers can image samples at higher resolution. Compared to light-field microscopy, light-film microscopy produces images that are faster to process, although the images are not as complete as they only capture information from one 2D plane at a time.
To combine the advantages of each technique into a single microscopy approach, EMBL researchers used light-field microscopy to image large 3D samples and light-sheet microscopy to train AI algorithms. The trained algorithms created an accurate 3D image of the sample.
Specifically, high-resolution 2D light sheet images served as training and validation data for a convolutional neural network (CNN), which then reconstructed the light-field microscopy data, providing an accurate 3D image of sample.
A representation of a neural network provides a backdrop to the beating heart of a fish larva. Courtesy of Tobias Wuesterfeld.
The use of light sheet microscopy to train CNN has ensured the proper functioning of AI algorithms. âIf you are creating algorithms that produce an image, you have to verify that those algorithms are building the right image,â said Anna Kreshuk, researcher at EMBL.
To demonstrate their approach, the team imaged the cardiac dynamics of the medaka (Japanese rice fish) and the neuronal activity of the zebrafish with volumetric imaging rates up to 100 Hz. During experiments, the researchers have showed that CNN could provide high quality 3D image reconstructions at video bitrate. They also showed that the reconstructed images could be further refined based on the high resolution light sheet images.
âUltimately, we were able to get the best of both worlds in this approach,â said researcher Nils Wagner. âAI has allowed us to combine different microscopy techniques, so that we can image as quickly as light-field microscopy allows and come close to the image resolution of light-layer microscopy. ”
Researcher Robert Prevedel, whose group developed the hybrid microscopy platform, said the real bottleneck in building better microscopes is often not optical technology, but computation. He and Kreshuk believe their approach could be modified to work with different types of microscopes, which would allow many types of specimens to be examined faster and more thoroughly. For example, it could be used to find genes involved in cardiac development or to measure the activity of thousands of neurons at the same time.
Next, the researchers plan to explore whether their method could be applied to samples of larger species, including mammals.
The research was published in Natural methods (www.doi.org/10.1038/s41592-021-01136-0).