Showing 1–15 of 15 results for author: Shechtman, Y

Fluorescence Microscopy: a statistics-optics perspective

Authors: Mohamadreza Fazel, Kristin S. Grussmayer, Boris Ferdman, Aleksandra Radenovic, Yoav Shechtman, Jörg Enderlein, Steve Pressé

Abstract: Fundamental properties of light unavoidably impose features on images collected using fluorescence microscopes. Modeling these features is ever more important in quantitatively interpreting microscopy images collected at scales on par or smaller than light's wavelength. Here we review the optics responsible for generating fluorescent images, fluorophore properties, microscopy modalities leveraging… ▽ More Fundamental properties of light unavoidably impose features on images collected using fluorescence microscopes. Modeling these features is ever more important in quantitatively interpreting microscopy images collected at scales on par or smaller than light's wavelength. Here we review the optics responsible for generating fluorescent images, fluorophore properties, microscopy modalities leveraging properties of both light and fluorophores, in addition to the necessarily probabilistic modeling tools imposed by the stochastic nature of light and measurement. △ Less

Submitted 17 October, 2023; v1 submitted 3 April, 2023; originally announced April 2023.

Additive manufacturing of solid diffractive optical elements via near index matching

Authors: Reut Kedem Orange, Nadav Opatovski, Dafei Xiao, Boris Ferdman, Onit Alalouf, Sushanta Kumar Pal, Ziyun Wang, Henrik von der Emde, Michael Weber, Steffen J. Sahl, Aleks Ponjavic, Ady Arie, Stefan W. Hell, Yoav Shechtman

Abstract: Diffractive optical elements (DOEs) have a wide range of applications in optics and photonics, thanks to their capability to perform complex wavefront shaping in a compact form. However, widespread applicability of DOEs is still limited, because existing fabrication methods are cumbersome and expensive. Here, we present a simple and cost-effective fabrication approach for solid, high-performance D… ▽ More Diffractive optical elements (DOEs) have a wide range of applications in optics and photonics, thanks to their capability to perform complex wavefront shaping in a compact form. However, widespread applicability of DOEs is still limited, because existing fabrication methods are cumbersome and expensive. Here, we present a simple and cost-effective fabrication approach for solid, high-performance DOEs. The method is based on conjugating two nearly refractive index-matched solidifiable transparent materials. The index matching allows for extreme scaling up of the elements in the axial dimension, which enables simple fabrication of a template using commercially available 3D printing at tens-of-micrometer resolution. We demonstrated the approach by fabricating and using DOEs serving as microlens arrays, vortex plates, including for highly sensitive applications such as vector beam generation and super-resolution microscopy using MINSTED, and phase-masks for three-dimensional single-molecule localization microscopy. Beyond the advantage of making DOEs widely accessible by drastically simplifying their production, the method also overcomes difficulties faced by existing methods in fabricating highly complex elements, such as high-order vortex plates, and spectrum-encoding phase masks for microscopy. △ Less

Submitted 27 March, 2023; originally announced March 2023.

Roadmap on Deep Learning for Microscopy

Authors: Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C. D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu , et al. (50 additional authors not shown)

Abstract: Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the… ▽ More Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Diffractive optical system design by cascaded propagation

Authors: Boris Ferdman, Alon Saguy, Onit Alalouf, Yoav Shechtman

Abstract: Modern design of complex optical systems relies heavily on computational tools. These typically utilize geometrical optics as well as Fourier optics, which enables the use of diffractive elements to manipulate light with features on the scale of a wavelength. Fourier optics is typically used for designing thin elements, placed in the system's aperture, generating a shift-invariant Point Spread Fun… ▽ More Modern design of complex optical systems relies heavily on computational tools. These typically utilize geometrical optics as well as Fourier optics, which enables the use of diffractive elements to manipulate light with features on the scale of a wavelength. Fourier optics is typically used for designing thin elements, placed in the system's aperture, generating a shift-invariant Point Spread Function (PSF). A major bottleneck in applying Fourier Optics in many cases of interest, e.g. when dealing with multiple, or out-of-aperture elements, comes from numerical complexity. In this work, we propose and implement an efficient and differentiable propagation model based on the Collins integral, which enables the optimization of diffraction optical systems with unprecedented design freedom using backpropagation. We demonstrate the applicability of our method, numerically and experimentally, by engineering shift-variant PSFs via thin plate elements placed in arbitrary planes inside complex imaging systems, performing cascaded optimization of multiple planes, and designing optimal machine-vision systems by deep learning. △ Less

Submitted 23 February, 2022; originally announced February 2022.

Comments: Manuscript and supplementary information combined

Programmable thermocapillary shaping of thin liquid films

Authors: Ran Eshel, Valeri Frumkin, Matan Nice, Omer Luria, Boris Ferdman, Nadav Opatovski, Khaled Gommed, Maxim Shusteff, Yoav Shechtman, Moran Bercovici

Abstract: We present a method that leverages projected light patterns as a mechanism for freeform deformations of a thin liquid film via the thermocapillary effect. We developed a closed-form solution for the inverse problem of the thin-film evolution equation, allowing to obtain the projection pattern required in order to achieve a desired topography. We experimentally implement the method using a computer… ▽ More We present a method that leverages projected light patterns as a mechanism for freeform deformations of a thin liquid film via the thermocapillary effect. We developed a closed-form solution for the inverse problem of the thin-film evolution equation, allowing to obtain the projection pattern required in order to achieve a desired topography. We experimentally implement the method using a computer controlled light projector, which illuminates any desired pattern onto the bottom of a fluidic chamber patterned with heat absorbing metal pads. The resulting heat map induces surface tension gradients in the liquid-air interface, giving rise to thermocapillary flow that deforms the liquid surface. If a polymer is used for the liquid film, it can then be photocured to yield a solid device. Based on the inverse problem solutions and using this system, we demonstrate the fabrication of several diffractive optical elements (DOEs), including phase masks for extended depth of field imaging, and for 3D localization microscopy. The entire process, from projection to solidification, is completed in less than five minutes, and yields a sub-nanometric surface quality without any post-processing. △ Less

Submitted 2 May, 2022; v1 submitted 31 August, 2021; originally announced September 2021.

Comments: Manuscript and supplementary information combined

Journal ref: Flow , Volume 2 , 2022 , E27

Multiplexed PSF engineering for 3D multicolor particle tracking

Authors: Nadav Opatovski, Yael Shalev Ezra, Lucien E. Weiss, Boris Ferdman, Reut Orange, Yoav Shechtman

Abstract: Three-dimensional spatiotemporal tracking of microscopic particles in multiple colors is a challenging optical imaging task. Existing approaches require a trade-off between photon-efficiency, field of view, mechanical complexity, spectral specificity and speed. Here, we introduce multiplexed point-spread function engineering that achieves photon efficient, 3D, multicolor particle tracking over a l… ▽ More Three-dimensional spatiotemporal tracking of microscopic particles in multiple colors is a challenging optical imaging task. Existing approaches require a trade-off between photon-efficiency, field of view, mechanical complexity, spectral specificity and speed. Here, we introduce multiplexed point-spread function engineering that achieves photon efficient, 3D, multicolor particle tracking over a large field of view. This is accomplished by first chromatically splitting the emission path of a microscope to different channels, engineering the point-spread function of each, and then recombining them onto the same region of the camera. We demonstrate our technique for simultaneously tracking five types of emitters in-vitro, as well as co-localization of DNA loci in live yeast cells. △ Less

Submitted 12 February, 2021; originally announced February 2021.

Comments: 9 pages, 4 figures

Learning an optimal PSF-pair for ultra-dense 3D localization microscopy

Authors: Elias Nehme, Boris Ferdman, Lucien E. Weiss, Tal Naor, Daniel Freedman, Tomer Michaeli, Yoav Shechtman

Abstract: A long-standing challenge in multiple-particle-tracking is the accurate and precise 3D localization of individual particles at close proximity. One established approach for snapshot 3D imaging is point-spread-function (PSF) engineering, in which the PSF is modified to encode the axial information. However, engineered PSFs are challenging to localize at high densities due to lateral PSF overlaps. H… ▽ More A long-standing challenge in multiple-particle-tracking is the accurate and precise 3D localization of individual particles at close proximity. One established approach for snapshot 3D imaging is point-spread-function (PSF) engineering, in which the PSF is modified to encode the axial information. However, engineered PSFs are challenging to localize at high densities due to lateral PSF overlaps. Here we suggest using multiple PSFs simultaneously to help overcome this challenge, and investigate the problem of engineering multiple PSFs for dense 3D localization. We implement our approach using a bifurcated optical system that modifies two separate PSFs, and design the PSFs using three different approaches including end-to-end learning. We demonstrate our approach experimentally by volumetric imaging of fluorescently labelled telomeres in cells. △ Less

Submitted 29 September, 2020; originally announced September 2020.

Comments: 20 pages, 20 figures

DeepSTORM3D: dense three dimensional localization microscopy and point spread function design by deep learning

Authors: Elias Nehme, Daniel Freedman, Racheli Gordon, Boris Ferdman, Lucien E. Weiss, Onit Alalouf, Reut Orange, Tomer Michaeli, Yoav Shechtman

Abstract: Localization microscopy is an imaging technique in which the positions of individual nanoscale point emitters (e.g. fluorescent molecules) are determined at high precision from their images. This is the key ingredient in single/multiple-particle-tracking and several super-resolution microscopy approaches. Localization in three-dimensions (3D) can be performed by modifying the image that a point-so… ▽ More Localization microscopy is an imaging technique in which the positions of individual nanoscale point emitters (e.g. fluorescent molecules) are determined at high precision from their images. This is the key ingredient in single/multiple-particle-tracking and several super-resolution microscopy approaches. Localization in three-dimensions (3D) can be performed by modifying the image that a point-source creates on the camera, namely, the point-spread function (PSF). The PSF is engineered using additional optical elements to vary distinctively with the depth of the point-source. However, localizing multiple adjacent emitters in 3D poses a significant algorithmic challenge, due to the lateral overlap of their PSFs. Here, we train a neural network to receive an image containing densely overlapping PSFs of multiple emitters over a large axial range and output a list of their 3D positions. Furthermore, we then use the network to design the optimal PSF for the multi-emitter case. We demonstrate our approach numerically as well as experimentally by 3D STORM imaging of mitochondria, and volumetric imaging of dozens of fluorescently-labeled telomeres occupying a mammalian nucleus in a single snapshot. △ Less

Submitted 12 September, 2019; v1 submitted 21 June, 2019; originally announced June 2019.

Comments: main text: 9 pages, 5 figures, supplementary information: 29 pages, 20 figures

Journal ref: Nature Methods 17 (2020) 734-740

Multicolor localization microscopy by deep learning

Authors: Eran Hershko*, Lucien E. Weiss*, Tomer Michaeli, Yoav Shechtman

Abstract: Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we apply deep learning to microscopy, and demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing single-molecule methods for spectral classification requi… ▽ More Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we apply deep learning to microscopy, and demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing single-molecule methods for spectral classification require additional optical elements in the emission path, e.g. spectral filters, prisms, or phase masks, our neural net correctly identifies static as well as mobile emitters with high efficiency using a standard, unmodified single-channel configuration. Furthermore, we demonstrate how deep learning can be used to design phase-modulating elements that, when implemented into the imaging path, result in further improved color differentiation between species. △ Less

Submitted 4 July, 2018; originally announced July 2018.

Comments: 34 pages, 16 figures

Fundamental precision bounds for three-dimensional optical localization microscopy with Poisson statistics

Authors: Mikael P. Backlund, Yoav Shechtman, Ronald L. Walsworth

Abstract: Point source localization is a problem of persistent interest in optical imaging. In particular, a number of widely used biological microscopy techniques rely on precise three-dimensional localization of single fluorophores. As emitter depth localization is more challenging than lateral localization, considerable effort has been spent on engineering the response of the microscope in a way that rev… ▽ More Point source localization is a problem of persistent interest in optical imaging. In particular, a number of widely used biological microscopy techniques rely on precise three-dimensional localization of single fluorophores. As emitter depth localization is more challenging than lateral localization, considerable effort has been spent on engineering the response of the microscope in a way that reveals increased depth information. Here we consider the theoretical limits of such approaches by deriving the quantum Cramér-Rao bound (QCRB). We show that existing methods for depth localization with single-objective detection exceed the QCRB by a factor $>\sqrt{2}$, and propose an interferometer arrangement that approaches the bound. We also show that for detection with two opposed objectives, established interferometric measurement techniques globally reach the QCRB. △ Less

Submitted 5 March, 2018; v1 submitted 5 March, 2018; originally announced March 2018.

Comments: Updated to include Supplemental Material as ancillary file

Journal ref: Phys. Rev. Lett. 121, 023904 (2018)

Deep-STORM: super-resolution single-molecule microscopy by deep learning

Authors: Elias Nehme, Lucien E. Weiss, Tomer Michaeli, Yoav Shechtman

Abstract: We present an ultra-fast, precise, parameter-free method, which we term Deep-STORM, for obtaining super-resolution images from stochastically-blinking emitters, such as fluorescent molecules used for localization microscopy. Deep-STORM uses a deep convolutional neural network that can be trained on simulated data or experimental measurements, both of which are demonstrated. The method achieves sta… ▽ More We present an ultra-fast, precise, parameter-free method, which we term Deep-STORM, for obtaining super-resolution images from stochastically-blinking emitters, such as fluorescent molecules used for localization microscopy. Deep-STORM uses a deep convolutional neural network that can be trained on simulated data or experimental measurements, both of which are demonstrated. The method achieves state-of-the-art resolution under challenging signal-to-noise conditions and high emitter densities, and is significantly faster than existing approaches. Additionally, no prior information on the shape of the underlying structure is required, making the method applicable to any blinking data-set. We validate our approach by super-resolution image reconstruction of simulated and experimentally obtained data. △ Less

Submitted 2 May, 2018; v1 submitted 29 January, 2018; originally announced January 2018.

Comments: 7 pages, added code download reference and DOI for the journal version

Journal ref: Optica 5 (2018) 458-464

Measurement-based estimation of global pupil functions in 3D localization microscopy

Authors: Petar N. Petrov, Yoav Shechtman, W. E. Moerner

Abstract: We report the use of a phase retrieval procedure based on maximum likelihood estimation (MLE) to produce an improved, experimentally calibrated model of a point spread function (PSF) for use in three-dimensional (3D) localization microscopy experiments. The method estimates a global pupil phase function (which includes both the PSF and system aberrations) over the full axial range from a simple ca… ▽ More We report the use of a phase retrieval procedure based on maximum likelihood estimation (MLE) to produce an improved, experimentally calibrated model of a point spread function (PSF) for use in three-dimensional (3D) localization microscopy experiments. The method estimates a global pupil phase function (which includes both the PSF and system aberrations) over the full axial range from a simple calibration scan. The pupil function is used to refine the PSF model and hence enable superior localizations from experimental data. To demonstrate the utility of the procedure, we apply it to experimental data acquired with a microscope employing a tetrapod PSF with a 6 micron axial range. The phase-retrieved model demonstrates significant improvements in both accuracy and precision of 3D localizations relative to the model based on scalar diffraction theory. The localization precision of the phase-retrieved model is shown to be near the limits imposed by estimation theory, and the reproducibility of the procedure is characterized and discussed. Code which performs the phase retrieval algorithm is provided. △ Less

Submitted 26 May, 2017; originally announced May 2017.

Comments: 14 pages, 4 figures. This paper was published in Optics Express and is made available as an electronic reprint with the permission of OSA. The paper can be found at the following URL on the OSA website: https://doi.org/10.1364/OE.25.007945

Journal ref: Optics Express, Vol. 25, Issue 7, pp. 7945-7959 (2017)

Sparsity-based single-shot sub-wavelength coherent diffractive imaging

Authors: A. Szameit, Y. Shechtman, E. Osherovich, E. Bullkich, P. Sidorenko, H. Dana, S. Steiner, E. B. Kley, S. Gazit, T. Cohen-Hyams, S. Shoham, M. Zibulevsky, I. Yavneh, Y. C. Eldar, O. Cohen, M. Segev

Abstract: We present the experimental reconstruction of sub-wavelength features from the far-field intensity of sparse optical objects: sparsity-based sub-wavelength imaging combined with phase-retrieval. As examples, we demonstrate the recovery of random and ordered arrangements of 100 nm features with the resolution of 30 nm, with an illuminating wavelength of 532 nm. Our algorithmic technique relies on m… ▽ More We present the experimental reconstruction of sub-wavelength features from the far-field intensity of sparse optical objects: sparsity-based sub-wavelength imaging combined with phase-retrieval. As examples, we demonstrate the recovery of random and ordered arrangements of 100 nm features with the resolution of 30 nm, with an illuminating wavelength of 532 nm. Our algorithmic technique relies on minimizing the number of degrees of freedom; it works in real-time, requires no scanning, and can be implemented in all existing microscopes - optical and non-optical. △ Less

Submitted 20 December, 2011; originally announced December 2011.

Journal ref: Nature Materials 11,455--459(2012)

Sparsity based sub-wavelength imaging with partially incoherent light via quadratic compressed sensing

Authors: Yoav Shechtman, Yonina C. Eldar, Alexander Szameit, Mordechai Segev

Abstract: We demonstrate that sub-wavelength optical images borne on partially-spatially-incoherent light can be recovered, from their far-field or from the blurred image, given the prior knowledge that the image is sparse, and only that. The reconstruction method relies on the recently demonstrated sparsity-based sub-wavelength imaging. However, for partially-spatially-incoherent light, the relation betwee… ▽ More We demonstrate that sub-wavelength optical images borne on partially-spatially-incoherent light can be recovered, from their far-field or from the blurred image, given the prior knowledge that the image is sparse, and only that. The reconstruction method relies on the recently demonstrated sparsity-based sub-wavelength imaging. However, for partially-spatially-incoherent light, the relation between the measurements and the image is quadratic, yielding non-convex measurement equations that do not conform to previously used techniques. Consequently, we demonstrate new algorithmic methodology, referred to as quadratic compressed sensing, which can be applied to a range of other problems involving information recovery from partial correlation measurements, including when the correlation function has local dependencies. Specifically for microscopy, this method can be readily extended to white light microscopes with the additional knowledge of the light source spectrum. △ Less

Submitted 22 April, 2011; originally announced April 2011.

Comments: 16 pages

Far-Field Microscopy of Sparse Subwavelength Objects

Authors: A. Szameit, Y. Shechtman, H. Dana, S. Steiner, S. Gazit, T. Cohen-Hyams, E. Bullkich, O. Cohen, Y. C. Eldar, S. Shoham, E. B. Kley, M. Segev

Abstract: We present the experimental reconstruction of sub-wavelength features from the far-field of sparse optical objects. We show that it is sufficient to know that the object is sparse, and only that, and recover 100 nm features with the resolution of 30 nm, for an illuminating wavelength of λ=532 nm. Our technique works in real-time, requires no scanning, and can be implemented in all existing micro… ▽ More We present the experimental reconstruction of sub-wavelength features from the far-field of sparse optical objects. We show that it is sufficient to know that the object is sparse, and only that, and recover 100 nm features with the resolution of 30 nm, for an illuminating wavelength of λ=532 nm. Our technique works in real-time, requires no scanning, and can be implemented in all existing microscopes - optical and non-optical. △ Less

Submitted 4 October, 2010; originally announced October 2010.