Abstract
This personal hybrid review piece, written in light of my recipience of the UIPAB 2020 young investigator award, contains a mixture of my scientific biography and work so far. This paper is not intended to be a comprehensive review, but only to highlight my contributions to computation-related aspects of super-resolution microscopy, as well as their origins and future directions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Backer AS, Moerner WE (2014) Extending single-molecule microscopy using optical Fourier processing. J Phys Chem B 118(28):8313–8329. https://doi.org/10.1021/jp501778z
Backlund MP, Lew MD, Backer AS, Sahl SJ, Grover G, Agrawal A, Piestun R, Moerner WE (2012) Simultaneous, accurate measurement of the 3D position and orientation of single molecules. Proc Natl Acad Sci U S A 109(47):19087–19092. https://doi.org/10.1073/pnas.1216687109
Baddeley D, Cannell MB, Soeller C (2011)Three-dimensional sub-100 nm super-resolution imaging of biological samples using a phase ramp in the objective pupil. Nano Res 4(6):589–598. https://doi.org/10.1007/s12274-011-0115-z
Badieirostami M, Lew MD, Thompson MA, Moerner WE (2010)Three-dimensional localization precision of the double-helix point spread function versus astigmatism and biplane. Appl Phys Lett 97(16):161103. https://doi.org/10.1063/1.3499652
Barsic A, Grover G, Piestun R (2014)Three-dimensional super-resolution and localization of dense clusters of single molecules. Sci Rep 4:5388. https://doi.org/10.1038/srep05388
Basiji DA, Ortyn WE, Liang L, Venkatachalam V, Morrissey P (2007) Cellular image analysis and imaging by flow cytometry. Clin Lab Med 27(3):653–670. https://doi.org/10.1016/j.cll.2007.05.008
Belthangady C, Royer LA (2019) Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat Methods. https://doi.org/10.1038/s41592-019-0458-z
Ben-Eliezer E, Zalevsky Z, Marom E, Konforti N (2003)All-optical extended depth of field imaging system. J Opt A Pure Appl Opt 5(5):S164–S169. https://doi.org/10.1088/1464-4258/5/5/359
Betzig E, Patterson GH, Sougrat R, Lindwasser OW, Olenych S, Bonifacino JS, Davidson MW, Lippincott-Schwartz J, Hess HF (2006) Imaging intracellular fluorescent proteins at nanometer resolution. Science (New York, NY) 313(5793):1642–1645. https://doi.org/10.1126/science.1127344
Broeken J, Rieger B, Stallinga S (2014) Simultaneous measurement of position and color of single fluorescent emitters using diffractive optics. Opt Lett 39(11):3352–3355. https://doi.org/10.1364/OL.39.003352
Candès EJ (2006) Compressive sampling. Proceedings of the international congress of mathematicians 3:1433–1452
Cohen AE, Moerner WE (2005) Method for trapping and manipulating nanoscale objects in solution. Appl Phys Lett 86(9):1–3. https://doi.org/10.1063/1.1872220
Cox S, Rosten E, Monypenny J, Jovanovic-Talisman T, Burnette DT, Lippincott-Schwartz J, Jones GE, Heintzmann R (2011) Bayesian localization microscopy reveals nanoscale podosome dynamics. Nat Methods 9(2):195–200. https://doi.org/10.1038/nmeth.1812
Dardikman G, Nygate YN, Barnea I, Turko NA, Singh G, Javidi B, Shaked NT (2018) Integral refractive index imaging of flowing cell nuclei using quantitative phase microscopy combined with fluorescence microscopy. Biomed Optics Express. https://doi.org/10.1364/boe.9.001177
Dertinger T, Colyer R, Iyer G, Weiss S, Enderlein J (2009) Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI). Proc Natl Acad Sci 106(52):22287–22292. https://doi.org/10.1073/pnas.0907866106
Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307. https://doi.org/10.1109/TPAMI.2015.2439281
Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory. https://doi.org/10.1109/TIT.2006.871582
Dosset P, Rassam P, Fernandez L, Espenel C, Rubinstein E, Margeat E, Milhiet PE (2016) Automatic detection of diffusion modes within biological membranes using back-propagation neural network. BMC Bioinformatics. https://doi.org/10.1186/s12859-016-1064-z
Dowski ER, Cathey WT (1995) Extended depth of field through wave-front coding. Appl Opt 34(11):1859. https://doi.org/10.1364/AO.34.001859
Ferdman B, Weiss LE, Alalouf O, Haimovich Y, Shechtman Y (2018) Ultrasensitive refractometry via supercritical angle fluorescence. ACS Nano 12(12):11892–11898. https://doi.org/10.1021/acsnano.8b05849
Ferdman B, Nehme E, Weiss LE, Orange R, Alalouf O, Shechtman Y (2020) VIPR: vectorial implementation of phase retrieval for fast and accurate microscopic pixel-wise pupil estimation. Opt Express. https://doi.org/10.1364/oe.388248
Gazagnes S, Soubies E, & Blanc-Féraud L (2017) High density molecule localization for super-resolution microscopy using CEL0 based sparse approximation. ISBI 2017-IEEE Int Symp Biomed Imaging, 4
George TC, Basiji DA, Hall BE, Lynch DH, Ortyn WE, Perry DJ, Seo MJ, Zimmerman CA, Morrissey PJ (2004) Distinguishing modes of cell death using the ImageStream® multispectral imaging flow cytometer. Cytometry A 59A(2):237–245. https://doi.org/10.1002/cyto.a.20048
Gordon-Soffer R, Weiss LE, Eshel R, Ferdman B, Nehme E, Bercovici M, Shechtman Y (2020) Microscopic scan-free surface profiling over extended axial ranges by point-spread-function engineering. Sci Adv 6(44):eabc0332. https://doi.org/10.1126/sciadv.abc0332
Granik N, Weiss LE, Nehme E, Levin M, Chein M, Perlson E, Roichman Y, Shechtman Y (2019)Single-particle diffusion characterization by deep learning. Biophys J 117(2):185–192. https://doi.org/10.1016/j.bpj.2019.06.015
Gualda EJ, Pereira H, Martins GG, Gardner R, Moreno N (2017)Three-dimensional imaging flow cytometry through light-sheet fluorescence microscopy. Cytometry A 91(2):144–151. https://doi.org/10.1002/cyto.a.23046
Hell SW, Sahl SJ, Bates M, Zhuang X, Heintzmann R, Booth MJ, Bewersdorf J, Shtengel G, Hess H, Tinnefeld P, Honigmann A, Jakobs S, Testa I, Cognet L, Lounis B, Ewers H, Davis SJ, … Cordes T (2015) The 2015 super-resolution microscopy roadmap. In Journal of Physics D: Applied Physics (Vol. 48, Issue 44). Institute of Physics Publishing. https://doi.org/10.1088/0022-3727/48/44/443001
Hershko E, Weiss LE, Michaeli T, Shechtman Y (2019) Multicolor localization microscopy and point-spread-function engineering by deep learning. Opt Express. https://doi.org/10.1364/oe.27.006158
Hess ST, Girirajan TPK, Mason MD (2006)Ultra-high resolution imaging by fluorescence photoactivation localization microscopy. Biophys J 91(11):4258–4272. https://doi.org/10.1529/biophysj.106.091116
Holden SJ, Uphoff S, Kapanidis AN (2011) DAOSTORM: an algorithm for high- density super-resolution microscopy. Nat Methods 8(4):279–280. https://doi.org/10.1038/nmeth0411-279
Huang B, Wang W, Bates M, Zhuang X (2008)Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy. Science 319(8):810–813. https://doi.org/10.1126/science.1153529
Huang F, Schwartz SL, Byars JM, Lidke KA (2011) Simultaneous multiple-emitter fitting for single molecule super-resolution imaging. Biomed Optics Express 2(5):1377. https://doi.org/10.1364/BOE.2.001377
Hugelier S, De Rooi JJ, Bernex R, Duwé S, Devos O, Sliwa M, Dedecker P, Eilers PHC, Ruckebusch C (2016) Sparse deconvolution of high-density super-resolution images. Sci Rep. https://doi.org/10.1038/srep21413
Jesacher A, Bernet S, Ritsch-Marte M (2014) Colour hologram projection with an SLM by exploiting its full phase modulation range. Opt Express 22(17):20530–20541. https://doi.org/10.1364/OE.22.020530
Jia S, Vaughan JC, Zhuang X (2014) Isotropic three-dimensional super-resolution imaging with a self-bending point spread function. Nat Photonics 8(4):302–306. https://doi.org/10.1038/nphoton.2014.13
Kay SM (1995) Fundamentals of statistical signal processing: estimation theory. In: Technometrics, p 303. https://doi.org/10.2307/1269750
Kay DB, Wheeless LL (1976) Laser stroboscopic photography. Technique for cell orientation studies in flow. J Histochem Cytochem 24(1):265–268. https://doi.org/10.1177/24.1.768371
Kay DB, Cambier JL, Wheeless LL (1979) Imaging in flow. J Histochem Cytochem 27(1):329–334. https://doi.org/10.1177/27.1.374597
Kim J, Lee JK, & Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2016.182
Kowalek P, Loch-Olszewska H, Szwabiński J (2019) Classification of diffusion modes in single-particle tracking data: feature-based versus deep-learning approach. Phys Rev E. https://doi.org/10.1103/PhysRevE.100.032410
Lew MD, Lee SF, Badieirostami M, Moerner WE (2011) Corkscrew point spread function for far-field three-dimensional nanoscale localization of pointlike objects. Opt Lett. https://doi.org/10.1364/ol.36.000202
Muñoz-Gil G, Garcia-March MA, Manzo C, Martín-Guerrero JD, Lewenstein M (2020) Single trajectory characterization via machine learning. New J Phys. https://doi.org/10.1088/1367-2630/ab6065
Muthumbi A, Chaware A, Kim K, Zhou KC, Konda PC, Chen R, Judkewitz B, Erdmann A, Kappes B, Horstmeyer R (2019) Learned sensing: jointly optimized microscope hardware for accurate image classification. Biomed Optics Express. https://doi.org/10.1364/boe.10.006351
Nehme E, Weiss LE, Michaeli T, Shechtman Y (2018) Deep-STORM: super-resolution single-molecule microscopy by deep learning. Optica 5(4):458. https://doi.org/10.1364/OPTICA.5.000458
Nehme E, Ferdman B, Weiss LE, Naor T, Freedman D, Michaeli T, & Shechtman Y (2020a) Learning an optimal PSF-pair for ultra-dense 3D localization microscopy. http://arxiv.org/abs/2009.14303
Nehme E, Freedman D, Gordon R, Ferdman B, Weiss LE, Alalouf O, Naor T, Michaeli T, Shechtman Y (2020b) DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning. Nat Methods
Ober RJ, Ram S, Ward ES (2004) Localization accuracy in single-molecule microscopy. Biophys J 86(2):1185–1200. https://doi.org/10.1016/S0006-3495(04)74193-4
Ouyang W, Aristov A, Lelek M, Hao X, Zimmer C (2018) Deep learning massively accelerates super-resolution localization microscopy. Nat Biotechnol 36(5):460–468. https://doi.org/10.1038/nbt.4106
Pavani SRP, Thompson MA, Biteen JS, Lord SJ, Liu N, Twieg RJ, Piestun R, Moerner WE (2009) Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using a double-helix point spread function. Proc Natl Acad Sci 106(9):2995–2999. https://doi.org/10.1073/pnas.0900245106
Petrov PN, Shechtman Y, Moerner WE (2017)Measurement-based estimation of global pupil functions in 3D localization microscopy. Opt Express 25(7):7945. https://doi.org/10.1364/OE.25.007945
Pinkard H, Phillips Z, Babakhani A, Fletcher DA, Waller L (2019) Deep learning for single-shot autofocus microscopy. Optica. https://doi.org/10.1364/optica.6.000794
Qu X, Wu D, Mets L, Scherer NF (2004)Nanometer-localized multiple single-molecule fluorescence microscopy. Proc Natl Acad Sci U S A 101(31):11298–11303. https://doi.org/10.1073/pnas.0402155101
Rivenson Y, Göröcs Z, Günaydin H, Zhang Y, Wang H, Ozcan A (2017) Deep learning microscopy. Optica 4(11):1437. https://doi.org/10.1364/OPTICA.4.001437
Rust MJ, Bates M, Zhuang X (2006)Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat Methods 3(10):793–796. https://doi.org/10.1038/nmeth929
Sergé A, Bertaux N, Rigneault H, Marguet D (2008) Dynamic multiple-target tracing to probe spatiotemporal cartography of cell membranes. Nat Methods 5(8):687–694. https://doi.org/10.1038/nmeth.1233
Shechtman Y, Gazit S, Szameit A, Eldar YC, Segev M (2010)Super-resolution and reconstruction of sparse images carried by incoherent light. Opt Lett 35(8):1148–1150
Shechtman Y, Beck A, Eldar YC (2014a) GESPAR: efficient phase retrieval of sparse signals. IEEE Trans Signal Process 62(4):928–938
Shechtman Y, Sahl SJ, Backer AS, Moerner WE (2014b) Optimal point spread function design for 3D imaging. Phys Rev Lett 113(3):133902. https://doi.org/10.1103/PhysRevLett.113.133902
Shechtman Y, Weiss LE, Backer AS, Sahl SJ, Moerner WE (2015) Precise three-dimensional scan-free multiple-particle tracking over large axial ranges with tetrapod point spread functions. Nano Lett 15(6):4194–4199
Shechtman Y, Weiss LE, Backer AS, Lee MY, Moerner WE (2016) Multicolour localization microscopy by point-spread-function engineering. Nat Photonics 10(9):590–594. https://doi.org/10.1038/nphoton.2016.137
Solomon O, Mutzafi M, Segev M, Eldar YC (2018)Sparsity-based super-resolution microscopy from correlation information. Opt Express 26(14):18238. https://doi.org/10.1364/oe.26.018238
Squires A, Cohen A, Biophysics W M-E of & 2018 U (2018)Anti-Brownian traps. Springer, Berlin
Sung Y, Lue N, Hamza B, Martel J, Irimia D, Dasari RR, Choi W, Yaqoob Z, So P (2014)Three-dimensional holographic refractive-index measurement of continuously flowing cells in a microfluidic channel. Phys Rev Appl. https://doi.org/10.1103/PhysRevApplied.1.014002
Szameit A, Shechtman Y, Osherovich E, Bullkich E, Sidorenko P, Dana H, Steiner S, Kley EB, Gazit S, Cohen-Hyams T, Shoham S, Zibulevsky M, Yavneh I, Eldar YC, Cohen O, Segev M (2012)Sparsity-based single-shot subwavelength coherent diffractive imaging. Nat Mater 11(5):455–459. https://doi.org/10.1038/nmat3289
von Chamier L, Laine R, Jukkala J, Spahn C, Krentzel D, Nehme E, Lerche M, Hernández-Pérez S, Mattila P, Karinou E, Holden S, Solak AC, Krull A, Buchholz T-O, Jones M, Royer L, Leterrier C et al (2020) ZeroCostDL4Mic: an open platform to use deep-learning in microscopy. BioRxiv 2020(03):20.000133. https://doi.org/10.1101/2020.03.20.000133
Von Diezmann A, Shechtman Y, & Moerner WE (2017)Three-dimensional localization of single molecules for super-resolution imaging and single-particle tracking. In Chemical Reviews (Vol. 117, Issue 11, pp. 7244–7275). https://doi.org/10.1021/acs.chemrev.6b00629
Wang Q, Moerner WE (2014)Single-molecule motions enable direct visualization of biomolecular interactions in solution. Nat Methods 11(5):555–558. https://doi.org/10.1038/nmeth.2882
Wang Z, Liu D, Yang J, Han W, & Huang T (2015) Deep networks for image super-resolution with sparse prior. Proceedings of the IEEE International Conference on Computer Vision https://doi.org/10.1109/ICCV.2015.50
Weigert M, Schmidt U, Boothe T, Müller A, Dibrov A, Jain A, Wilhelm B, Schmidt D, Broaddus C, Culley S, Rocha-Martins M, Segovia-Miranda F, Norden C, Henriques R, Zerial M, Solimena M, Rink J et al (2018)Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat Methods. https://doi.org/10.1038/s41592-018-0216-7
Weiss LE, Shalev Ezra Y, Goldberg S, Ferdman B, Adir O, Schroeder A, Alalouf O, Shechtman Y (2020)Three-dimensional localization microscopy in live flowing cells. Nat Nanotechnol. https://doi.org/10.1038/s41565-020-0662-0
Wu J, Li J, Chan RKY (2013) A light sheet based high throughput 3D-imaging flow cytometer for phytoplankton analysis. Opt Express. https://doi.org/10.1364/oe.21.014474
Zhang P, Liu S, Chaurasia A, Ma D, Mlodzianoski MJ, Culurciello E, Huang F (2018) Analyzing complex single-molecule emission patterns with deep learning. Nat Methods 15(11):913–916. https://doi.org/10.1038/s41592-018-0153-5
Zhu L, Zhang W, Elnatan D, Huang B (2012) Faster STORM using compressed sensing. Nat Methods 9(7):721–723. https://doi.org/10.1038/nmeth.1978
Acknowledgments
All work presented here is the result of group effort. I have been extremely fortunate in the scientific environments in which I worked, and I have learned from every mentor, colleague, and student I had worked with.
Funding
The author received support from the Zuckerman foundation; from the various funding agencies for the work presented here; and from the private companies, philanthropists, and tax-paying citizens of Israel, the USA, the European Union, and other countries who ultimately pay for this work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shechtman, Y. Recent advances in point spread function engineering and related computational microscopy approaches: from one viewpoint. Biophys Rev 12, 1303–1309 (2020). https://doi.org/10.1007/s12551-020-00773-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12551-020-00773-7