Super-Resolution Enhancement by Quantum Image Scanning Micros
Super-Resolution Enhancement by Quantum Image Scanning Micros
Super-Resolution Enhancement by Quantum Image Scanning Micros
https://doi.org/10.1038/s41566-018-0324-z
The principles of quantum optics have yielded a plethora of ideas to surpass the classical limitations of sensitivity and resolu-
tion in optical microscopy. While some ideas have been applied in proof-of-principle experiments, imaging a biological sample
has remained challenging, mainly due to the inherently weak signal measured and the fragility of quantum states of light. In
principle, however, these quantum protocols can add new information without sacrificing the classical information and can
therefore enhance the capabilities of existing super-resolution techniques. Image scanning microscopy, a recent addition to
the family of super-resolution methods, generates a robust resolution enhancement without reducing the signal level. Here, we
introduce quantum image scanning microscopy: combining image scanning microscopy with the measurement of quantum pho-
ton correlation allows increasing the resolution of image scanning microscopy up to twofold, four times beyond the diffraction
limit. We introduce the Q-ISM principle and obtain super-resolved optical images of a biological sample stained with fluores-
cent quantum dots using photon antibunching, a quantum effect, as a resolution-enhancing contrast mechanism.
T
he diffraction limit, as formulated by Abbe, sets the attain- ISM each pixel in a detector array acts as a small pinhole in a confo-
able resolution in far-field optical microscopy to about half of cal laser-scanning microscope (CLSM), recording an image with a
the visible wavelength1, hindering its applicability in life sci- resolution twice as fine as the diffraction limit16,19. Thus, one can
ence studies at very small scales. Over the past two decades, several construct a super-resolved image without reducing the collected sig-
super-resolution methods have successfully overcome the diffrac- nal level20,21. Some variants offer the same resolution improvement
tion limit, including emission depletion microscopy, localization within an all-optical set-up, reducing the requirement for computa-
microscopy and structured illumination microscopy2–6. The con- tional power and fast imagers22–24. Although modest, this resolution
tinuous and rapid improvement in detector technology has enabled improvement is robust to changes in the sample and label type and
two more recent developments in the field of super-resolution can be integrated with additional microscopy modalities25,26.
microscopy, which are the centre of this work: quantum super-res- We present here a new super-resolution scheme, quantum image
olution microscopy and image scanning microscopy (ISM). As for scanning microscopy (Q-ISM), utilizing the measurement of quan-
the first, a surge of interest in super-resolution imaging based on tum correlations in an ISM architecture. During a confocal scan,
quantum optics concepts7–13, inspired and facilitated by the progress each pair of detectors in a detector array generates a sharp image
in high temporal resolution imagers, resulted in a few successful using photon correlations. These images are merged together to
proof-of-principle demonstrations7,8,14. The second, ISM, relies on surpass the resolution of both ISM and photon correlation measure-
a small array of fast detectors and offers a twofold enhancement in ments separately. By violating both the classical light and uniform
resolution15,16. Because ISM is compatible with a standard confocal illumination assumptions, we were able to obtain super-resolved
microscope architecture it has already been integrated into com- images of a biological sample based on a quantum optical effect,
mercial products. namely photon antibunching.
While all super-resolution modalities violate at least one of the
basic assumptions of Abbe’s theory, many rely on breaking more Resolution improvement in Q-ISM
than one. For instance, stimulated emission depletion and saturated Similarly to ISM, Q-ISM relies on a small modification of the stan-
structured illumination microscopy breach both the assumption of dard confocal microscope set-up (Fig. 1a). We use a standard con-
a linear response of a fluorophore to the excitation light and that of focal excitation scheme in which a pulsed blue laser beam (473 nm)
a uniform illumination field17,18. In contrast, the few demonstrations is focused through a high-numerical-aperture (NA) objective lens
of quantum super-resolution microscopy7,8,14 relied solely on violat- while the sample is scanned with a piezo stage. In a modified detec-
ing the implicit assumption, underlying Abbe’s derivation, that light tion scheme the sample fluorescence is collected through the same
behaves as waves rather than particles. ISM, as well, depends on objective, further magnified, and then imaged onto a honeycomb
violating a single assumption, uniform illumination of the sample. lattice fibre bundle (inset of Fig. 1a). Each fibre (14 in total) routes
An alternative approach to take advantage of the quantum prop- the light impinging on it to an individual singe-photon avalanche
erties of light to break Abbe’s limit is to enhance an already estab- detector (SPAD) (SPCM-AQ4C, Perkin-Elmer) feeding a time-
lished super-resolution method with the extra information held in correlated single-photon counting (TCSPC) card logging the arrival
a photon correlation measurement12. ISM is a natural candidate for times of the detected photons. Further information on the optical
this purpose as it does not compromise the collected signal level. In set-up is provided in the Methods.
Department of Physics of Complex Systems, Weizmann Institute of Science, Rehovot, Israel. 2Department of Physics, Stanford University, Stanford, CA,
1
USA. 3Institute of Experimental Physics, Faculty of Physics, University of Warsaw, Warsaw, Poland. 4These authors contributed equally: Ron Tenne, Uri
Rossman, Batel Rephael. *e-mail: dan.oron@weizmann.ac.il
TCSPC SPAD
T1
(1) K (1)
( 1
)
image G ISM (x ) = ∑ α=1 Gα x − xa . As an illustrative example,
2
Supplementary Section 2 presents a derivation of an explicit expres-
L1 x3 x2 sion of the ISM image in the case of a Gaussian PSF.
(1)
FB Although G ISM (x ) contains spatial frequencies two times larger
ISM
than the maximal spatial frequency contained in a widefield image,
c x1 ISM d 3,000 its resolution is enhanced only by a factor of ~ 2 . To extend the
Q-ISM image resolution to its Fourier limit one needs to perform Fourier
2,500 g (2)(0) = 0.95
0 95
Detector reweighting16. By digitally amplifying the high spatial frequency
2,000 content, one obtains an image with an up to twofold resolution
Photon pairs
1,500
enhancement (Supplementary Section 3).
g (2)(0) = 0.90 Light emitted from fluorophores, such as dye molecules, quan-
1,000 tum dots and solid-state defects, exhibits photon antibunching29–33,
500 a well-known effect in quantum optics. Each such fluorophore
x3 x3 g (2)(0) = 0.85 emits at most a single photon for every excitation pulse that is much
0 shorter than the fluorescence lifetime. Therefore, the number of
5
–500 0 500
Q-ISM
τ (ns) simultaneously detected photon pairs is lower than the number of
photon pairs at a delay of one or more pulses. The spatial distri-
Fig. 1 | Q-ISM principle of operation. a, The optical set-up used in this bution of missing events of simultaneously detected photon pairs
work. The pinhole in a standard confocal set-up is replaced with a fibre measured by a pair of detectors positioned at xa and xβ yields an
bundle (FB, shown in the inset) routing light into 14 individual SPADs. image of the photon correlation contrast following (Supplementary
Obj, objective lens; DM, dichroic mirror; T1, variable telescope; L1, lens. Section 1)
b, Schematic of the ISM method. The ISM PSF for each detector (green
M
circles) is a product of the excitation laser beam profile (blue circle) and
the detection probability distribution (red circles) centred on the detector’s
(2)
ΔGαβ (x ) ∝ ∑ [h (xi − x )] 2 × h (xa − (xi − x ))
i=1
(3)
position (black circles). For simplicity, the PSFs are drawn as cylindrically
symmetric, although the principle of operation of ISM and Q-ISM does not ×h (xβ − (xi − x ))
require this symmetry. c, Schematic of Q-ISM. The effective PSF for each
detector pair (orange circles) is a product of the two ISM PSFs of the two The effective two-photon absorption and two-photon detection
detectors (green circles). d, The unnormalized second-order correlation contrast results in a PSF, schematically illustrated in Fig. 1c (orange
function, G(2)(τ), generated for three positions in the scan shown in Fig. 3. circles), which is a product of two effective ISM PSFs centred around
1 1
Black dashed lines indicate the average value of incident photon pairs with x and 2 xβ (green circles). The photon correlation image is a sum
2 a
a time delay longer than one pulse. The calculated normalized second- of effective PSFs that are approximately centred at xi + 1 (xa + xβ )
4
order correlation function, g(2)(τ = 0), is shown next to each of the curves. and are narrower by a factor of two compared to the original PSF.
Similar to ISM, the K (K − 1) images of photon correlations can be
2
shifted and summed to obtain the merged Q-ISM image:
In the following, we outline the theoretical concept of super-
resolution in Q-ISM (described in full in Supplementary Section 1). K K
1
(2)
To explore the resolution gain in Q-ISM in this type of set-up, let us −ISM (x ) = ∑
GQ(2) ∑ ΔGαβ x − (xa + xβ ) (4)
4
consider the case of M identical emitters positioned at xi (i = 1,..., M) α = 1 β=α+ 1
and imaged by a unity magnification imaging system whose inco-
herent point spread function (PSF) is h(x). Applying a uniform illu-
mination field yields an intensity image As in the case of ISM, the resolution enhancement of GQ(2) −ISM (x )
(approximately two) can be extended further by performing Fourier
M reweighting. This can increase the resolution of a high signal-to-
G (1) (x ) ∝ ∑ h (x − xi ) (1) noise ratio (SNR) Q-ISM image up to four times beyond the diffrac-
i=1 tion limit (Supplementary Section 3). A more concrete mathematical
treatment of Q-ISM image formation from photon correlations for
Here we follow the notations used in ref. 11 indicating that the fluo- the case of a Gaussian PSF is provided in Supplementary Section 2.
rescence intensity can be viewed as the equal-time first-order Glauber’s To clarify the source of contrast in Q-ISM, Fig. 1d presents
correlation function27,28. In standard ISM, presented schematically unnormalized second-order correlation curves for three positions
in Fig. 1b, the sample is illuminated with a laser beam focused to a in a sample scan. The curves are constructed by performing pixel
(diffraction-limited) spot centred around x = 0 (blue circle)21. As the reassignment for the second-order correlation measurement for
sample position is scanned, the photoluminescence signal is recorded each detector pair, similar to the one performed in equation (4).
by an array of point-like detectors (filled black circles), whose image Each of the curves exhibits a plateau whose value is defined as
plane positions are denoted by xa (α =1, ..., K). Neglecting the fluores- G (2) (∞) , marked by a dashed line, and a small, yet clearly visible,
cence Stokes shift, a single detector α obtains an intensity image antibunching dip at τ = 0. The value of a Q-ISM image at each posi-
tion is simply the difference ΔG (2) = G (2) (∞) − G (2) (0). Note that
M while the relative size of this dip (in proportion to the value of the
Gα(1) (x ) ∝ ∑ h (xi − x ) × h (xa − (xi − x )) (2) plateau) approaches zero for a large number of emitters, the abso-
i=1 lute magnitude of ΔG (2) increases with the number of emitters and
a ×104 Fourier reweighted (FR) ISM, Q-ISM and FR Q-ISM analyses from
CLSM
a scan of a few quantum dots. The standard CLSM image (Fig. 2a)
3
was obtained by summing the intensities of all detectors, effec-
tively treating the fibre bundle as a semi-open pinhole in a confo-
2 cal microscope with an effective diameter of 0.6 Airy units (AU,
λ
1AU = 1.22 NA ). Here we observe a single unresolved elongated
1 patch containing fluctuations, probably resulting from quantum dot
blinking. By shifting the image obtained from each of the detectors,
a process termed pixel reassignment19 (Supplementary Section 6),
b ISM c FR ISM ×104
5 and summing those together, we generate an ISM image (Fig. 2b). In
addition to the enhanced resolution, temporal fluctuations in fluo-
4
rescence during the sample scan (Fig. 2a) are smoothed, becoming
3 unnoticeable in ISM (Fig. 2b), due to the delay accrued between the
2 shifted detector images. The FR ISM image, calculated by applying
a filter on the ISM image shown in Fig. 2b in the Fourier domain, is
1
shown in Fig. 2c (Supplementary Section 3).
To generate a Q-ISM image we analysed the same data set to
d e
Q-ISM FR Q-ISM
100 calculate the second-order photon correlation (Fig. 1d). These
correlations were analysed for each position along the scan using
the photon arrival times in every pair of detectors (Supplementary
Section 5). As described above, taking the difference between the
50
coincident and delayed photon pair detections yields an antibunch-
(2)
ing contrast image for each detector pair, ΔGαβ (x ) , which can be
merged according to equation (4) into a Q-ISM image, GQ(2) −ISM (x ) ,
0 shown in Fig. 2d (Supplementary Sections 5 and 6). Finally, FR
f was performed on the Q-ISM image (Fig. 2e) to flatten the spa-
tial frequency response of the imaging technique and bring the
resolution closer to the theoretical, fourfold enhancement limit
Photons (×104)
Photon pairs
4 100
(Supplementary Section 3). For a clearer comparison of the resolu-
tion improvement, Fig. 2f presents interpolated cross-sections from
2 50
the four images in Fig. 2b–e.
To perform a quantitative assessment of the resolution improve-
0
0.2 0.4 0.6 0.8
0 ment we imaged a few tens of isolated fluorescent beads and derived
Position (μm) the PSFs for wide-field, ISM, FR ISM, Q-ISM and FR Q-ISM imag-
ing. From the width of the PSF we extracted the expected resolu-
Fig. 2 | Resolving emitters with Q-ISM. A 1.5 μm × 1.5 μm confocal scan tion according to the Rayleigh criterion (Supplementary Section 7)
(50 nm steps, 200 ms pixel dwell time) of a sample of a cluster of CdSe/ as 454 nm in the standard wide-field microscope, 352 nm in ISM,
CdS/ZnS quantum dots. a, The summed intensity over the whole detector 231 nm in FR ISM, 261 nm in Q-ISM and finally 194 nm in the FR
array per scan position. b, ISM image, GISM(1)
(x) : the scan image from Q-ISM images. The resolution enhancement factors compared to
each detector is shifted before summation. c, A Fourier reweighted ISM the wide-field image are 1.29, 1.97, 1.74 and 2.34 for the ISM, FR
image obtained from b (Supplementary Section 3). The colour bars for ISM, Q-ISM and FR Q-ISM images respectively. The fact that both
a–c represent the number of detected photon counts. d, Q-ISM image, the ISM and Q-ISM images do not fulfil the theoretical enhance-
GQ(2)
−ISM (x ) : the antibunching signal image of each detector pair is shifted
ment limit of 2 and 2, respectively, is probably due to the non-
and summed (equation (4) and Supplementary Sections 1 and 5). e, A negligible detector size in our system (a de-magnified diameter
Fourier reweighted (FR) image obtained from d (Supplementary Section 3). of ~0.25λ). In FR ISM and especially in FR Q-ISM, the resolution
The colour bars for d and e represent the number of missing detected enhancement is further limited by the finite SNR of the unfiltered
photon pairs. For clarity, the maximal values of the Fourier reweighted images. Although, according to theoretical considerations, the reso-
images in c and e are scaled to that of the ISM (b) and Q-ISM (d) images, lutions of FR ISM and Q-ISM are equal, in the current experimental
respectively. f, Interpolated cross-sections for the dotted white line shown realization FR ISM reaches a somewhat higher resolution. However,
in e for four different analyses: ISM (dashed blue), FR ISM (solid blue), comparing Fig. 2c,d one can see that a significant portion of the
Q-ISM (dashed red) and FR Q-ISM (solid red). Scale bar, 0.5 μm. light intensity in the PSF of FR ISM is distributed in secondary rings
around the emitter. These rings are generated due to the frequency
cutoff enforced by Fourier reweighting (Supplementary Section 3).
While the intensity of these side rings does not reduce the resolu-
its visibility remains unchanged (see Supplementary Section 4 for tion according to the Rayleigh criterion, it can degrade the visibility
detailed discussion of SNR in photon correlation measurements).
(2)
of features in a biological scene, which is usually quite dense.
The normalized second-order correlation dip, g (2) (0) = G(2) (0) , is
G (∞)
shown next to each curve. Quantum microscopy of a fixed cell sample
We further demonstrate the Q-ISM method by imaging a biologi-
Demonstration of resolution enhancement in Q-ISM cal sample of microtubules labelled with quantum dots in fixed
To demonstrate the concept of Q-ISM we used the microscope set- 3T3 cells (see Methods). Figure 3 presents a super-resolved FR
up described in the previous section to image a sample of quantum Q-ISM image of a 3 μm × 3 μm area as well as CLSM, ISM, FR ISM
dots randomly dispersed on a substrate. The detection times of pho- and Q-ISM images analysed from the same data set. Although
tons collected during the scan were parsed and analysed to gener- the SNR is lower in the Q-ISM and FR Q-ISM images (Fig. 3d,e),
ate the ISM and Q-ISM images (see Methods and Supplementary all the visible features in the CLSM (Fig. 3a) and ISM (Fig. 3b)
Section 5 for further details). Figure 2 compares the CLSM, ISM, images are present with a finer resolution. This observation
d Q-ISM e FR Q-ISM
0 50 100 150 0 50 100
Fig. 3 | Q-ISM of labelled microtubule cell samples. a–e, Images analysed from a confocal scan (50 nm steps, 100 ms pixel dwell time) of a 3 μm × 3 μm
section of microtubules in a fixed 3T3 cell labelled with fluorescent quantum dots (QDot 625, Thermo Fisher): CLSM image (a), ISM image (b), FR ISM
image (c), Q-ISM image (d) and FR Q-ISM image (e). The colour bars in a–e follow the same scheme as in Fig. 2a–e, respectively. For clarity, the maximum
intensity of the Fourier reweighted images shown in c and e are scaled to the maximum intensity of their source images in b and d, respectively. Enlarged
images of a section in each image, framed by the white dashed line, are shown below each image. Scale bar, 0.5 μm.
demonstrates the valuable information contained in photon corre- same sample shown in Fig. 3, for different imaging planes, shifted
lation data even though it is based on rare events. Supplementary by 0.4 μm relative to each other. Defocusing the objective results
Section 4 provides a more elaborate discussion regarding SNR in in widening of both the excitation and detection PSFs. Therefore
Q-ISM. As expected from theoretical considerations, the FR ISM a decrease in the integrated signal level accompanies a blurring of
image (Fig. 3c) provides a similar transverse resolution to that the image. Figure 4a–d show the ISM images obtained for objec-
of the unfiltered Q-ISM image (Fig. 3d). However, in the rather tive positions at −0.8, −0.4, 0 and 0.4 μm, respectively (0 designates
dense scene shown in these images, the addition of a pronounced the position of the collection optimal focus). As expected from a
ringing artefact by Fourier reweighting can obscure faint fea- confocal microscope the fine image features are blurred and their
tures adjacent to bright ones. Another advantage of Q-ISM over total intensities decrease as we move from the focus position19,20.
CLSM, ISM and FR ISM, presented in the following section, is an In comparison, the contrast of Q-ISM images, presented in Fig. 4e–
improved axial resolution. h, has a stronger dependence on defocusing, rejecting more of the
out-of-focus contributions.
Z-sectioning in Q-ISM Heuristically, the sharper decrease in Q-ISM image intensity
Q-ISM offers an improved resolution compared to ISM and FR occurs because the contrast results from missing events in which
ISM, not only in the transverse directions but also in the axial two excitation photons are absorbed and two emitted photons are
direction. A theoretical discussion and numerical results compar- detected in a small detector array. The probability of those inde-
ing the z-sectioning capabilities of Q-ISM and ISM are provided in pendent events occurring simultaneously decreases substantially
Supplementary Section 8. To experimentally explore the axial reso- with defocusing because both the excitation power density and the
lution enhancement, Fig. 4 presents images of a thin area from the probability to detect over the small array decrease (Supplementary
ISM
4
e f g h
200
150
Q-ISM
100
50
i ×106
3
6,000
Photon pairs
Photons
2 4,000
1 2,000
0
0
–1 –0.5 0 0.5 1
z (µm)
Fig. 4 | Resolving power of Q-ISM in the axial dimension. a–d, ISM images of a 1 μm × 1 μm (50 nm pixel size) axially thin region in the 3T3 cells sample
at objective defocus positions of −0.8, −0.4, 0 and 0.4 μm, respectively. 0 is the sample focus plane. The colour bar indicates the number of detected
photons. e–h, Q-ISM images analysed from the same measurement as in a–d, respectively. The colour bar indicates the number of missing photon pairs
per scan position. i, Integrated signal of the ISM and Q-ISM images over the area enclosed by the green dashed rectangle in a, shown as blue and red
circles respectively. Violet vertical lines show the defocus positions of the images shown in a–h. Scale bars, 0.25 μm.
Section 8). This effect is highlighted in Fig. 4i, which displays the Relying on the photon correlation contrast poses some additional
intensity sum for ISM (blue) and Q-ISM (red) over a thin section in constraint on sample labelling. First, an isolated fluorescent probe
the sample versus the defocusing position. Although the improved should present some degree of antibunching. Notably, however, this
axial resolution is clearly visible in Fig. 4, the non-negligible sam- is not a particularly limiting condition, because commercial dye
ple thickness prevents a quantitative estimation of the resolution molecules, quantum dots, solid-state defects and some fluorescent
enhancement. To overcome this issue we analysed images of a thin proteins fulfil this requirement33,40,41. Since the photon correlation
sample of quantum dots spin-coated on a coverslip obtained under SNR increases monotonically with fluorescence quantum yield and
similar experimental conditions (Supplementary Section 8). The excitation repetition rate, labels should exhibit a high quantum yield
full-width at half-maximum (FWHM) of the integrated intensity under close-to-saturation conditions. Photostability for a duration
from a wide area in the case of Q-ISM (1 μm) is narrower by a factor on the scale of a second, necessary for high SNR and to avoid scan-
of ~2.1 than the FWHM for ISM (2.1 μm). ning artefacts, has been demonstrated for colloidal quantum dots,
solid-state defects and a variety of dye molecules routinely used in
Discussion bio-imaging33,39,41. Finally, to achieve high photon rates under satu-
Q-ISM is compatible with standard confocal microscopy, with a ration conditions, the emitter’s excited-state lifetime should prefer-
few additional requirements from the experimental set-up and ably be below a microsecond.
the fluorescent markers. First, the set-up requires a small array Unlike some of the previous implementations of imaging
of detectors with nanosecond-scale temporal resolution, such assisted by a photon correlation measurement12, Q-ISM does not
as a small monolithic SPAD array or a fibre bundle detector34–36. rely on spatial sparsity of emitters. Because each emitter adds to the
The latter set-up avoids inter-detector crosstalk, a crucial advan- occurrence of missing photon pairs, dense scenes, such as those
tage for accurate measurement of antibunching. To speed up the presented in Fig. 3, can be imaged. However, increasing the label-
acquisition process, conceptually similar ideas11 may be imple- ling density of the sample does not improve the SNR because both
mented in a wide-field configuration by employing a large-area the number of missing photon pairs and its standard error grow
SPAD array37. Alternatively, using a large SPAD array one can linearly with emitter density (Supplementary Section 4). In fact,
also perform a multifocal adaptation38 of the Q-ISM concept. As given a magnification and pixel size, the saturation of the single-
for the excitation source, either a pulsed or a continuous wave photon detectors sets an upper limit on the sample density, and
(c.w.) laser can be used without affecting the method’s resolution crossing that limit can degrade the image. In principle, the Q-ISM
or SNR. Here, we chose to work with a pulsed excitation source method could be extended to higher correlation orders to achieve
because it is favourable in avoiding photobleaching under saturat- an even higher resolution, but the SNR of correlation orders higher
ing illumination39. than two will decrease with labelling density (Supplementary