-
Noise2Image: Noise-Enabled Static Scene Recovery for Event Cameras
Authors:
Ruiming Cao,
Dekel Galor,
Amit Kohli,
Jacob L Yates,
Laura Waller
Abstract:
Event cameras capture changes of intensity over time as a stream of 'events' and generally cannot measure intensity itself; hence, they are only used for imaging dynamic scenes. However, fluctuations due to random photon arrival inevitably trigger noise events, even for static scenes. While previous efforts have been focused on filtering out these undesirable noise events to improve signal quality…
▽ More
Event cameras capture changes of intensity over time as a stream of 'events' and generally cannot measure intensity itself; hence, they are only used for imaging dynamic scenes. However, fluctuations due to random photon arrival inevitably trigger noise events, even for static scenes. While previous efforts have been focused on filtering out these undesirable noise events to improve signal quality, we find that, in the photon-noise regime, these noise events are correlated with the static scene intensity. We analyze the noise event generation and model its relationship to illuminance. Based on this understanding, we propose a method, called Noise2Image, to leverage the illuminance-dependent noise characteristics to recover the static parts of a scene, which are otherwise invisible to event cameras. We experimentally collect a dataset of noise events on static scenes to train and validate Noise2Image. Our results show that Noise2Image can robustly recover intensity images solely from noise events, providing a novel approach for capturing static scenes in event cameras, without additional hardware.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
Wavefront Randomization Improves Deconvolution
Authors:
Amit Kohli,
Anastasios N. Angelopoulos,
Laura Waller
Abstract:
The performance of an imaging system is limited by optical aberrations, which cause blurriness in the resulting image. Digital correction techniques, such as deconvolution, have limited ability to correct the blur, since some spatial frequencies in the scene are not measured adequately (i.e., 'zeros' of the system transfer function). We prove that the addition of a random mask to an imaging system…
▽ More
The performance of an imaging system is limited by optical aberrations, which cause blurriness in the resulting image. Digital correction techniques, such as deconvolution, have limited ability to correct the blur, since some spatial frequencies in the scene are not measured adequately (i.e., 'zeros' of the system transfer function). We prove that the addition of a random mask to an imaging system removes its dependence on aberrations, reducing the likelihood of zeros in the transfer function and consequently decreasing the sensitivity to noise during deconvolution. In simulation, we show that this strategy improves image quality over a range of aberration types, aberration strengths, and signal-to-noise ratios.
△ Less
Submitted 12 February, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
-
Ring Deconvolution Microscopy: An Exact Solution for Spatially-Varying Aberration Correction
Authors:
Amit Kohli,
Anastasios N. Angelopoulos,
David McAllister,
Esther Whang,
Sixian You,
Kyrollos Yanny,
Federico M. Gasparoli,
Laura Waller
Abstract:
The most ubiquitous form of computational aberration correction for microscopy is deconvolution. However, deconvolution relies on the assumption that the point spread function is the same across the entire field-of-view. It is well recognized that this assumption is often inadequate, but space-variant deblurring techniques generally require impractical amounts of calibration and computation. We pr…
▽ More
The most ubiquitous form of computational aberration correction for microscopy is deconvolution. However, deconvolution relies on the assumption that the point spread function is the same across the entire field-of-view. It is well recognized that this assumption is often inadequate, but space-variant deblurring techniques generally require impractical amounts of calibration and computation. We present a new imaging pipeline, ring deconvolution microscopy (RDM), that leverages the rotational symmetry of most optical systems to provide simple and fast spatially-varying aberration correction. We formally derive theory and algorithms for exact image recovery and additionally propose a neural network based on Seidel coefficients as a fast alternative. We showcase significant enhancements both visually and quantitatively compared to standard deconvolution and existing spatially-varying deconvolution across a diverse range of microscope modalities, including miniature microscopy, multicolor fluorescence microscopy, and point-scanning multimode fiber micro-endoscopy. Our approach enables near-isotropic, subcellular resolution in each of these applications.
△ Less
Submitted 14 March, 2024; v1 submitted 17 June, 2022;
originally announced June 2022.
-
Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging
Authors:
Anastasios N Angelopoulos,
Amit P Kohli,
Stephen Bates,
Michael I Jordan,
Jitendra Malik,
Thayer Alshaabi,
Srigokul Upadhyayula,
Yaniv Romano
Abstract:
Image-to-image regression is an important learning task, used frequently in biological imaging. Current algorithms, however, do not generally offer statistical guarantees that protect against a model's mistakes and hallucinations. To address this, we develop uncertainty quantification techniques with rigorous statistical guarantees for image-to-image regression problems. In particular, we show how…
▽ More
Image-to-image regression is an important learning task, used frequently in biological imaging. Current algorithms, however, do not generally offer statistical guarantees that protect against a model's mistakes and hallucinations. To address this, we develop uncertainty quantification techniques with rigorous statistical guarantees for image-to-image regression problems. In particular, we show how to derive uncertainty intervals around each pixel that are guaranteed to contain the true value with a user-specified confidence probability. Our methods work in conjunction with any base machine learning model, such as a neural network, and endow it with formal mathematical guarantees -- regardless of the true unknown data distribution or choice of model. Furthermore, they are simple to implement and computationally inexpensive. We evaluate our procedure on three image-to-image regression tasks: quantitative phase microscopy, accelerated magnetic resonance imaging, and super-resolution transmission electron microscopy of a Drosophila melanogaster brain.
△ Less
Submitted 10 February, 2022;
originally announced February 2022.
