Showing 1–50 of 54 results for author: Bouman, K L

Neural Approximate Mirror Maps for Constrained Diffusion Models

Authors: Berthy T. Feng, Ricardo Baptista, Katherine L. Bouman

Abstract: Diffusion models excel at creating visually-convincing images, but they often struggle to meet subtle constraints inherent in the training data. Such constraints could be physics-based (e.g., satisfying a PDE), geometric (e.g., respecting symmetry), or semantic (e.g., including a particular number of objects). When the training data all satisfy a certain constraint, enforcing this constraint on a… ▽ More Diffusion models excel at creating visually-convincing images, but they often struggle to meet subtle constraints inherent in the training data. Such constraints could be physics-based (e.g., satisfying a PDE), geometric (e.g., respecting symmetry), or semantic (e.g., including a particular number of objects). When the training data all satisfy a certain constraint, enforcing this constraint on a diffusion model not only improves its distribution-matching accuracy but also makes it more reliable for generating valid synthetic data and solving constrained inverse problems. However, existing methods for constrained diffusion models are inflexible with different types of constraints. Recent work proposed to learn mirror diffusion models (MDMs) in an unconstrained space defined by a mirror map and to impose the constraint with an inverse mirror map, but analytical mirror maps are challenging to derive for complex constraints. We propose neural approximate mirror maps (NAMMs) for general constraints. Our approach only requires a differentiable distance function from the constraint set. We learn an approximate mirror map that pushes data into an unconstrained space and a corresponding approximate inverse that maps data back to the constraint set. A generative model, such as an MDM, can then be trained in the learned mirror space and its samples restored to the constraint set by the inverse map. We validate our approach on a variety of constraints, showing that compared to an unconstrained diffusion model, a NAMM-based MDM substantially improves constraint satisfaction. We also demonstrate how existing diffusion-based inverse-problem solvers can be easily applied in the learned mirror space to solve constrained inverse problems. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Event-horizon-scale Imaging of M87* under Different Assumptions via Deep Generative Image Priors

Authors: Berthy T. Feng, Katherine L. Bouman, William T. Freeman

Abstract: Reconstructing images from the Event Horizon Telescope (EHT) observations of M87*, the supermassive black hole at the center of the galaxy M87, depends on a prior to impose desired image statistics. However, given the impossibility of directly observing black holes, there is no clear choice for a prior. We present a framework for flexibly designing a range of priors, each bringing different biases… ▽ More Reconstructing images from the Event Horizon Telescope (EHT) observations of M87*, the supermassive black hole at the center of the galaxy M87, depends on a prior to impose desired image statistics. However, given the impossibility of directly observing black holes, there is no clear choice for a prior. We present a framework for flexibly designing a range of priors, each bringing different biases to the image reconstruction. These priors can be weak (e.g., impose only basic natural-image statistics) or strong (e.g., impose assumptions of black-hole structure). Our framework uses Bayesian inference with score-based priors, which are data-driven priors arising from a deep generative model that can learn complicated image distributions. Using our Bayesian imaging approach with sophisticated data-driven priors, we can assess how visual features and uncertainty of reconstructed images change depending on the prior. In addition to simulated data, we image the real EHT M87* data and discuss how recovered features are influenced by the choice of prior. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priors

Authors: Zihui Wu, Yu Sun, Yifan Chen, Bingliang Zhang, Yisong Yue, Katherine L. Bouman

Abstract: Diffusion models (DMs) have recently shown outstanding capability in modeling complex image distributions, making them expressive image priors for solving Bayesian inverse problems. However, most existing DM-based methods rely on approximations in the generative process to be generic to different inverse problems, leading to inaccurate sample distributions that deviate from the target posterior de… ▽ More Diffusion models (DMs) have recently shown outstanding capability in modeling complex image distributions, making them expressive image priors for solving Bayesian inverse problems. However, most existing DM-based methods rely on approximations in the generative process to be generic to different inverse problems, leading to inaccurate sample distributions that deviate from the target posterior defined within the Bayesian framework. To harness the generative power of DMs while avoiding such approximations, we propose a Markov chain Monte Carlo algorithm that performs posterior sampling for general inverse problems by reducing it to sampling the posterior of a Gaussian denoising problem. Crucially, we leverage a general DM formulation as a unified interface that allows for rigorously solving the denoising problem with a range of state-of-the-art DMs. We demonstrate the effectiveness of the proposed method on six inverse problems (three linear and three nonlinear), including a real-world black hole imaging problem. Experimental results indicate that our proposed method offers more accurate reconstructions and posterior estimation compared to existing DM-based imaging inverse methods. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Score-Based Diffusion Models for Photoacoustic Tomography Image Reconstruction

Authors: Sreemanti Dey, Snigdha Saha, Berthy T. Feng, Manxiu Cui, Laure Delisle, Oscar Leong, Lihong V. Wang, Katherine L. Bouman

Abstract: Photoacoustic tomography (PAT) is a rapidly-evolving medical imaging modality that combines optical absorption contrast with ultrasound imaging depth. One challenge in PAT is image reconstruction with inadequate acoustic signals due to limited sensor coverage or due to the density of the transducer array. Such cases call for solving an ill-posed inverse reconstruction problem. In this work, we use… ▽ More Photoacoustic tomography (PAT) is a rapidly-evolving medical imaging modality that combines optical absorption contrast with ultrasound imaging depth. One challenge in PAT is image reconstruction with inadequate acoustic signals due to limited sensor coverage or due to the density of the transducer array. Such cases call for solving an ill-posed inverse reconstruction problem. In this work, we use score-based diffusion models to solve the inverse problem of reconstructing an image from limited PAT measurements. The proposed approach allows us to incorporate an expressive prior learned by a diffusion model on simulated vessel structures while still being robust to varying transducer sparsity conditions. △ Less

Submitted 30 March, 2024; originally announced April 2024.

Comments: 5 pages

Journal ref: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Korea, Republic of, 2024, pp. 2470-2474

Ordered magnetic fields around the 3C 84 central black hole

Authors: G. F. Paraschos, J. -Y. Kim, M. Wielgus, J. Röder, T. P. Krichbaum, E. Ros, I. Agudo, I. Myserlis, M. Moscibrodzka, E. Traianou, J. A. Zensus, L. Blackburn, C. -K. Chan, S. Issaoun, M. Janssen, M. D. Johnson, V. L. Fish, K. Akiyama, A. Alberdi, W. Alef, J. C. Algaba, R. Anantua, K. Asada, R. Azulay, U. Bach , et al. (258 additional authors not shown)

Abstract: 3C84 is a nearby radio source with a complex total intensity structure, showing linear polarisation and spectral patterns. A detailed investigation of the central engine region necessitates the use of VLBI above the hitherto available maximum frequency of 86GHz. Using ultrahigh resolution VLBI observations at the highest available frequency of 228GHz, we aim to directly detect compact structures a… ▽ More 3C84 is a nearby radio source with a complex total intensity structure, showing linear polarisation and spectral patterns. A detailed investigation of the central engine region necessitates the use of VLBI above the hitherto available maximum frequency of 86GHz. Using ultrahigh resolution VLBI observations at the highest available frequency of 228GHz, we aim to directly detect compact structures and understand the physical conditions in the compact region of 3C84. We used EHT 228GHz observations and, given the limited (u,v)-coverage, applied geometric model fitting to the data. We also employed quasi-simultaneously observed, multi-frequency VLBI data for the source in order to carry out a comprehensive analysis of the core structure. We report the detection of a highly ordered, strong magnetic field around the central, SMBH of 3C84. The brightness temperature analysis suggests that the system is in equipartition. We determined a turnover frequency of $ν_m=(113\pm4)$GHz, a corresponding synchrotron self-absorbed magnetic field of $B_{SSA}=(2.9\pm1.6)$G, and an equipartition magnetic field of $B_{eq}=(5.2\pm0.6)$G. Three components are resolved with the highest fractional polarisation detected for this object ($m_\textrm{net}=(17.0\pm3.9)$%). The positions of the components are compatible with those seen in low-frequency VLBI observations since 2017-2018. We report a steeply negative slope of the spectrum at 228GHz. We used these findings to test models of jet formation, propagation, and Faraday rotation in 3C84. The findings of our investigation into different flow geometries and black hole spins support an advection-dominated accretion flow in a magnetically arrested state around a rapidly rotating supermassive black hole as a model of the jet-launching system in the core of 3C84. However, systematic uncertainties due to the limited (u,v)-coverage, however, cannot be ignored. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: 15 pages, 6 figures, published in A&A

Journal ref: Issue: A&A Volume 682, February 2024; Article number: L3; Number of pages: 15

Filamentary structures as the origin of blazar jet radio variability

Authors: Antonio Fuentes, José L. Gómez, José M. Martí, Manel Perucho, Guang-Yao Zhao, Rocco Lico, Andrei P. Lobanov, Gabriele Bruni, Yuri Y. Kovalev, Andrew Chael, Kazunori Akiyama, Katherine L. Bouman, He Sun, Ilje Cho, Efthalia Traianou, Teresa Toscano, Rohan Dahale, Marianna Foschi, Leonid I. Gurvits, Svetlana Jorstad, Jae-Young Kim, Alan P. Marscher, Yosuke Mizuno, Eduardo Ros, Tuomas Savolainen

Abstract: Supermassive black holes at the centre of active galactic nuclei power some of the most luminous objects in the Universe. Typically, very long baseline interferometric (VLBI) observations of blazars have revealed only funnel-like morphologies with little information of the ejected plasma internal structure, or lacked the sufficient dynamic range to reconstruct the extended jet emission. Here we sh… ▽ More Supermassive black holes at the centre of active galactic nuclei power some of the most luminous objects in the Universe. Typically, very long baseline interferometric (VLBI) observations of blazars have revealed only funnel-like morphologies with little information of the ejected plasma internal structure, or lacked the sufficient dynamic range to reconstruct the extended jet emission. Here we show microarcsecond-scale angular resolution images of the blazar 3C 279 obtained at 22 GHz with the space VLBI mission RadioAstron, which allowed us to resolve the jet transversely and reveal several filaments produced by plasma instabilities in a kinetically dominated flow. Our high angular resolution and dynamic range image suggests that emission features traveling down the jet may manifest as a result of differential Doppler-boosting within the filaments, as opposed to the standard shock-in-jet model invoked to explain blazar jet radio variability. Moreover, we infer that the filaments in 3C 279 are possibly threaded by a helical magnetic field rotating clockwise, as seen in the direction of the flow motion, with an intrinsic helix pitch angle of ~45 degrees in a jet with a Lorentz factor of ~13 at the time of observation. △ Less

Submitted 3 November, 2023; originally announced November 2023.

Comments: 15 pages, 7 figures. Initial version of an article published in Nature Astronomy

Journal ref: Nature Astronomy 7 (2023) 1359

Provable Probabilistic Imaging using Score-Based Generative Priors

Authors: Yu Sun, Zihui Wu, Yifan Chen, Berthy T. Feng, Katherine L. Bouman

Abstract: Estimating high-quality images while also quantifying their uncertainty are two desired features in an image reconstruction algorithm for solving ill-posed inverse problems. In this paper, we propose plug-and-play Monte Carlo (PMC) as a principled framework for characterizing the space of possible solutions to a general inverse problem. PMC is able to incorporate expressive score-based generative… ▽ More Estimating high-quality images while also quantifying their uncertainty are two desired features in an image reconstruction algorithm for solving ill-posed inverse problems. In this paper, we propose plug-and-play Monte Carlo (PMC) as a principled framework for characterizing the space of possible solutions to a general inverse problem. PMC is able to incorporate expressive score-based generative priors for high-quality image reconstruction while also performing uncertainty quantification via posterior sampling. In particular, we introduce two PMC algorithms which can be viewed as the sampling analogues of the traditional plug-and-play priors (PnP) and regularization by denoising (RED) algorithms. We also establish a theoretical analysis for characterizing the convergence of the PMC algorithms. Our analysis provides non-asymptotic stationarity guarantees for both algorithms, even in the presence of non-log-concave likelihoods and imperfect score networks. We demonstrate the performance of the PMC algorithms on multiple representative inverse problems with both linear and nonlinear forward models. Experimental results show that PMC significantly improves reconstruction quality and enables high-fidelity uncertainty quantification. △ Less

Submitted 29 December, 2023; v1 submitted 16 October, 2023; originally announced October 2023.

Orbital Polarimetric Tomography of a Flare Near the Sagittarius A* Supermassive Black Hole

Authors: Aviad Levis, Andrew A. Chael, Katherine L. Bouman, Maciek Wielgus, Pratul P. Srinivasan

Abstract: The interaction between the supermassive black hole at the center of the Milky Way, Sagittarius A*, and its accretion disk occasionally produces high-energy flares seen in X-ray, infrared, and radio. One proposed mechanism that produces flares is the formation of compact, bright regions that appear within the accretion disk and close to the event horizon. Understanding these flares provides a wind… ▽ More The interaction between the supermassive black hole at the center of the Milky Way, Sagittarius A*, and its accretion disk occasionally produces high-energy flares seen in X-ray, infrared, and radio. One proposed mechanism that produces flares is the formation of compact, bright regions that appear within the accretion disk and close to the event horizon. Understanding these flares provides a window into accretion processes. Although sophisticated simulations predict the formation of these flares, their structure has yet to be recovered by observations. Here we show the first three-dimensional (3D) reconstruction of an emission flare recovered from ALMA light curves observed on April 11, 2017. Our recovery shows compact, bright regions at a distance of roughly six times the event horizon. Moreover, it suggests a clockwise rotation in a low-inclination orbital plane, consistent with prior studies by GRAVITY and EHT. To recover this emission structure, we solve an ill-posed tomography problem by integrating a neural 3D representation with a gravitational model for black holes. Although the recovery is subject to, and sometimes sensitive to, the model assumptions, under physically motivated choices, our results are stable, and our approach is successful on simulated data. △ Less

Submitted 16 April, 2024; v1 submitted 11 October, 2023; originally announced October 2023.

Single View Refractive Index Tomography with Neural Fields

Authors: Brandon Zhao, Aviad Levis, Liam Connor, Pratul P. Srinivasan, Katherine L. Bouman

Abstract: Refractive Index Tomography is the inverse problem of reconstructing the continuously-varying 3D refractive index in a scene using 2D projected image measurements. Although a purely refractive field is not directly visible, it bends light rays as they travel through space, thus providing a signal for reconstruction. The effects of such fields appear in many scientific computer vision settings, ran… ▽ More Refractive Index Tomography is the inverse problem of reconstructing the continuously-varying 3D refractive index in a scene using 2D projected image measurements. Although a purely refractive field is not directly visible, it bends light rays as they travel through space, thus providing a signal for reconstruction. The effects of such fields appear in many scientific computer vision settings, ranging from refraction due to transparent cells in microscopy to the lensing of distant galaxies caused by dark matter in astrophysics. Reconstructing these fields is particularly difficult due to the complex nonlinear effects of the refractive field on observed images. Furthermore, while standard 3D reconstruction and tomography settings typically have access to observations of the scene from many viewpoints, many refractive index tomography problem settings only have access to images observed from a single viewpoint. We introduce a method that leverages prior knowledge of light sources scattered throughout the refractive medium to help disambiguate the single-view refractive index tomography problem. We differentiably trace curved rays through a neural field representation of the refractive field, and optimize its parameters to best reproduce the observed image. We demonstrate the efficacy of our approach by reconstructing simulated refractive fields, analyze the effects of light source distribution on the recovered field, and test our method on a simulated dark matter mapping problem where we successfully recover the 3D refractive field caused by a realistic dark matter distribution. △ Less

Submitted 1 December, 2023; v1 submitted 8 September, 2023; originally announced September 2023.

Efficient Bayesian Computational Imaging with a Surrogate Score-Based Prior

Authors: Berthy T. Feng, Katherine L. Bouman

Abstract: We propose a surrogate function for efficient use of score-based priors for Bayesian inverse imaging. Recent work turned score-based diffusion models into probabilistic priors for solving ill-posed imaging problems by appealing to an ODE-based log-probability function. However, evaluating this function is computationally inefficient and inhibits posterior estimation of high-dimensional images. Our… ▽ More We propose a surrogate function for efficient use of score-based priors for Bayesian inverse imaging. Recent work turned score-based diffusion models into probabilistic priors for solving ill-posed imaging problems by appealing to an ODE-based log-probability function. However, evaluating this function is computationally inefficient and inhibits posterior estimation of high-dimensional images. Our proposed surrogate prior is based on the evidence lower-bound of a score-based diffusion model. We demonstrate the surrogate prior on variational inference for efficient approximate posterior sampling of large images. Compared to the exact prior in previous work, our surrogate prior accelerates optimization of the variational image distribution by at least two orders of magnitude. We also find that our principled approach achieves higher-fidelity images than non-Bayesian baselines that involve hyperparameter-tuning at inference. Our work establishes a practical path forward for using score-based diffusion models as general-purpose priors for imaging. △ Less

Submitted 5 September, 2023; originally announced September 2023.

A search for pulsars around Sgr A* in the first Event Horizon Telescope dataset

Authors: Pablo Torne, Kuo Liu, Ralph P. Eatough, Jompoj Wongphechauxsorn, James M. Cordes, Gregory Desvignes, Mariafelicia De Laurentis, Michael Kramer, Scott M. Ransom, Shami Chatterjee, Robert Wharton, Ramesh Karuppusamy, Lindy Blackburn, Michael Janssen, Chi-kwan Chan, Geoffrey B. Crew, Lynn D. Matthews, Ciriaco Goddi, Helge Rottmann, Jan Wagner, Salvador Sanchez, Ignacio Ruiz, Federico Abbate, Geoffrey C. Bower, Juan J. Salamanca , et al. (261 additional authors not shown)

Abstract: The Event Horizon Telescope (EHT) observed in 2017 the supermassive black hole at the center of the Milky Way, Sagittarius A* (Sgr A*), at a frequency of 228.1 GHz ($λ$=1.3 mm). The fundamental physics tests that even a single pulsar orbiting Sgr A* would enable motivate searching for pulsars in EHT datasets. The high observing frequency means that pulsars - which typically exhibit steep emission… ▽ More The Event Horizon Telescope (EHT) observed in 2017 the supermassive black hole at the center of the Milky Way, Sagittarius A* (Sgr A*), at a frequency of 228.1 GHz ($λ$=1.3 mm). The fundamental physics tests that even a single pulsar orbiting Sgr A* would enable motivate searching for pulsars in EHT datasets. The high observing frequency means that pulsars - which typically exhibit steep emission spectra - are expected to be very faint. However, it also negates pulse scattering, an effect that could hinder pulsar detections in the Galactic Center. Additionally, magnetars or a secondary inverse Compton emission could be stronger at millimeter wavelengths than at lower frequencies. We present a search for pulsars close to Sgr A* using the data from the three most-sensitive stations in the EHT 2017 campaign: the Atacama Large Millimeter/submillimeter Array, the Large Millimeter Telescope and the IRAM 30 m Telescope. We apply three detection methods based on Fourier-domain analysis, the Fast-Folding-Algorithm and single pulse search targeting both pulsars and burst-like transient emission; using the simultaneity of the observations to confirm potential candidates. No new pulsars or significant bursts were found. Being the first pulsar search ever carried out at such high radio frequencies, we detail our analysis methods and give a detailed estimation of the sensitivity of the search. We conclude that the EHT 2017 observations are only sensitive to a small fraction ($\lesssim$2.2%) of the pulsars that may exist close to Sgr A*, motivating further searches for fainter pulsars in the region. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Comments: 33 pages, 7 figures, 6 Tables. Accepted for publication in ApJ

Learning Task-Specific Strategies for Accelerated MRI

Authors: Zihui Wu, Tianwei Yin, Yu Sun, Robert Frost, Andre van der Kouwe, Adrian V. Dalca, Katherine L. Bouman

Abstract: Compressed sensing magnetic resonance imaging (CS-MRI) seeks to recover visual information from subsampled measurements for diagnostic tasks. Traditional CS-MRI methods often separately address measurement subsampling, image reconstruction, and task prediction, resulting in a suboptimal end-to-end performance. In this work, we propose TACKLE as a unified co-design framework for jointly optimizing… ▽ More Compressed sensing magnetic resonance imaging (CS-MRI) seeks to recover visual information from subsampled measurements for diagnostic tasks. Traditional CS-MRI methods often separately address measurement subsampling, image reconstruction, and task prediction, resulting in a suboptimal end-to-end performance. In this work, we propose TACKLE as a unified co-design framework for jointly optimizing subsampling, reconstruction, and prediction strategies for the performance on downstream tasks. The naïve approach of simply appending a task prediction module and training with a task-specific loss leads to suboptimal downstream performance. Instead, we develop a training procedure where a backbone architecture is first trained for a generic pre-training task (image reconstruction in our case), and then fine-tuned for different downstream tasks with a prediction head. Experimental results on multiple public MRI datasets show that TACKLE achieves an improved performance on various tasks over traditional CS-MRI methods. We also demonstrate that TACKLE is robust to distribution shifts by showing that it generalizes to a new dataset we experimentally collected using different acquisition setups from the training data. Without additional fine-tuning, TACKLE leads to both numerical and visual improvements compared to existing baselines. We have further implemented a learned 4$\times$-accelerated sequence on a Siemens 3T MRI Skyra scanner. Compared to the fully-sampling scan that takes 335 seconds, our optimized sequence only takes 84 seconds, achieving a four-fold time reduction as desired, while maintaining high performance. △ Less

Submitted 5 December, 2023; v1 submitted 24 April, 2023; originally announced April 2023.

Score-Based Diffusion Models as Principled Priors for Inverse Imaging

Authors: Berthy T. Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L. Bouman, William T. Freeman

Abstract: Priors are essential for reconstructing images from noisy and/or incomplete measurements. The choice of the prior determines both the quality and uncertainty of recovered images. We propose turning score-based diffusion models into principled image priors ("score-based priors") for analyzing a posterior of images given measurements. Previously, probabilistic priors were limited to handcrafted regu… ▽ More Priors are essential for reconstructing images from noisy and/or incomplete measurements. The choice of the prior determines both the quality and uncertainty of recovered images. We propose turning score-based diffusion models into principled image priors ("score-based priors") for analyzing a posterior of images given measurements. Previously, probabilistic priors were limited to handcrafted regularizers and simple distributions. In this work, we empirically validate the theoretically-proven probability function of a score-based diffusion model. We show how to sample from resulting posteriors by using this probability function for variational inference. Our results, including experiments on denoising, deblurring, and interferometric imaging, suggest that score-based priors enable principled inference with a sophisticated, data-driven image prior. △ Less

Submitted 28 August, 2023; v1 submitted 23 April, 2023; originally announced April 2023.

Comments: ICCV 2023

Key Science Goals for the Next-Generation Event Horizon Telescope

Authors: Michael D. Johnson, Kazunori Akiyama, Lindy Blackburn, Katherine L. Bouman, Avery E. Broderick, Vitor Cardoso, R. P. Fender, Christian M. Fromm, Peter Galison, José L. Gómez, Daryl Haggard, Matthew L. Lister, Andrei P. Lobanov, Sera Markoff, Ramesh Narayan, Priyamvada Natarajan, Tiffany Nichols, Dominic W. Pesce, Ziri Younsi, Andrew Chael, Koushik Chatterjee, Ryan Chaves, Juliusz Doboszewski, Richard Dodson, Sheperd S. Doeleman , et al. (20 additional authors not shown)

Abstract: The Event Horizon Telescope (EHT) has led to the first images of a supermassive black hole, revealing the central compact objects in the elliptical galaxy M87 and the Milky Way. Proposed upgrades to this array through the next-generation EHT (ngEHT) program would sharply improve the angular resolution, dynamic range, and temporal coverage of the existing EHT observations. These improvements will u… ▽ More The Event Horizon Telescope (EHT) has led to the first images of a supermassive black hole, revealing the central compact objects in the elliptical galaxy M87 and the Milky Way. Proposed upgrades to this array through the next-generation EHT (ngEHT) program would sharply improve the angular resolution, dynamic range, and temporal coverage of the existing EHT observations. These improvements will uniquely enable a wealth of transformative new discoveries related to black hole science, extending from event-horizon-scale studies of strong gravity to studies of explosive transients to the cosmological growth and influence of supermassive black holes. Here, we present the key science goals for the ngEHT and their associated instrument requirements, both of which have been formulated through a multi-year international effort involving hundreds of scientists worldwide. △ Less

Submitted 21 April, 2023; originally announced April 2023.

Comments: 32 pages, 11 figures, accepted for publication in a special issue of Galaxies on the ngEHT (https://www.mdpi.com/journal/galaxies/special_issues/ngEHT_blackholes)

Journal ref: Galaxies 2023, 11(3), 61

Discovering Structure From Corruption for Unsupervised Image Reconstruction

Authors: Oscar Leong, Angela F. Gao, He Sun, Katherine L. Bouman

Abstract: We consider solving ill-posed imaging inverse problems without access to an image prior or ground-truth examples. An overarching challenge in these inverse problems is that an infinite number of images, including many that are implausible, are consistent with the observed measurements. Thus, image priors are required to reduce the space of possible solutions to more desirable reconstructions. Howe… ▽ More We consider solving ill-posed imaging inverse problems without access to an image prior or ground-truth examples. An overarching challenge in these inverse problems is that an infinite number of images, including many that are implausible, are consistent with the observed measurements. Thus, image priors are required to reduce the space of possible solutions to more desirable reconstructions. However, in many applications it is difficult or potentially impossible to obtain example images to construct an image prior. Hence inaccurate priors are often used, which inevitably result in biased solutions. Rather than solving an inverse problem using priors that encode the spatial structure of any one image, we propose to solve a set of inverse problems jointly by incorporating prior constraints on the collective structure of the underlying images. The key assumption of our work is that the underlying images we aim to reconstruct share common, low-dimensional structure. We show that such a set of inverse problems can be solved simultaneously without the use of a spatial image prior by instead inferring a shared image generator with a low-dimensional latent space. The parameters of the generator and latent embeddings are found by maximizing a proxy for the Evidence Lower Bound (ELBO). Once identified, the generator and latent embeddings can be combined to provide reconstructed images for each inverse problem. The framework we propose can handle general forward model corruptions, and we show that measurements derived from only a small number of ground-truth images ($\leqslant 150$) are sufficient for image reconstruction. We demonstrate our approach on a variety of convex and non-convex inverse problems, including denoising, phase retrieval, and black hole video reconstruction. △ Less

Submitted 1 November, 2023; v1 submitted 11 April, 2023; originally announced April 2023.

Comments: Extended version of arXiv:2303.12217

Image Reconstruction without Explicit Priors

Authors: Angela F. Gao, Oscar Leong, He Sun, Katherine L. Bouman

Abstract: We consider solving ill-posed imaging inverse problems without access to an explicit image prior or ground-truth examples. An overarching challenge in inverse problems is that there are many undesired images that fit to the observed measurements, thus requiring image priors to constrain the space of possible solutions to more plausible reconstructions. However, in many applications it is difficult… ▽ More We consider solving ill-posed imaging inverse problems without access to an explicit image prior or ground-truth examples. An overarching challenge in inverse problems is that there are many undesired images that fit to the observed measurements, thus requiring image priors to constrain the space of possible solutions to more plausible reconstructions. However, in many applications it is difficult or potentially impossible to obtain ground-truth images to learn an image prior. Thus, inaccurate priors are often used, which inevitably result in biased solutions. Rather than solving an inverse problem using priors that encode the explicit structure of any one image, we propose to solve a set of inverse problems jointly by incorporating prior constraints on the collective structure of the underlying images.The key assumption of our work is that the ground-truth images we aim to reconstruct share common, low-dimensional structure. We show that such a set of inverse problems can be solved simultaneously by learning a shared image generator with a low-dimensional latent space. The parameters of the generator and latent embedding are learned by maximizing a proxy for the Evidence Lower Bound (ELBO). Once learned, the generator and latent embeddings can be combined to provide reconstructions for each inverse problem. The framework we propose can handle general forward model corruptions, and we show that measurements derived from only a few ground-truth images (O(10)) are sufficient for image reconstruction without explicit priors. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Comments: ICASSP 2023

Comparison of Polarized Radiative Transfer Codes used by the EHT Collaboration

Authors: Ben S. Prather, Jason Dexter, Monika Moscibrodzka, Hung-Yi Pu, Thomas Bronzwaer, Jordy Davelaar, Ziri Younsi, Charles F. Gammie, Roman Gold, George N. Wong, Kazunori Akiyama, Antxon Alberdi, Walter Alef, Juan Carlos Algaba, Richard Anantua, Keiichi Asada, Rebecca Azulay, Uwe Bach, Anne-Kathrin Baczko, David Ball, Mislav Baloković, John Barrett, Michi Bauböck, Bradford A. Benson, Dan Bintley , et al. (248 additional authors not shown)

Abstract: Interpretation of resolved polarized images of black holes by the Event Horizon Telescope (EHT) requires predictions of the polarized emission observable by an Earth-based instrument for a particular model of the black hole accretion system. Such predictions are generated by general relativistic radiative transfer (GRRT) codes, which integrate the equations of polarized radiative transfer in curve… ▽ More Interpretation of resolved polarized images of black holes by the Event Horizon Telescope (EHT) requires predictions of the polarized emission observable by an Earth-based instrument for a particular model of the black hole accretion system. Such predictions are generated by general relativistic radiative transfer (GRRT) codes, which integrate the equations of polarized radiative transfer in curved spacetime. A selection of ray-tracing GRRT codes used within the EHT collaboration is evaluated for accuracy and consistency in producing a selection of test images, demonstrating that the various methods and implementations of radiative transfer calculations are highly consistent. When imaging an analytic accretion model, we find that all codes produce images similar within a pixel-wise normalized mean squared error (NMSE) of 0.012 in the worst case. When imaging a snapshot from a cell-based magnetohydrodynamic simulation, we find all test images to be similar within NMSEs of 0.02, 0.04, 0.04, and 0.12 in Stokes I, Q, U , and V respectively. We additionally find the values of several image metrics relevant to published EHT results to be in agreement to much better precision than measurement uncertainties. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Comments: Accepted for publication in ApJ

The Event Horizon Telescope Image of the Quasar NRAO 530

Authors: Svetlana Jorstad, Maciek Wielgus, Rocco Lico, Sara Issaoun, Avery E. Broderick, Dominic W. Pesce, Jun Liu, Guang-Yao Zhao, Thomas P. Krichbaum, Lindy Blackburn, Chi-Kwan Chan, Michael Janssen, Venkatessh Ramakrishnan, Kazunori Akiyama, Antxon Alberdi, Juan Carlos Algaba, Katherine L. Bouman, Ilje Cho, Antonio Fuentes, Jose L. Gomez, Mark Gurwell, Michael D. Johnson, Jae-Young Kim, Ru-Sen Lu, Ivan Marti-Vidal , et al. (5 additional authors not shown)

Abstract: We report on the observations of the quasar NRAO 530 with the Event Horizon Telescope (EHT) on 2017 April 5-7, when NRAO 530 was used as a calibrator for the EHT observations of Sagittarius A*. At z=0.902 this is the most distant object imaged by the EHT so far. We reconstruct the first images of the source at 230 GHz, at an unprecedented angular resolution of $\sim$ 20 $μ$as, both in total intens… ▽ More We report on the observations of the quasar NRAO 530 with the Event Horizon Telescope (EHT) on 2017 April 5-7, when NRAO 530 was used as a calibrator for the EHT observations of Sagittarius A*. At z=0.902 this is the most distant object imaged by the EHT so far. We reconstruct the first images of the source at 230 GHz, at an unprecedented angular resolution of $\sim$ 20 $μ$as, both in total intensity and in linear polarization. We do not detect source variability, allowing us to represent the whole data set with static images. The images reveal a bright feature located on the southern end of the jet, which we associate with the core. The feature is linearly polarized, with a fractional polarization of $\sim$5-8% and has a sub-structure consisting of two components. Their observed brightness temperature suggests that the energy density of the jet is dominated by the magnetic field. The jet extends over 60 $μ$as along a position angle PA$\sim -$28$^\circ$. It includes two features with orthogonal directions of polarization (electric vector position angle, EVPA), parallel and perpendicular to the jet axis, consistent with a helical structure of the magnetic field in the jet. The outermost feature has a particularly high degree of linear polarization, suggestive of a nearly uniform magnetic field. Future EHT observations will probe the variability of the jet structure on $μ$as scales, while simultaneous multi-wavelength monitoring will provide insight into the high energy emission origin. △ Less

Submitted 9 February, 2023; originally announced February 2023.

Journal ref: The Astrophysical Journal 943:170 (2023)

Gravitationally Lensed Black Hole Emission Tomography

Authors: Aviad Levis, Pratul P. Srinivasan, Andrew A. Chael, Ren Ng, Katherine L. Bouman

Abstract: Measurements from the Event Horizon Telescope enabled the visualization of light emission around a black hole for the first time. So far, these measurements have been used to recover a 2D image under the assumption that the emission field is static over the period of acquisition. In this work, we propose BH-NeRF, a novel tomography approach that leverages gravitational lensing to recover the conti… ▽ More Measurements from the Event Horizon Telescope enabled the visualization of light emission around a black hole for the first time. So far, these measurements have been used to recover a 2D image under the assumption that the emission field is static over the period of acquisition. In this work, we propose BH-NeRF, a novel tomography approach that leverages gravitational lensing to recover the continuous 3D emission field near a black hole. Compared to other 3D reconstruction or tomography settings, this task poses two significant challenges: first, rays near black holes follow curved paths dictated by general relativity, and second, we only observe measurements from a single viewpoint. Our method captures the unknown emission field using a continuous volumetric function parameterized by a coordinate-based neural network, and uses knowledge of Keplerian orbital dynamics to establish correspondence between 3D points over time. Together, these enable BH-NeRF to recover accurate 3D emission fields, even in challenging situations with sparse measurements and uncertain orbital dynamics. This work takes the first steps in showing how future measurements from the Event Horizon Telescope could be used to recover evolving 3D emission around the supermassive black hole in our Galactic center. △ Less

Submitted 7 April, 2022; originally announced April 2022.

Comments: To appear in the IEEE Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), 2022. Supplemental material including accompanying pdf, code, and video highlight can be found in the project page: http://imaging.cms.caltech.edu/bhnerf/

Self-Supervised Online Learning for Safety-Critical Control using Stereo Vision

Authors: Ryan K. Cosner, Ivan D. Jimenez Rodriguez, Tamas G. Molnar, Wyatt Ubellacker, Yisong Yue, Aaron D. Ames, Katherine L. Bouman

Abstract: With the increasing prevalence of complex vision-based sensing methods for use in obstacle identification and state estimation, characterizing environment-dependent measurement errors has become a difficult and essential part of modern robotics. This paper presents a self-supervised learning approach to safety-critical control. In particular, the uncertainty associated with stereo vision is estima… ▽ More With the increasing prevalence of complex vision-based sensing methods for use in obstacle identification and state estimation, characterizing environment-dependent measurement errors has become a difficult and essential part of modern robotics. This paper presents a self-supervised learning approach to safety-critical control. In particular, the uncertainty associated with stereo vision is estimated, and adapted online to new visual environments, wherein this estimate is leveraged in a safety-critical controller in a robust fashion. To this end, we propose an algorithm that exploits the structure of stereo-vision to learn an uncertainty estimate without the need for ground-truth data. We then robustify existing Control Barrier Function-based controllers to provide safety in the presence of this uncertainty estimate. We demonstrate the efficacy of our method on a quadrupedal robot in a variety of environments. When not using our method safety is violated. With offline training alone we observe the robot is safe, but overly-conservative. With our online method the quadruped remains safe and conservatism is reduced. △ Less

Submitted 2 March, 2022; originally announced March 2022.

Comments: 7 pages, 4 figures, conference publication at ICRA 2022

MeqSilhouette v2: Spectrally-resolved polarimetric synthetic data generation for the Event Horizon Telescope

Authors: Iniyan Natarajan, Roger Deane, Iván Martí-Vidal, Freek Roelofs, Michael Janssen, Maciek Wielgus, Lindy Blackburn, Tariq Blecher, Simon Perkins, Oleg Smirnov, Jordy Davelaar, Monika Moscibrodzka, Andrew Chael, Katherine L. Bouman, Jae-Young Kim, Gianni Bernardi, Ilse van Bemmel, Heino Falcke, Feryal Özel, Dimitrios Psaltis

Abstract: We present MeqSilhouette v2.0 (MeqSv2), a fully polarimetric, time-and frequency-resolved synthetic data generation software for simulating millimetre (mm) wavelength very long baseline interferometry (VLBI) observations with heterogeneous arrays. Synthetic data are a critical component in understanding real observations, testing calibration and imaging algorithms, and predicting performance metri… ▽ More We present MeqSilhouette v2.0 (MeqSv2), a fully polarimetric, time-and frequency-resolved synthetic data generation software for simulating millimetre (mm) wavelength very long baseline interferometry (VLBI) observations with heterogeneous arrays. Synthetic data are a critical component in understanding real observations, testing calibration and imaging algorithms, and predicting performance metrics of existing or proposed sites. MeqSv2 applies physics-based instrumental and atmospheric signal corruptions constrained by empirically-derived site and station parameters to the data. The new version is capable of applying instrumental polarization effects and various other spectrally-resolved effects using the Radio Interferometry Measurement Equation (RIME) formalism and produces synthetic data compatible with calibration pipelines designed to process real data. We demonstrate the various corruption capabilities of MeqSv2 using different arrays, with a focus on the effect of complex bandpass gains on closure quantities for the EHT at 230 GHz. We validate the frequency-dependent polarization leakage implementation by performing polarization self-calibration of synthetic EHT data using PolSolve. We also note the potential applications for cm-wavelength VLBI array analysis and design and future directions. △ Less

Submitted 23 February, 2022; originally announced February 2022.

Comments: accepted for publication in MNRAS

alpha-Deep Probabilistic Inference (alpha-DPI): efficient uncertainty quantification from exoplanet astrometry to black hole feature extraction

Authors: He Sun, Katherine L. Bouman, Paul Tiede, Jason J. Wang, Sarah Blunt, Dimitri Mawet

Abstract: Inference is crucial in modern astronomical research, where hidden astrophysical features and patterns are often estimated from indirect and noisy measurements. Inferring the posterior of hidden features, conditioned on the observed measurements, is essential for understanding the uncertainty of results and downstream scientific interpretations. Traditional approaches for posterior estimation incl… ▽ More Inference is crucial in modern astronomical research, where hidden astrophysical features and patterns are often estimated from indirect and noisy measurements. Inferring the posterior of hidden features, conditioned on the observed measurements, is essential for understanding the uncertainty of results and downstream scientific interpretations. Traditional approaches for posterior estimation include sampling-based methods and variational inference. However, sampling-based methods are typically slow for high-dimensional inverse problems, while variational inference often lacks estimation accuracy. In this paper, we propose alpha-DPI, a deep learning framework that first learns an approximate posterior using alpha-divergence variational inference paired with a generative neural network, and then produces more accurate posterior samples through importance re-weighting of the network samples. It inherits strengths from both sampling and variational inference methods: it is fast, accurate, and scalable to high-dimensional problems. We apply our approach to two high-impact astronomical inference problems using real data: exoplanet astrometry and black hole feature extraction. △ Less

Submitted 10 April, 2022; v1 submitted 20 January, 2022; originally announced January 2022.

Collimation of the relativistic jet in the quasar 3C 273

Authors: Hiroki Okino, Kazunori Akiyama, Keiichi Asada, José L. Gómez, Kazuhiro Hada, Mareki Honma, Thomas P. Krichbaum, Motoki Kino, Hiroshi Nagai, Uwe Bach, Lindy Blackburn, Katherine L. Bouman, Andrew Chael, Geoffrey B. Crew, Sheperd S. Doeleman, Vincent L. Fish, Ciriaco Goddi, Sara Issaoun, Michael D. Johnson, Svetlana Jorstad, Shoko Koyama, Colin J. Lonsdale, Ru-sen Lu, Ivan Martí-Vidal, Lynn D. Matthews , et al. (10 additional authors not shown)

Abstract: The collimation of relativistic jets launched from the vicinity of supermassive black holes (SMBHs) at the centers of active galactic nuclei (AGN) is one of the key questions to understand the nature of AGN jets. However, little is known about the detailed jet structure for AGN like quasars since very high angular resolutions are required to resolve these objects. We present very long baseline int… ▽ More The collimation of relativistic jets launched from the vicinity of supermassive black holes (SMBHs) at the centers of active galactic nuclei (AGN) is one of the key questions to understand the nature of AGN jets. However, little is known about the detailed jet structure for AGN like quasars since very high angular resolutions are required to resolve these objects. We present very long baseline interferometry (VLBI) observations of the archetypical quasar 3C 273 at 86 GHz, performed with the Global Millimeter VLBI Array, for the first time including the Atacama Large Millimeter/submillimeter Array. Our observations achieve a high angular resolution down to $\sim$60 ${\rm μ}$as, resolving the innermost part of the jet ever on scales of $\sim 10^5$ Schwarzschild radii. Our observations, including close-in-time High Sensitivity Array observations of 3C 273 at 15, 22, and 43 GHz, suggest that the inner jet collimates parabolically, while the outer jet expands conically, similar to jets from other nearby low luminosity AGN. We discovered the jet collimation break around $10^{7}$ Schwarzschild radii, providing the first compelling evidence for structural transition in a quasar jet. The location of the collimation break for 3C 273 is farther downstream the sphere of gravitational influence (SGI) from the central SMBH. With the results for other AGN jets, our results show that the end of the collimation zone in AGN jets is governed not only by the SGI of the SMBH but also by the more diverse properties of the central nuclei. △ Less

Submitted 7 October, 2022; v1 submitted 22 December, 2021; originally announced December 2021.

Comments: 22 pages, 13 figures, accepted for publication in ApJ

Event Horizon Telescope observations of the jet launching and collimation in Centaurus A

Authors: Michael Janssen, Heino Falcke, Matthias Kadler, Eduardo Ros, Maciek Wielgus, Kazunori Akiyama, Mislav Baloković, Lindy Blackburn, Katherine L. Bouman, Andrew Chael, Chi-kwan Chan, Koushik Chatterjee, Jordy Davelaar, Philip G. Edwards, Christian M. Fromm, José L. Gómez, Ciriaco Goddi, Sara Issaoun, Michael D. Johnson, Junhan Kim, Jun Yi Koay, Thomas P. Krichbaum, Jun Liu, Elisabetta Liuzzo, Sera Markoff , et al. (215 additional authors not shown)

Abstract: Very-long-baseline interferometry (VLBI) observations of active galactic nuclei at millimeter wavelengths have the power to reveal the launching and initial collimation region of extragalactic radio jets, down to $10-100$ gravitational radii ($r_g=GM/c^2$) scales in nearby sources. Centaurus A is the closest radio-loud source to Earth. It bridges the gap in mass and accretion rate between the supe… ▽ More Very-long-baseline interferometry (VLBI) observations of active galactic nuclei at millimeter wavelengths have the power to reveal the launching and initial collimation region of extragalactic radio jets, down to $10-100$ gravitational radii ($r_g=GM/c^2$) scales in nearby sources. Centaurus A is the closest radio-loud source to Earth. It bridges the gap in mass and accretion rate between the supermassive black holes (SMBHs) in Messier 87 and our galactic center. A large southern declination of $-43^{\circ}$ has however prevented VLBI imaging of Centaurus A below $λ1$cm thus far. Here, we show the millimeter VLBI image of the source, which we obtained with the Event Horizon Telescope at $228$GHz. Compared to previous observations, we image Centaurus A's jet at a tenfold higher frequency and sixteen times sharper resolution and thereby probe sub-lightday structures. We reveal a highly-collimated, asymmetrically edge-brightened jet as well as the fainter counterjet. We find that Centaurus A's source structure resembles the jet in Messier 87 on ${\sim}500r_g$ scales remarkably well. Furthermore, we identify the location of Centaurus A's SMBH with respect to its resolved jet core at $λ1.3$mm and conclude that the source's event horizon shadow should be visible at THz frequencies. This location further supports the universal scale invariance of black holes over a wide range of masses. △ Less

Submitted 5 November, 2021; originally announced November 2021.

Comments: 27 pages, 9 figures. This is a post-peer-review, pre-copyedit version of an article published in Nature Astronomy. The final authenticated version is available online at: https://doi.org/10.1038/s41550-021-01417-w

Journal ref: Nature Astronomy, July 2021, Volume 5, p. 1017-1028

Deep Radio Interferometric Imaging with POLISH: DSA-2000 and weak lensing

Authors: Liam Connor, Katherine L. Bouman, Vikram Ravi, Gregg Hallinan

Abstract: Radio interferometry allows astronomers to probe small spatial scales that are often inaccessible with single-dish instruments. However, recovering the radio sky from an interferometer is an ill-posed deconvolution problem that astronomers have worked on for half a century. More challenging still is achieving resolution below the array's diffraction limit, known as super-resolution imaging. To thi… ▽ More Radio interferometry allows astronomers to probe small spatial scales that are often inaccessible with single-dish instruments. However, recovering the radio sky from an interferometer is an ill-posed deconvolution problem that astronomers have worked on for half a century. More challenging still is achieving resolution below the array's diffraction limit, known as super-resolution imaging. To this end, we have developed a new learning-based approach for radio interferometric imaging, leveraging recent advances in the classical computer vision problems of single-image super-resolution (SISR) and deconvolution. We have developed and trained a high dynamic range residual neural network to learn the mapping between the dirty image and the true radio sky. We call this procedure POLISH, in contrast to the traditional CLEAN algorithm. The feed forward nature of learning-based approaches like POLISH is critical for analyzing data from the upcoming Deep Synoptic Array (DSA-2000). We show that POLISH achieves super-resolution, and we demonstrate its ability to deconvolve real observations from the Very Large Array (VLA). Super-resolution on DSA-2000 will allow us to measure the shapes and orientations of several hundred million star forming radio galaxies (SFGs), making it a powerful cosmological weak lensing survey and probe of dark energy. We forecast its ability to constrain the lensing power spectrum, finding that it will be complementary to next-generation optical surveys such as Euclid. △ Less

Submitted 6 May, 2022; v1 submitted 5 November, 2021; originally announced November 2021.

The Variability of the Black-Hole Image in M87 at the Dynamical Time Scale

Authors: Kaushik Satapathy, Dimitrios Psaltis, Feryal Ozel, Lia Medeiros, Sean T. Dougall, Chi-kwan Chan, Maciek Wielgus, Ben S. Prather, George N. Wong, Charles F. Gammie, Kazunori Akiyama, Antxon Alberdi, Walter Alef, Juan Carlos Algaba, Richard Anantua, Keiichi Asada, Rebecca Azulay, Anne-Kathrin Baczko, David R. Ball, Mislav Baloković, John Barrett, Bradford A. Benson, Dan Bintley, Lindy Blackburn, Raymond Blundell , et al. (213 additional authors not shown)

Abstract: The black-hole images obtained with the Event Horizon Telescope (EHT) are expected to be variable at the dynamical timescale near their horizons. For the black hole at the center of the M87 galaxy, this timescale (5-61 days) is comparable to the 6-day extent of the 2017 EHT observations. Closure phases along baseline triangles are robust interferometric observables that are sensitive to the expect… ▽ More The black-hole images obtained with the Event Horizon Telescope (EHT) are expected to be variable at the dynamical timescale near their horizons. For the black hole at the center of the M87 galaxy, this timescale (5-61 days) is comparable to the 6-day extent of the 2017 EHT observations. Closure phases along baseline triangles are robust interferometric observables that are sensitive to the expected structural changes of the images but are free of station-based atmospheric and instrumental errors. We explored the day-to-day variability in closure phase measurements on all six linearly independent non-trivial baseline triangles that can be formed from the 2017 observations. We showed that three triangles exhibit very low day-to-day variability, with a dispersion of $\sim3-5^\circ$. The only triangles that exhibit substantially higher variability ($\sim90-180^\circ$) are the ones with baselines that cross visibility amplitude minima on the $u-v$ plane, as expected from theoretical modeling. We used two sets of General Relativistic magnetohydrodynamic simulations to explore the dependence of the predicted variability on various black-hole and accretion-flow parameters. We found that changing the magnetic field configuration, electron temperature model, or black-hole spin has a marginal effect on the model consistency with the observed level of variability. On the other hand, the most discriminating image characteristic of models is the fractional width of the bright ring of emission. Models that best reproduce the observed small level of variability are characterized by thin ring-like images with structures dominated by gravitational lensing effects and thus least affected by turbulence in the accreting plasmas. △ Less

Submitted 1 November, 2021; originally announced November 2021.

Comments: Accepted for Publication in ApJ

Unsupervised learning of MRI tissue properties using MRI physics models

Authors: Divya Varadarajan, Katherine L. Bouman, Andre van der Kouwe, Bruce Fischl, Adrian V. Dalca

Abstract: In neuroimaging, MRI tissue properties characterize underlying neurobiology, provide quantitative biomarkers for neurological disease detection and analysis, and can be used to synthesize arbitrary MRI contrasts. Estimating tissue properties from a single scan session using a protocol available on all clinical scanners promises to reduce scan time and cost, enable quantitative analysis in routine… ▽ More In neuroimaging, MRI tissue properties characterize underlying neurobiology, provide quantitative biomarkers for neurological disease detection and analysis, and can be used to synthesize arbitrary MRI contrasts. Estimating tissue properties from a single scan session using a protocol available on all clinical scanners promises to reduce scan time and cost, enable quantitative analysis in routine clinical scans and provide scan-independent biomarkers of disease. However, existing tissue properties estimation methods - most often $\mathbf{T_1}$ relaxation, $\mathbf{T_2^*}$ relaxation, and proton density ($\mathbf{PD}$) - require data from multiple scan sessions and cannot estimate all properties from a single clinically available MRI protocol such as the multiecho MRI scan. In addition, the widespread use of non-standard acquisition parameters across clinical imaging sites require estimation methods that can generalize across varying scanner parameters. However, existing learning methods are acquisition protocol specific and cannot estimate from heterogenous clinical data from different imaging sites. In this work we propose an unsupervised deep-learning strategy that employs MRI physics to estimate all three tissue properties from a single multiecho MRI scan session, and generalizes across varying acquisition parameters. The proposed strategy optimizes accurate synthesis of new MRI contrasts from estimated latent tissue properties, enabling unsupervised training, we also employ random acquisition parameters during training to achieve acquisition generalization. We provide the first demonstration of estimating all tissue properties from a single multiecho scan session. We demonstrate improved accuracy and generalizability for tissue property estimation and MRI synthesis. △ Less

Submitted 6 July, 2021; originally announced July 2021.

Comments: 11 Pages, 9 figures

Constraints on black-hole charges with the 2017 EHT observations of M87*

Authors: Prashant Kocherlakota, Luciano Rezzolla, Heino Falcke, Christian M. Fromm, Michael Kramer, Yosuke Mizuno, Antonios Nathanail, Hector Olivares, Ziri Younsi, Kazunori Akiyama, Antxon Alberdi, Walter Alef, Juan Carlos Algaba, Richard Anantua, Keiichi Asada, Rebecca Azulay, Anne-Kathrin Baczko, David Ball, Mislav Balokovic, John Barrett, Bradford A. Benson, Dan Bintley, Lindy Blackburn, Raymond Blundell, Wilfred Boland , et al. (212 additional authors not shown)

Abstract: Our understanding of strong gravity near supermassive compact objects has recently improved thanks to the measurements made by the Event Horizon Telescope (EHT). We use here the M87* shadow size to infer constraints on the physical charges of a large variety of nonrotating or rotating black holes. For example, we show that the quality of the measurements is already sufficient to rule out that M87*… ▽ More Our understanding of strong gravity near supermassive compact objects has recently improved thanks to the measurements made by the Event Horizon Telescope (EHT). We use here the M87* shadow size to infer constraints on the physical charges of a large variety of nonrotating or rotating black holes. For example, we show that the quality of the measurements is already sufficient to rule out that M87* is a highly charged dilaton black hole. Similarly, when considering black holes with two physical and independent charges, we are able to exclude considerable regions of the space of parameters for the doubly-charged dilaton and the Sen black holes. △ Less

Submitted 19 May, 2021; originally announced May 2021.

Comments: 15 pages, 4 figures, published in PRD on May 19

End-to-End Sequential Sampling and Reconstruction for MRI

Authors: Tianwei Yin, Zihui Wu, He Sun, Adrian V. Dalca, Yisong Yue, Katherine L. Bouman

Abstract: Accelerated MRI shortens acquisition time by subsampling in the measurement $κ$-space. Recovering a high-fidelity anatomical image from subsampled measurements requires close cooperation between two components: (1) a sampler that chooses the subsampling pattern and (2) a reconstructor that recovers images from incomplete measurements. In this paper, we leverage the sequential nature of MRI measure… ▽ More Accelerated MRI shortens acquisition time by subsampling in the measurement $κ$-space. Recovering a high-fidelity anatomical image from subsampled measurements requires close cooperation between two components: (1) a sampler that chooses the subsampling pattern and (2) a reconstructor that recovers images from incomplete measurements. In this paper, we leverage the sequential nature of MRI measurements, and propose a fully differentiable framework that jointly learns a sequential sampling policy simultaneously with a reconstruction strategy. This co-designed framework is able to adapt during acquisition in order to capture the most informative measurements for a particular target. Experimental results on the fastMRI knee dataset demonstrate that the proposed approach successfully utilizes intermediate information during the sampling process to boost reconstruction performance. In particular, our proposed method can outperform the current state-of-the-art learned $κ$-space sampling baseline on over 96% of test samples. We also investigate the individual and collective benefits of the sequential sampling and co-design strategies. △ Less

Submitted 16 July, 2022; v1 submitted 13 May, 2021; originally announced May 2021.

Comments: Code and supplementary materials are available at http://imaging.cms.caltech.edu/seq-mri

Journal ref: Proceedings of Machine Learning for Health, PMLR 158:261-281, 2021

The Polarized Image of a Synchrotron Emitting Ring of Gas Orbiting a Black Hole

Authors: Ramesh Narayan, Daniel C. M. Palumbo, Michael D. Johnson, Zachary Gelles, Elizabeth Himwich, Dominic O. Chang, Angelo Ricarte, Jason Dexter, Charles F. Gammie, Andrew A. Chael, The Event Horizon Telescope Collaboration, :, Kazunori Akiyama, Antxon Alberdi, Walter Alef, Juan Carlos Algaba, Richard Anantua, Keiichi Asada, Rebecca Azulay, Anne-Kathrin Baczko, David Ball, Mislav Balokovic, John Barrett, Bradford A. Benson, Dan Bintley , et al. (215 additional authors not shown)

Abstract: Synchrotron radiation from hot gas near a black hole results in a polarized image. The image polarization is determined by effects including the orientation of the magnetic field in the emitting region, relativistic motion of the gas, strong gravitational lensing by the black hole, and parallel transport in the curved spacetime. We explore these effects using a simple model of an axisymmetric, equ… ▽ More Synchrotron radiation from hot gas near a black hole results in a polarized image. The image polarization is determined by effects including the orientation of the magnetic field in the emitting region, relativistic motion of the gas, strong gravitational lensing by the black hole, and parallel transport in the curved spacetime. We explore these effects using a simple model of an axisymmetric, equatorial accretion disk around a Schwarzschild black hole. By using an approximate expression for the null geodesics derived by Beloborodov (2002) and conservation of the Walker-Penrose constant, we provide analytic estimates for the image polarization. We test this model using currently favored general relativistic magnetohydrodynamic simulations of M87*, using ring parameters given by the simulations. For a subset of these with modest Faraday effects, we show that the ring model broadly reproduces the polarimetric image morphology. Our model also predicts the polarization evolution for compact flaring regions, such as those observed from Sgr A* with GRAVITY. With suitably chosen parameters, our simple model can reproduce the EVPA pattern and relative polarized intensity in Event Horizon Telescope images of M87*. Under the physically motivated assumption that the magnetic field trails the fluid velocity, this comparison is consistent with the clockwise rotation inferred from total intensity images. △ Less

Submitted 13 May, 2021; v1 submitted 4 May, 2021; originally announced May 2021.

Comments: 29 pages, 11 figures, published in ApJ on May 3

Journal ref: ApJ 912 35 (2021)

Measurement-Robust Control Barrier Functions: Certainty in Safety with Uncertainty in State

Authors: Ryan K. Cosner, Andrew W. Singletary, Andrew J. Taylor, Tamas G. Molnar, Katherine L. Bouman, Aaron D. Ames

Abstract: The increasing complexity of modern robotic systems and the environments they operate in necessitates the formal consideration of safety in the presence of imperfect measurements. In this paper we propose a rigorous framework for safety-critical control of systems with erroneous state estimates. We develop this framework by leveraging Control Barrier Functions (CBFs) and unifying the method of Bac… ▽ More The increasing complexity of modern robotic systems and the environments they operate in necessitates the formal consideration of safety in the presence of imperfect measurements. In this paper we propose a rigorous framework for safety-critical control of systems with erroneous state estimates. We develop this framework by leveraging Control Barrier Functions (CBFs) and unifying the method of Backup Sets for synthesizing control invariant sets with robustness requirements -- the end result is the synthesis of Measurement-Robust Control Barrier Functions (MR-CBFs). This provides theoretical guarantees on safe behavior in the presence of imperfect measurements and improved robustness over standard CBF approaches. We demonstrate the efficacy of this framework both in simulation and experimentally on a Segway platform using an onboard stereo-vision camera for state estimation. △ Less

Submitted 28 April, 2021; originally announced April 2021.

Comments: 6 pages, 4 figures

Visual Vibration Tomography: Estimating Interior Material Properties from Monocular Video

Authors: Berthy T. Feng, Alexander C. Ogren, Chiara Daraio, Katherine L. Bouman

Abstract: An object's interior material properties, while invisible to the human eye, determine motion observed on its surface. We propose an approach that estimates heterogeneous material properties of an object from a monocular video of its surface vibrations. Specifically, we show how to estimate Young's modulus and density throughout a 3D object with known geometry. Knowledge of how these values change… ▽ More An object's interior material properties, while invisible to the human eye, determine motion observed on its surface. We propose an approach that estimates heterogeneous material properties of an object from a monocular video of its surface vibrations. Specifically, we show how to estimate Young's modulus and density throughout a 3D object with known geometry. Knowledge of how these values change across the object is useful for simulating its motion and characterizing any defects. Traditional non-destructive testing approaches, which often require expensive instruments, generally estimate only homogenized material properties or simply identify the presence of defects. In contrast, our approach leverages monocular video to (1) identify image-space modes from an object's sub-pixel motion, and (2) directly infer spatially-varying Young's modulus and density values from the observed modes. We demonstrate our approach on both simulated and real videos. △ Less

Submitted 23 April, 2023; v1 submitted 6 April, 2021; originally announced April 2021.

Wind speed inference from environmental flow-structure interactions

Authors: Jennifer L. Cardona, Katherine L. Bouman, John O. Dabiri

Abstract: This study aims to leverage the relationship between fluid dynamic loading and resulting structural deformation to infer the incident flow speed from measurements of time-dependent structure kinematics. Wind tunnel studies are performed on cantilevered cylinders and trees. Tip deflections of the wind-loaded structures are captured in time series data, and a physical model of the relationship betwe… ▽ More This study aims to leverage the relationship between fluid dynamic loading and resulting structural deformation to infer the incident flow speed from measurements of time-dependent structure kinematics. Wind tunnel studies are performed on cantilevered cylinders and trees. Tip deflections of the wind-loaded structures are captured in time series data, and a physical model of the relationship between force and deflection is applied to calculate the instantaneous wind speed normalized with respect to a known reference wind speed. Wind speeds inferred from visual measurements showed consistent agreement with ground truth anemometer measurements for different cylinder and tree configurations. These results suggest an approach for non-intrusive, quantitative flow velocimetry that eliminates the need to directly visualize or instrument the flow itself. △ Less

Submitted 18 March, 2021; v1 submitted 18 November, 2020; originally announced November 2020.

Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging

Authors: He Sun, Katherine L. Bouman

Abstract: Computational image reconstruction algorithms generally produce a single image without any measure of uncertainty or confidence. Regularized Maximum Likelihood (RML) and feed-forward deep learning approaches for inverse problems typically focus on recovering a point estimate. This is a serious limitation when working with underdetermined imaging systems, where it is conceivable that multiple image… ▽ More Computational image reconstruction algorithms generally produce a single image without any measure of uncertainty or confidence. Regularized Maximum Likelihood (RML) and feed-forward deep learning approaches for inverse problems typically focus on recovering a point estimate. This is a serious limitation when working with underdetermined imaging systems, where it is conceivable that multiple image modes would be consistent with the measured data. Characterizing the space of probable images that explain the observational data is therefore crucial. In this paper, we propose a variational deep probabilistic imaging approach to quantify reconstruction uncertainty. Deep Probabilistic Imaging (DPI) employs an untrained deep generative model to estimate a posterior distribution of an unobserved image. This approach does not require any training data; instead, it optimizes the weights of a neural network to generate image samples that fit a particular measurement dataset. Once the network weights have been learned, the posterior distribution can be efficiently sampled. We demonstrate this approach in the context of interferometric radio imaging, which is used for black hole imaging with the Event Horizon Telescope, and compressed sensing Magnetic Resonance Imaging (MRI). △ Less

Submitted 17 December, 2020; v1 submitted 27 October, 2020; originally announced October 2020.

Comments: This paper has been accepted to AAAI 2021. Keywords: Computational Imaging, Normalizing Flow, Uncertainty Quantification, Interferometry, MRI

SYMBA: An end-to-end VLBI synthetic data generation pipeline

Authors: F. Roelofs, M. Janssen, I. Natarajan, R. Deane, J. Davelaar, H. Olivares, O. Porth, S. N. Paine, K. L. Bouman, R. P. J. Tilanus, I. M. van Bemmel, H. Falcke, K. Akiyama, A. Alberdi, W. Alef, K. Asada, R. Azulay, A. Baczko, D. Ball, M. Baloković, J. Barrett, D. Bintley, L. Blackburn, W. Boland, G. C. Bower , et al. (183 additional authors not shown)

Abstract: Realistic synthetic observations of theoretical source models are essential for our understanding of real observational data. In using synthetic data, one can verify the extent to which source parameters can be recovered and evaluate how various data corruption effects can be calibrated. These studies are important when proposing observations of new sources, in the characterization of the capabili… ▽ More Realistic synthetic observations of theoretical source models are essential for our understanding of real observational data. In using synthetic data, one can verify the extent to which source parameters can be recovered and evaluate how various data corruption effects can be calibrated. These studies are important when proposing observations of new sources, in the characterization of the capabilities of new or upgraded instruments, and when verifying model-based theoretical predictions in a comparison with observational data. We present the SYnthetic Measurement creator for long Baseline Arrays (SYMBA), a novel synthetic data generation pipeline for Very Long Baseline Interferometry (VLBI) observations. SYMBA takes into account several realistic atmospheric, instrumental, and calibration effects. We used SYMBA to create synthetic observations for the Event Horizon Telescope (EHT), a mm VLBI array, which has recently captured the first image of a black hole shadow. After testing SYMBA with simple source and corruption models, we study the importance of including all corruption and calibration effects. Based on two example general relativistic magnetohydrodynamics (GRMHD) model images of M87, we performed case studies to assess the attainable image quality with the current and future EHT array for different weather conditions. The results show that the effects of atmospheric and instrumental corruptions on the measured visibilities are significant. Despite these effects, we demonstrate how the overall structure of the input models can be recovered robustly after performing calibration steps. With the planned addition of new stations to the EHT array, images could be reconstructed with higher angular resolution and dynamic range. In our case study, these improvements allowed for a distinction between a thermal and a non-thermal GRMHD model based on salient features in reconstructed images. △ Less

Submitted 2 April, 2020; originally announced April 2020.

Comments: 20 pages, 15 figures, accepted for publication in A&A

Learning a Probabilistic Strategy for Computational Imaging Sensor Selection

Authors: He Sun, Adrian V. Dalca, Katherine L. Bouman

Abstract: Optimized sensing is important for computational imaging in low-resource environments, when images must be recovered from severely limited measurements. In this paper, we propose a physics-constrained, fully differentiable, autoencoder that learns a probabilistic sensor-sampling strategy for optimized sensor design. The proposed method learns a system's preferred sampling distribution that charact… ▽ More Optimized sensing is important for computational imaging in low-resource environments, when images must be recovered from severely limited measurements. In this paper, we propose a physics-constrained, fully differentiable, autoencoder that learns a probabilistic sensor-sampling strategy for optimized sensor design. The proposed method learns a system's preferred sampling distribution that characterizes the correlations between different sensor selections as a binary, fully-connected Ising model. The learned probabilistic model is achieved by using a Gibbs sampling inspired network architecture, and is trained end-to-end with a reconstruction network for efficient co-design. The proposed framework is applicable to sensor selection problems in a variety of computational imaging applications. In this paper, we demonstrate the approach in the context of a very-long-baseline-interferometry (VLBI) array design task, where sensor correlations and atmospheric noise present unique challenges. We demonstrate results broadly consistent with expectation, and draw attention to particular structures preferred in the telescope array geometry that can be leveraged to plan future observations and design array expansions. △ Less

Submitted 23 March, 2020; originally announced March 2020.

Comments: This paper has been accepted to the IEEE International Conference on Computational Photography (ICCP) 2020. Keywords: Computational Imaging, Optimized Sensing, Ising Model, Deep Learning, VLBI, Interferometry

Studying Black Holes on Horizon Scales with VLBI Ground Arrays

Authors: Lindy Blackburn, Sheperd Doeleman, Jason Dexter, José L. Gómez, Michael D. Johnson, Daniel C. Palumbo, Jonathan Weintroub, Katherine L. Bouman, Andrew A. Chael, Joseph R. Farah, Vincent Fish, Laurent Loinard, Colin Lonsdale, Gopal Narayanan, Nimesh A. Patel, Dominic W. Pesce, Alexander Raymond, Remo Tilanus, Maciek Wielgus, Kazunori Akiyama, Geoffrey Bower, Avery Broderick, Roger Deane, Christian M. Fromm, Charles Gammie , et al. (13 additional authors not shown)

Abstract: High-resolution imaging of supermassive black holes is now possible, with new applications to testing general relativity and horizon-scale accretion and relativistic jet formation processes. Over the coming decade, the EHT will propose to add new strategically placed VLBI elements operating at 1.3mm and 0.87mm wavelength. In parallel, development of next-generation backend instrumentation, coupled… ▽ More High-resolution imaging of supermassive black holes is now possible, with new applications to testing general relativity and horizon-scale accretion and relativistic jet formation processes. Over the coming decade, the EHT will propose to add new strategically placed VLBI elements operating at 1.3mm and 0.87mm wavelength. In parallel, development of next-generation backend instrumentation, coupled with high throughput correlation architectures, will boost sensitivity, allowing the new stations to be of modest collecting area while still improving imaging fidelity and angular resolution. The goal of these efforts is to move from imaging static horizon scale structure to dynamic reconstructions that capture the processes of accretion and jet launching in near real time. △ Less

Submitted 8 November, 2019; v1 submitted 3 September, 2019; originally announced September 2019.

Comments: Astro2020 APC White Paper, 10 pages, 7 figures

Studying black holes on horizon scales with space-VLBI

Authors: Kari Haworth, Michael D. Johnson, Dominic W. Pesce, Daniel C. M. Palumbo, Lindy Blackburn, Kazunori Akiyama, Don Boroson, Katherine L. Bouman, Joseph R. Farah, Vincent L. Fish, Mareki Honma, Tomohisa Kawashima, Motoki Kino, Alexander Raymond, Mark Silver, Jonathan Weintroub, Maciek Wielgus, Sheperd S. Doeleman, Jose L. Gomez, Jens Kauffmann, Garrett K. Keating, Thomas P. Krichbaum, Laurent Loinard, Gopal Narayanan, Akihiro Doi David J. James , et al. (3 additional authors not shown)

Abstract: The Event Horizon Telescope (EHT) recently produced the first horizon-scale image of a supermassive black hole. Expanding the array to include a 3-meter space telescope operating at >200 GHz enables mass measurements of many black holes, movies of black hole accretion flows, and new tests of general relativity that are impossible from the ground. The Event Horizon Telescope (EHT) recently produced the first horizon-scale image of a supermassive black hole. Expanding the array to include a 3-meter space telescope operating at >200 GHz enables mass measurements of many black holes, movies of black hole accretion flows, and new tests of general relativity that are impossible from the ground. △ Less

Submitted 3 September, 2019; originally announced September 2019.

The DSA-2000 -- A Radio Survey Camera

Authors: G. Hallinan, V. Ravi, S. Weinreb, J. Kocz, Y. Huang, D. P. Woody, J. Lamb, L. D'Addario, M. Catha, J. Shi, C. Law, S. R. Kulkarni, E. S. Phinney, M. W. Eastwood, K. L. Bouman, M. A. McLaughlin, S. M. Ransom, X. Siemens, J. M. Cordes, R. S. Lynch, D. L. Kaplan, S. Chatterjee, J. Lazio, A. Brazier, S. Bhatnagar , et al. (3 additional authors not shown)

Abstract: We present the DSA-2000: a world-leading radio survey telescope and multi-messenger discovery engine for the next decade. The array will be the first true radio camera, outputting science-ready image data over the 0.7 - 2 GHz frequency range with a spatial resolution of 3.5 arcsec. With 2000 x 5 m dishes, the DSA-2000 will have an equivalent point-source sensitivity to SKA1-mid, but with ten times… ▽ More We present the DSA-2000: a world-leading radio survey telescope and multi-messenger discovery engine for the next decade. The array will be the first true radio camera, outputting science-ready image data over the 0.7 - 2 GHz frequency range with a spatial resolution of 3.5 arcsec. With 2000 x 5 m dishes, the DSA-2000 will have an equivalent point-source sensitivity to SKA1-mid, but with ten times the survey speed. The DSA-2000 is envisaged as an all-sky survey instrument complementary to the ngVLA, and as a counterpart to the LSST (optical), SPHEREx (near-infrared) and SRG/eROSITA (X-ray) all-sky surveys. Over a five-year prime phase, the DSA-2000 will image the entire sky above declination -30 degrees every four months, detecting > 1 unique billion radio sources in a combined full-Stokes sky map with 500 nJy/beam rms noise. This all-sky survey will be complemented by intermediate and deep surveys, as well as spectral and polarization image cubes. The array will be a cornerstone for multi-messenger science, serving as the principal instrument for the US pulsar timing array community, and by searching for radio afterglows of compact object mergers detected by LIGO and Virgo. The array will simultaneously detect and localize ~10,000 fast radio bursts each year, realizing their ultimate use as a cosmological tool. The DSA-2000 will be proposed to the NSF Mid-Scale Research Infrastructure-2 program with a view to first light in 2026 △ Less

Submitted 17 July, 2019; originally announced July 2019.

Comments: Submitted as a Project White Paper for the Astro2020 Decadal Survey (12 pages, 7 figures, 2 table)

Metrics and Motivations for Earth-Space VLBI: Time-Resolving Sgr A* with the Event Horizon Telescope

Authors: Daniel C. M. Palumbo, Sheperd S. Doeleman, Michael D. Johnson, Katherine L. Bouman, Andrew A. Chael

Abstract: Very-long-baseline interferometry (VLBI) at frequencies above 230 GHz with Earth-diameter baselines gives spatial resolution finer than the ${\sim}50 μ$as "shadow" of the supermassive black hole at the Galactic Center, Sagittarius A* (Sgr A*). Imaging static and dynamical structure near the "shadow" provides a test of general relativity and may allow measurement of black hole parameters. However,… ▽ More Very-long-baseline interferometry (VLBI) at frequencies above 230 GHz with Earth-diameter baselines gives spatial resolution finer than the ${\sim}50 μ$as "shadow" of the supermassive black hole at the Galactic Center, Sagittarius A* (Sgr A*). Imaging static and dynamical structure near the "shadow" provides a test of general relativity and may allow measurement of black hole parameters. However, traditional Earth-rotation synthesis is inapplicable for sources (such as Sgr A*) with intra-day variability. Expansions of ground-based arrays to include space-VLBI stations may enable imaging capability on time scales comparable to the prograde innermost stable circular orbit (ISCO) of Sgr A*, which is predicted to be 4-30 minutes, depending on black hole spin. We examine the basic requirements for space-VLBI, and we develop tools for simulating observations with orbiting stations. We also develop a metric to quantify the imaging capabilities of an array irrespective of detailed image morphology or reconstruction method. We validate this metric on example reconstructions of simulations of Sgr A* at 230 and 345 GHz, and use these results to motivate expanding the Event Horizon Telescope (EHT) to include small dishes in Low Earth Orbit (LEO). We demonstrate that high-sensitivity sites such as the Atacama Large Millimeter/Submillimeter Array (ALMA) make it viable to add small orbiters to existing ground arrays, as space-ALMA baselines would have sensitivity comparable to ground-based non-ALMA baselines. We show that LEO-enhanced arrays sample half of the diffraction-limited Fourier plane of Sgr A* in less than 30 minutes, enabling reconstructions of near-horizon structure with normalized root-mean-square error $\lesssim0.3$ on sub-ISCO timescales. △ Less

Submitted 20 June, 2019; originally announced June 2019.

Comments: 16 pages, 9 figures, accepted for publication in ApJ

EHT-HOPS pipeline for millimeter VLBI data reduction

Authors: Lindy Blackburn, Chi-kwan Chan, Geoffrey B. Crew, Vincent L. Fish, Sara Issaoun, Michael D. Johnson, Maciek Wielgus, Kazunori Akiyama, John Barrett, Katherine L. Bouman, Roger Cappallo, Andrew A. Chael, Michael Janssen, Colin J. Lonsdale, Sheperd S. Doeleman

Abstract: We present the design and implementation of an automated data calibration and reduction pipeline for very-long-baseline interferometric (VLBI) observations taken at millimeter wavelengths. These short radio-wavelengths provide the best imaging resolution available from ground-based VLBI networks such as the Event Horizon Telescope (EHT) and the Global Millimeter VLBI Array (GMVA), but require spec… ▽ More We present the design and implementation of an automated data calibration and reduction pipeline for very-long-baseline interferometric (VLBI) observations taken at millimeter wavelengths. These short radio-wavelengths provide the best imaging resolution available from ground-based VLBI networks such as the Event Horizon Telescope (EHT) and the Global Millimeter VLBI Array (GMVA), but require specialized processing due to the strong effects from atmospheric opacity and turbulence as well as the heterogeneous nature of existing global arrays. The pipeline builds upon a calibration suite (HOPS) originally designed for precision geodetic VLBI. To support the reduction of data for astronomical observations, we have developed an additional framework for global phase and amplitude calibration which provides output in a standard data format for astronomical imaging and analysis. We test the pipeline on observations taken at 3.5 mm (86 GHz) by the GMVA joined by the phased Atacama Large Millimeter/submillimeter Array in April 2017, and demonstrate the benefits from the specialized processing of high frequency VLBI data with respect to classical analysis techniques. △ Less

Submitted 28 August, 2019; v1 submitted 21 March, 2019; originally announced March 2019.

Journal ref: ApJ, 882, 23 (2019)

The Size, Shape, and Scattering of Sagittarius A* at 86 GHz: First VLBI with ALMA

Authors: S. Issaoun, M. D. Johnson, L. Blackburn, C. D. Brinkerink, M. Mościbrodzka, A. Chael, C. Goddi, I. Martí-Vidal, J. Wagner, S. S. Doeleman, H. Falcke, T. P. Krichbaum, K. Akiyama, U. Bach, K. L. Bouman, G. C. Bower, A. Broderick, I. Cho, G. Crew, J. Dexter, V. Fish, R. Gold, J. L. Gómez, K. Hada, A. Hernández-Gómez , et al. (19 additional authors not shown)

Abstract: The Galactic Center supermassive black hole Sagittarius A* (Sgr A*) is one of the most promising targets to study the dynamics of black hole accretion and outflow via direct imaging with very long baseline interferometry (VLBI). At 3.5 mm (86 GHz), the emission from Sgr A* is resolvable with the Global Millimeter VLBI Array (GMVA). We present the first observations of Sgr A* with the phased Atacam… ▽ More The Galactic Center supermassive black hole Sagittarius A* (Sgr A*) is one of the most promising targets to study the dynamics of black hole accretion and outflow via direct imaging with very long baseline interferometry (VLBI). At 3.5 mm (86 GHz), the emission from Sgr A* is resolvable with the Global Millimeter VLBI Array (GMVA). We present the first observations of Sgr A* with the phased Atacama Large Millimeter/submillimeter Array (ALMA) joining the GMVA. Our observations achieve an angular resolution of ~87μas, improving upon previous experiments by a factor of two. We reconstruct a first image of the unscattered source structure of Sgr A* at 3.5 mm, mitigating effects of interstellar scattering. The unscattered source has a major axis size of 120 $\pm$ 34μas (12 $\pm$ 3.4 Schwarzschild radii), and a symmetrical morphology (axial ratio of 1.2$^{+0.3}_{-0.2}$), which is further supported by closure phases consistent with zero within 3σ. We show that multiple disk-dominated models of Sgr A* match our observational constraints, while the two jet-dominated models considered are constrained to small viewing angles. Our long-baseline detections to ALMA also provide new constraints on the scattering of Sgr A*, and we show that refractive scattering effects are likely to be weak for images of Sgr A* at 1.3 mm with the Event Horizon Telescope. Our results provide the most stringent constraints to date for the intrinsic morphology and refractive scattering of Sgr A*, demonstrating the exceptional contribution of ALMA to millimeter VLBI. △ Less

Submitted 18 January, 2019; originally announced January 2019.

Comments: Accepted for publication in ApJ

Medical Image Imputation from Image Collections

Authors: Adrian V. Dalca, Katherine L. Bouman, William T. Freeman, Natalia S. Rost, Mert R. Sabuncu, Polina Golland

Abstract: We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large data sets of clinical images contain a wealth of information, time constraints during acquisition result in sparse scans that fail to capture much of the anatomy. These characteristics often render computational analysis… ▽ More We present an algorithm for creating high resolution anatomically plausible images consistent with acquired clinical brain MRI scans with large inter-slice spacing. Although large data sets of clinical images contain a wealth of information, time constraints during acquisition result in sparse scans that fail to capture much of the anatomy. These characteristics often render computational analysis impractical as many image analysis algorithms tend to fail when applied to such images. Highly specialized algorithms that explicitly handle sparse slice spacing do not generalize well across problem domains. In contrast, we aim to enable application of existing algorithms that were originally developed for high resolution research scans to significantly undersampled scans. We introduce a generative model that captures fine-scale anatomical structure across subjects in clinical image collections and derive an algorithm for filling in the missing data in scans with large inter-slice spacing. Our experimental results demonstrate that the resulting method outperforms state-of-the-art upsampling super-resolution techniques, and promises to facilitate subsequent analysis not previously possible with scans of this quality. Our implementation is freely available at https://github.com/adalca/papago . △ Less

Submitted 16 August, 2018; originally announced August 2018.

Comments: Accepted at IEEE Transactions on Medical Imaging (\c{opyright} 2018 IEEE)

Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

Authors: Tianfan Xue, Jiajun Wu, Katherine L. Bouman, William T. Freeman

Abstract: We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods that have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To sy… ▽ More We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods that have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To synthesize realistic movement of objects, we propose a novel network structure, namely a Cross Convolutional Network; this network encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, and on real-world video frames. We present analyses of the learned network representations, showing it is implicitly learning a compact encoding of object appearance and motion. We also demonstrate a few of its applications, including visual analogy-making and video extrapolation. △ Less

Submitted 9 August, 2019; v1 submitted 24 July, 2018; originally announced July 2018.

Comments: Journal preprint of arXiv:1607.02586 (IEEE TPAMI, 2019). The first two authors contributed equally to this work. Project page: http://visualdynamics.csail.mit.edu

Journal ref: IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 41, no. 9, pp. 2236-2250, 2019

Interferometric Imaging Directly with Closure Phases and Closure Amplitudes

Authors: Andrew A. Chael, Michael D. Johnson, Katherine L. Bouman, Lindy L. Blackburn, Kazunori Akiyama, Ramesh Narayan

Abstract: Interferometric imaging now achieves angular resolutions as fine as 10 microarcsec, probing scales that are inaccessible to single telescopes. Traditional synthesis imaging methods require calibrated visibilities; however, interferometric calibration is challenging, especially at high frequencies. Nevertheless, most studies present only a single image of their data after a process of "self-calibra… ▽ More Interferometric imaging now achieves angular resolutions as fine as 10 microarcsec, probing scales that are inaccessible to single telescopes. Traditional synthesis imaging methods require calibrated visibilities; however, interferometric calibration is challenging, especially at high frequencies. Nevertheless, most studies present only a single image of their data after a process of "self-calibration," an iterative procedure where the initial image and calibration assumptions can significantly influence the final image. We present a method for efficient interferometric imaging directly using only closure amplitudes and closure phases, which are immune to station-based calibration errors. Closure-only imaging provides results that are as non-committal as possible and allows for reconstructing an image independently from separate amplitude and phase self-calibration. While closure-only imaging eliminates some image information (e.g., the total image flux density and the image centroid), this information can be recovered through a small number of additional constraints. We demonstrate that closure-only imaging can produce high fidelity results, even for sparse arrays such as the Event Horizon Telescope, and that the resulting images are independent of the level of systematic amplitude error. We apply closure imaging to VLBA and ALMA data and show that it is capable of matching or exceeding the performance of traditional self-calibration and CLEAN for these data sets. △ Less

Submitted 19 March, 2018; originally announced March 2018.

Comments: 20 pages, 10 figures. Accepted to ApJ

Reconstructing Video from Interferometric Measurements of Time-Varying Sources

Authors: Katherine L. Bouman, Michael D. Johnson, Adrian V. Dalca, Andrew A. Chael, Freek Roelofs, Sheperd S. Doeleman, William T. Freeman

Abstract: Very long baseline interferometry (VLBI) makes it possible to recover images of astronomical sources with extremely high angular resolution. Most recently, the Event Horizon Telescope (EHT) has extended VLBI to short millimeter wavelengths with a goal of achieving angular resolution sufficient for imaging the event horizons of nearby supermassive black holes. VLBI provides measurements related to… ▽ More Very long baseline interferometry (VLBI) makes it possible to recover images of astronomical sources with extremely high angular resolution. Most recently, the Event Horizon Telescope (EHT) has extended VLBI to short millimeter wavelengths with a goal of achieving angular resolution sufficient for imaging the event horizons of nearby supermassive black holes. VLBI provides measurements related to the underlying source image through a sparse set spatial frequencies. An image can then be recovered from these measurements by making assumptions about the underlying image. One of the most important assumptions made by conventional imaging methods is that over the course of a night's observation the image is static. However, for quickly evolving sources, such as the galactic center's supermassive black hole (Sgr A*) targeted by the EHT, this assumption is violated and these conventional imaging approaches fail. In this work we propose a new way to model VLBI measurements that allows us to recover both the appearance and dynamics of an evolving source by reconstructing a video rather than a static image. By modeling VLBI measurements using a Gaussian Markov Model, we are able to propagate information across observations in time to reconstruct a video, while simultaneously learning about the dynamics of the source's emission region. We demonstrate our proposed Expectation-Maximization (EM) algorithm, StarWarps, on realistic synthetic observations of black holes, and show how it substantially improves results compared to conventional imaging algorithms. Additionally, we demonstrate StarWarps on real VLBI data of the M87 Jet from the VLBA. △ Less

Submitted 1 February, 2018; v1 submitted 3 November, 2017; originally announced November 2017.

Comments: Submitted to Transactions on Computational Imaging

Dynamical Imaging with Interferometry

Authors: Michael D. Johnson, Katherine L. Bouman, Lindy Blackburn, Andrew A. Chael, Julian Rosen, Hotaka Shiokawa, Freek Roelofs, Kazunori Akiyama, Vincent L. Fish, Sheperd S. Doeleman

Abstract: By linking widely separated radio dishes, the technique of very long baseline interferometry (VLBI) can greatly enhance angular resolution in radio astronomy. However, at any given moment, a VLBI array only sparsely samples the information necessary to form an image. Conventional imaging techniques partially overcome this limitation by making the assumption that the observed cosmic source structur… ▽ More By linking widely separated radio dishes, the technique of very long baseline interferometry (VLBI) can greatly enhance angular resolution in radio astronomy. However, at any given moment, a VLBI array only sparsely samples the information necessary to form an image. Conventional imaging techniques partially overcome this limitation by making the assumption that the observed cosmic source structure does not evolve over the duration of an observation, which enables VLBI networks to accumulate information as the Earth rotates and changes the projected array geometry. Although this assumption is appropriate for nearly all VLBI, it is almost certainly violated for submillimeter observations of the Galactic Center supermassive black hole, Sagittarius A* (Sgr A*), which has a gravitational timescale of only ~20 seconds and exhibits intra-hour variability. To address this challenge, we develop several techniques to reconstruct dynamical images ("movies") from interferometric data. Our techniques are applicable to both single-epoch and multi-epoch variability studies, and they are suitable for exploring many different physical processes including flaring regions, stable images with small time-dependent perturbations, steady accretion dynamics, or kinematics of relativistic jets. Moreover, dynamical imaging can be used to estimate time-averaged images from time-variable data, eliminating many spurious image artifacts that arise when using standard imaging methods. We demonstrate the effectiveness of our techniques using synthetic observations of simulated black hole systems and 7mm Very Long Baseline Array observations of M87, and we show that dynamical imaging is feasible for Event Horizon Telescope observations of Sgr A*. △ Less

Submitted 3 November, 2017; originally announced November 2017.

Comments: 16 Pages, 12 Figures, Accepted for publication in ApJ

Imaging the Schwarzschild-radius-scale Structure of M87 with the Event Horizon Telescope using Sparse Modeling

Authors: Kazunori Akiyama, Kazuki Kuramochi, Shiro Ikeda, Vincent L. Fish, Fumie Tazaki, Mareki Honma, Sheperd S. Doeleman, Avery Broderick, Jason Dexter, Monika Mościbrodzka, Katherine L. Bouman, Andrew Chael, Masamichi Zaizen

Abstract: We propose a new imaging technique for radio and optical/infrared interferometry. The proposed technique reconstructs the image from the visibility amplitude and closure phase, which are standard data products of short-millimeter very long baseline interferometers such as the Event Horizon Telescope (EHT) and optical/infrared interferometers, by utilizing two regularization functions: the… ▽ More We propose a new imaging technique for radio and optical/infrared interferometry. The proposed technique reconstructs the image from the visibility amplitude and closure phase, which are standard data products of short-millimeter very long baseline interferometers such as the Event Horizon Telescope (EHT) and optical/infrared interferometers, by utilizing two regularization functions: the $\ell_1$-norm and total variation (TV) of the brightness distribution. In the proposed method, optimal regularization parameters, which represent the sparseness and effective spatial resolution of the image, are derived from data themselves using cross validation (CV). As an application of this technique, we present simulated observations of M87 with the EHT based on four physically motivated models. We confirm that $\ell_1$+TV regularization can achieve an optimal resolution of $\sim 20-30$% of the diffraction limit $λ/D_{\rm max}$, which is the nominal spatial resolution of a radio interferometer. With the proposed technique, the EHT can robustly and reasonably achieve super-resolution sufficient to clearly resolve the black hole shadow. These results make it promising for the EHT to provide an unprecedented view of the event-horizon-scale structure in the vicinity of the super-massive black hole in M87 and also the Galactic center Sgr A*. △ Less

Submitted 23 February, 2017; originally announced February 2017.

Comments: 16 pages, 6 figures, accepted for publication in ApJ

Multi-resolution Data Fusion for Super-Resolution Electron Microscopy

Authors: Suhas Sreehari, S. V. Venkatakrishnan, Katherine L. Bouman, Jeffrey P. Simmons, Lawrence F. Drummy, Charles A. Bouman

Abstract: Perhaps surprisingly, the total electron microscopy (EM) data collected to date is less than a cubic millimeter. Consequently, there is an enormous demand in the materials and biological sciences to image at greater speed and lower dosage, while maintaining resolution. Traditional EM imaging based on homogeneous raster-order scanning severely limits the volume of high-resolution data that can be c… ▽ More Perhaps surprisingly, the total electron microscopy (EM) data collected to date is less than a cubic millimeter. Consequently, there is an enormous demand in the materials and biological sciences to image at greater speed and lower dosage, while maintaining resolution. Traditional EM imaging based on homogeneous raster-order scanning severely limits the volume of high-resolution data that can be collected, and presents a fundamental limitation to understanding physical processes such as material deformation, crack propagation, and pyrolysis. We introduce a novel multi-resolution data fusion (MDF) method for super-resolution computational EM. Our method combines innovative data acquisition with novel algorithmic techniques to dramatically improve the resolution/volume/speed trade-off. The key to our approach is to collect the entire sample at low resolution, while simultaneously collecting a small fraction of data at high resolution. The high-resolution measurements are then used to create a material-specific patch-library that is used within the "plug-and-play" framework to dramatically improve super-resolution of the low-resolution data. We present results using FEI electron microscope data that demonstrate super-resolution factors of 4x, 8x, and 16x, while substantially maintaining high image quality and reducing dosage. △ Less

Submitted 28 November, 2016; originally announced December 2016.

Observing---and Imaging---Active Galactic Nuclei with the Event Horizon Telescope

Authors: Vincent L. Fish, Kazunori Akiyama, Katherine L. Bouman, Andrew A. Chael, Michael D. Johnson, Sheperd S. Doeleman, Lindy Blackburn, John F. C. Wardle, William T. Freeman, the Event Horizon Telescope Collaboration

Abstract: Originally developed to image the shadow region of the central black hole in Sagittarius A* and in the nearby galaxy M87, the Event Horizon Telescope (EHT) provides deep, very high angular resolution data on other AGN sources too. The challenges of working with EHT data have spurred the development of new image reconstruction algorithms. This work briefly reviews the status of the EHT and its util… ▽ More Originally developed to image the shadow region of the central black hole in Sagittarius A* and in the nearby galaxy M87, the Event Horizon Telescope (EHT) provides deep, very high angular resolution data on other AGN sources too. The challenges of working with EHT data have spurred the development of new image reconstruction algorithms. This work briefly reviews the status of the EHT and its utility for observing AGN sources, with emphasis on novel imaging techniques that offer the promise of better reconstructions at 1.3 mm and other wavelengths. △ Less

Submitted 11 July, 2016; originally announced July 2016.

Comments: 10 pages, proceedings contribution for Blazars through Sharp Multi-Wavelength Eyes, submitted to Galaxies