research-article

Low-budget transient imaging using photonic mixer devices

Authors:

Matthias B. Hullin,

Wolfgang HeidrichAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 4

Article No.: 45, Pages 1 - 10

https://doi.org/10.1145/2461912.2461945

Published: 21 July 2013 Publication History

Abstract

Transient imaging is an exciting a new imaging modality that can be used to understand light propagation in complex environments, and to capture and analyze scene properties such as the shape of hidden objects or the reflectance properties of surfaces.

Unfortunately, research in transient imaging has so far been hindered by the high cost of the required instrumentation, as well as the fragility and difficulty to operate and calibrate devices such as femtosecond lasers and streak cameras.

In this paper, we explore the use of photonic mixer devices (PMD), commonly used in inexpensive time-of-flight cameras, as alternative instrumentation for transient imaging. We obtain a sequence of differently modulated images with a PMD sensor, impose a model for local light/object interaction, and use an optimization procedure to infer transient images given the measurements and model. The resulting method produces transient images at a cost several orders of magnitude below existing methods, while simultaneously simplifying and speeding up the capture process.

Supplementary Material

ZIP File (a45-heide.zip)

Supplemental material.

Download
110.72 MB

MP4 File (tp088.mp4)

Download
22.70 MB

References

[1]

Abramson, N. 1978. Light-in-flight recording by holography. Optics Letters 3, 4, 121--123.

[2]

Abramson, N. 1983. Light-in-flight recording: high-speed holographic motion pictures of ultrafast phenomena. Applied optics 22, 2, 215--232.

[3]

Audet, C., and Dennis Jr, J. 2006. Mesh adaptive direct search algorithms for constrained optimization. SIAM Journal on optimization 17, 1, 188--217.

Digital Library

[4]

Beck, A., and Teboulle, M. 2009. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2, 183--202.

Digital Library

[5]

Chambolle, A., and Pock, T. 2011. A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120--145.

Digital Library

[6]

Conroy, R., Dorrington, A., Künnemeyer, R., and Cree, M. 2009. Range imager performance comparison in homo-dyne and heterodyne operating modes. In Proc. SPIE, vol. 7239, 723905.

[7]

Dorrington, A., Cree, M., Payne, A., Conroy, R., and Carnegie, D. 2007. Achieving sub-millimetre precision with a solid-state full-field heterodyning range imaging camera. Measurement Science and Technology 18, 9, 2809.

[8]

Dorrington, A., Godbaz, J., Cree, M., Payne, A., and Streeter, L. 2011. Separating true range measurements from multi-path and scattering interference in commercial range cameras. In Proc. SPIE, vol. 7864.

[9]

Hassan, A., Künnemeyer, R., Dorrington, A., and Payne, A. 2010. Proof of concept of diffuse optical tomography using time-of-flight range imaging cameras. In Proc. Electronics New Zealand Conference, 115--120.

[10]

Huber, P. 1973. Robust regression: asymptotics, conjectures and monte carlo. The Annals of Statistics 1, 5, 799--821.

[11]

Kirmani, A., Hutchison, T., Davis, J., and Raskar, R. 2009. Looking around the corner using transient imaging. In Proc. ICCV, 159--166.

[12]

Lange, R., and Seitz, P. 2001. Solid-state time-of-flight range camera. IEEE J. Quantum Electronics 37, 3, 390--397.

[13]

Lange, R., Seitz, P., Biber, A., and Lauxtermann, S. 2000. Demodulation pixels in CCD and CMOS technologies for time-of-flight ranging. Sensors and camera systems for scientific, industrial, and digital photography applications, 177--188.

[14]

Naik, N., Zhao, S., Velten, A., Raskar, R., and Bala, K. 2011. Single view reflectance capture using multiplexed scattering and time-of-flight imaging. ACM Trans. Graph. 30, 6, 171.

Digital Library

[15]

Nilsson, B., and Carlsson, T. E. 1998. Direct three-dimensional shape measurement by digital light-in-flight holography. Applied Optics 37, 34, 7954--7959.

[16]

Pandharkar, R., Velten, A., Bardagjy, A., Lawson, E., Bawendi, M., and Raskar, R. 2011. Estimating motion and size of moving non-line-of-sight objects in cluttered environments. In Proc. CVPR, 265--272.

Digital Library

[17]

Schmidt, M., 2013. minConf (retrieved Jan 7, 2013). http://www.di.ens.fr/~mschmidt/Software/minConf.html.

[18]

Schwarte, R., Xu, Z., Heinol, H., Olk, J., Klein, R., Buxbaum, B., Fischer, H., and Schulte, J. 1997. New electro-optical mixing and correlating sensor: facilities and applications of the photonic mixer device. In Proc. SPIE, vol. 3100, 245--253.

[19]

Schwarte, R., 1997. Verfahren und Vorrichtung zur Bestimmung der Phasen und/oder Amplitudeninformation einer elektromagnetischen Welle. German Patent DE 19704496.

[20]

Smith, A., Skorupski, J., and Davis, J. 2008. Transient rendering. Tech. Rep. UCSC-SOE-08-26, University of California Santa Cruz, School of Engineering.

[21]

Velten, A., Raskar, R., and Bawendi, M. 2011. Picosecond camera for time-of-flight imaging. In OSA Imaging Systems and Applications.

[22]

Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M., and Raskar, R. 2012. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Communications 3, 745.

[23]

Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Joshi, C., Lawson, E., Bawendi, M., Gutierrez, D., and Raskar, R. 2013. Femto-photography: Capturing and visualizing the propagation of light. ACM Trans. Graph. 32.

Digital Library

[24]

Wan, G., Li, X., Agranov, G., Levoy, M., and Horowitz, W. 2012. CMOS image sensors with multi-bucket pixels for computational photography. IEEE J. Solid-State Circuits 47, 4, 1031--1042.

[25]

Wu, D., O'Toole, M., Velten, A., Agrawal, A., and Raskar, R. 2012. Decomposing global light transport using time of flight imaging. In Prov. CVPR, 366--373.

Digital Library

Cited By

Shahandashti PLópez PBrea VGarcía-Lesta DConde M(2024)Simultaneous Multifrequency Demodulation for Single-Shot Multiple-Path ToF ImagingIEEE Transactions on Computational Imaging10.1109/TCI.2023.334875810(54-68)Online publication date: 2024
https://doi.org/10.1109/TCI.2023.3348758
Wang ZSun MFaccio D(2024)From 3D to 5D: Computational light-in-flight imaging via an SPAD cameraOptics and Lasers in Engineering10.1016/j.optlaseng.2024.108088176(108088)Online publication date: May-2024
https://doi.org/10.1016/j.optlaseng.2024.108088
Royo DSultan TMuñoz AMasumnia-Bisheh KBrandt EGutierrez DVelten AMarco J(2023)Virtual Mirrors: Non-Line-of-Sight Imaging Beyond the Third BounceACM Transactions on Graphics10.1145/359242942:4(1-15)Online publication date: 26-Jul-2023
https://doi.org/10.1145/3592429
Show More Cited By

Index Terms

Low-budget transient imaging using photonic mixer devices
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Document scanning
      2. Graphics recognition and interpretation
2. Computing methodologies
  1. Computer graphics
    1. Image manipulation

Recommendations

Phasor Imaging: A Generalization of Correlation-Based Time-of-Flight Imaging

In correlation-based time-of-flight (C-ToF) imaging systems, light sources with temporally varying intensities illuminate the scene. Due to global illumination, the temporally varying radiance received at the sensor is a combination of light received ...
Looking Around the Corner using Ultrafast Transient Imaging

We propose a novel framework called transient imaging for image formation and scene understanding through impulse illumination and time images. Using time-of-flight cameras and multi-path analysis of global light transport, we pioneer new algorithms and ...
Single view reflectance capture using multiplexed scattering and time-of-flight imaging

This paper introduces the concept of time-of-flight reflectance estimation, and demonstrates a new technique that allows a camera to rapidly acquire reflectance properties of objects from a single view-point, over relatively long distances and without ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 4

July 2013

1215 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2461912

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2013

Published in TOG Volume 32, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

125
Total Citations
View Citations
1,151
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)8

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shahandashti PLópez PBrea VGarcía-Lesta DConde M(2024)Simultaneous Multifrequency Demodulation for Single-Shot Multiple-Path ToF ImagingIEEE Transactions on Computational Imaging10.1109/TCI.2023.334875810(54-68)Online publication date: 2024
https://doi.org/10.1109/TCI.2023.3348758
Wang ZSun MFaccio D(2024)From 3D to 5D: Computational light-in-flight imaging via an SPAD cameraOptics and Lasers in Engineering10.1016/j.optlaseng.2024.108088176(108088)Online publication date: May-2024
https://doi.org/10.1016/j.optlaseng.2024.108088
Royo DSultan TMuñoz AMasumnia-Bisheh KBrandt EGutierrez DVelten AMarco J(2023)Virtual Mirrors: Non-Line-of-Sight Imaging Beyond the Third BounceACM Transactions on Graphics10.1145/359242942:4(1-15)Online publication date: 26-Jul-2023
https://doi.org/10.1145/3592429
Reed AKim JBlanford TPediredla ABrown DJayasuriya S(2023)Neural Volumetric Reconstruction for Coherent Synthetic Aperture SonarACM Transactions on Graphics10.1145/359214142:4(1-20)Online publication date: 26-Jul-2023
https://doi.org/10.1145/3592141
Baek SWalsh NChugunov IShi ZHeide F(2023)Centimeter-wave Free-space Neural Time-of-Flight ImagingACM Transactions on Graphics10.1145/352267142:1(1-18)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1145/3522671
Zheng YAsif M(2023)Coded Illumination for Improved Lensless ImagingIEEE Transactions on Computational Imaging10.1109/TCI.2023.32348989(172-184)Online publication date: 2023
https://doi.org/10.1109/TCI.2023.3234898
Wei MNousias SGulve RLindell DKutulakos K(2023)Passive Ultra-Wideband Single-Photon Imaging2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00747(8101-8112)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.00747
Conde M(2023)Transient Dictionary Learning for Compressed Time-of-Flight ImagingICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10096385(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10096385
Kotwal ALevin AGkioulekas I(2023)Swept-Angle Synthetic Wavelength Interferometry2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00796(8233-8243)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00796
Kotwal ALevin AGkioulekas I(2023)Passive Micron-Scale Time-of-Flight with Sunlight Interferometry2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00403(4139-4149)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00403
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents