article

Looking Around the Corner using Ultrafast Transient Imaging

Authors:

Tyler Hutchison,

Ramesh RaskarAuthors Info & Claims

International Journal of Computer Vision, Volume 95, Issue 1

Pages 13 - 28

https://doi.org/10.1007/s11263-011-0470-y

Published: 01 October 2011 Publication History

Abstract

We propose a novel framework called transient imaging for image formation and scene understanding through impulse illumination and time images. Using time-of-flight cameras and multi-path analysis of global light transport, we pioneer new algorithms and systems for scene understanding through time images. We demonstrate that our proposed transient imaging framework allows us to accomplish tasks that are well beyond the reach of existing imaging technology. For example, one can infer the geometry of not only the visible but also the hidden parts of a scene, enabling us to look around corners. Traditional cameras estimate intensity per pixel I(x,y). Our transient imaging camera captures a 3D time-image I(x,y,t) for each pixel and uses an ultra-short pulse laser for illumination. Emerging technologies are supporting cameras with a temporal-profile per pixel at picosecond resolution, allowing us to capture an ultra-high speed time-image. This time-image contains the time profile of irradiance incident at a sensor pixel. We experimentally corroborated our theory with free space hardware experiments using a femtosecond laser and a picosecond accurate sensing device. The ability to infer the structure of hidden scene elements, unobservable by both the camera and illumination source, will create a range of new computer vision opportunities.

References

[1]

Arvo, J. (1993). Transfer equations in global illumination. In Global Illumination, SIGGRAPH '93 Course Notes.

[2]

Campillo, A., & Shapiro, S. (1983). Picosecond streak camera fluorometry--a review. IEEE Journal of Quantum Electronics.

[3]

Dattorro, J. (2006). Convex optimization & Euclidean distance geometry. Morrisville: Lulu.com.

[4]

Denk, W., Strickler, J.H., & Webb, W.W. (1990). Two-photon laser scanning fluorescence microscopy. Science, 248, 73-76.

[5]

Garren, D., Goldstein, J., Obuchon, D., Greene, R., & North, J. (2004). SAR image formation algorithm with multipath reflectivity estimation. In Proceedings of the IEEE radar conference, 2004 (pp. 323-328).

[6]

Garren, D., Sullivan, D., North, J., & Goldstein, J. (2005). Image preconditioning for a SAR image reconstruction algorithm for multipath scattering. In Proc. of IEEE int. radar conference.

[7]

Gonzalez-Banos, H., & Davis, J. (2004). Computing depth under ambient illumination using multi-shuttered light. Computer Vision and Pattern Recognition.

[8]

Iddan, G. J., & Yahav, G. (2001). 3D imaging in the studio (and elsewhere...). In SPIE.

[9]

Immel, D. S., Cohen, M. F., & Greenberg, D. P. (1986). A radiosity method for non-diffuse environments. In ACM SIGGRAPH.

[10]

Itatani, J., Quéré, F., Yudin, G., Ivanov, M., Krausz, F., & Corkum, P. (2002). Attosecond streak camera. Physical Review Letters.

[11]

Kajiya, J. T. (1986). The rendering equation. In ACM SIGGRAPH.

[12]

Kamerman, G. (1993). Active electro-optical system. In The infrared and electro-optical system handbook: Vol. 6. Laser radar {M}. Chapter 1.

[13]

Kutulakos, K. N., & Steger, E. (2007). A theory of refractive and specular 3d shape by light-path triangulation. International Journal of Computer Vision.

[14]

Lange, R., & Seitz, P. (2001). Solid-state time-of-flight range camera. IEEE Journal of Quantum Electronics.

[15]

Morris, N. J. W., & Kutulakos, K. N. (2007). Reconstructing the surface of inhomogeneous transparent scenes by scatter trace photography. International Conference on Computer Vision.

[16]

Nayar, S. K., Ikeuchi, K., & Kanade, T. (1990). Shape from interreflections. International Conference on Computer Vision.

[17]

Nayar, S. K., Krishnan, G., Grossberg, M. D., & Raskar, R. (2006). Fast separation of direct and global components of a scene using high frequency illumination. In ACM SIGGRAPH.

[18]

Ng, R., Marc, L., Mathieu, B., Gene, D., Mark, H., & Pat, H. (2005). Light field photography with a hand-held plenoptic camera. Stanford University Computer Science Tech Report.

[19]

Patow, G., & Pueyo, X. (2003). A survey of inverse rendering problems. Computer Graphics Forum.

[20]

Ramamoorthi, R., & Hanrahan, P. (2001). A signal-processing framework for inverse rendering. Computer Graphics and Interactive Techniques.

[21]

Sarunic, P., White, K., & Rutten, M. (2001). Over-the-horizon radar multipath and multisensor track fusion algorithm development.

[22]

Schmitt, J. M. (1999). Optical coherence tomography (oct): a review. IEEE Quantum Electronics.

[23]

Seitz, S. M., Matsushita, Y., & Kutulakos, K. N. (2005). A theory of inverse light transport. IEEE International Conference on Computer Vision.

[24]

Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., & Lensch, H. P. A. (2005). Dual photography. In ACM SIGGRAPH.

[25]

Vandapel, N., Amidi, O., & Miller, J. (2004). Toward laser pulse waveform analysis for scene interpretation. IEEE International Conference on Robotics and Automation.

[26]

Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., & Tumblin, J. (2007). Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocussing. In ACM SIGGRAPH.

[27]

Raskar, R., & Davis, J. 5d time-light transport matrix: What can we reason about scene properties, Int. Memo 2007.

[28]

Smith, A., Skorupski, J., & Davis, J. Transient rendering, UC Santa Cruz TR UCSC-SOE-08-26, Feb 2008. 2.

Cited By

Zhang JGeng RDu XChen YLi HHu Y(2025)Passive Non-Line-of-Sight Imaging With Light Transport ModulationIEEE Transactions on Image Processing10.1109/TIP.2024.351809734(410-424)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TIP.2024.3518097
Dodds LShanbhag HGuan JGupta SHassanieh HGanesan DLane NShi W(2024)Around the Corner mmWave Imaging in Practical EnvironmentsProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3690671(953-967)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3690671
Geng RHu YLu ZYu CLi HZhang HChen Y(2022)Passive Non-Line-of-Sight Imaging Using Optimal TransportIEEE Transactions on Image Processing10.1109/TIP.2021.312831231(110-124)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TIP.2021.3128312
Show More Cited By

Looking Around the Corner using Ultrafast Transient Imaging
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks

Recommendations

Image-based rendering of diffuse, specular and glossy surfaces from a single image
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques

In this paper, we present a new method to recover an approximation of the bidirectional reflectance distribution function (BRDF) of the surfaces present in a real scene. This is done from a single photograph and a 3D geometric model of the scene. The ...
The beam radiance estimate for volumetric photon mapping
SIGGRAPH '08: ACM SIGGRAPH 2008 classes

We present a new method for efficiently simulating the scattering of light within participating media. Using a theoretical reformulation of volumetric photon mapping, we develop a novel photon gathering technique for participating media. Traditional ...
Efficient Simulation of Light Transport in Scenes with Participating Media Using Photon Maps
Seminal Graphics Papers: Pushing the Boundaries, Volume 2

This paper presents a new method for computing global illumination in scenes with participating media. The method is based on bidirectional Monte Carlo ray tracing and uses photon maps to increase efficiency and reduce noise. We remove previous ...

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 95, Issue 1

October 2011

98 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2011 Springer Science+Business Media, LLC.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 October 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang JGeng RDu XChen YLi HHu Y(2025)Passive Non-Line-of-Sight Imaging With Light Transport ModulationIEEE Transactions on Image Processing10.1109/TIP.2024.351809734(410-424)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1109/TIP.2024.3518097
Dodds LShanbhag HGuan JGupta SHassanieh HGanesan DLane NShi W(2024)Around the Corner mmWave Imaging in Practical EnvironmentsProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3690671(953-967)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3690671
Geng RHu YLu ZYu CLi HZhang HChen Y(2022)Passive Non-Line-of-Sight Imaging Using Optimal TransportIEEE Transactions on Image Processing10.1109/TIP.2021.312831231(110-124)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TIP.2021.3128312
Wu LCai GRamamoorthi RZhao S(2021)Differentiable time-gated renderingACM Transactions on Graphics10.1145/3478513.348048940:6(1-16)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480489
Velten AWu DMasia BJarabo ABarsi CJoshi CLawson EBawendi MGutierrez DRaskar R(2016)Imaging the propagation of light through scenes at picosecond resolutionCommunications of the ACM10.1145/297516559:9(79-86)Online publication date: 24-Aug-2016
https://dl.acm.org/doi/10.1145/2975165
Jarabo AMarco JMuñoz ABuisan RJarosz WGutierrez D(2014)A framework for transient renderingACM Transactions on Graphics10.1145/2661229.266125133:6(1-10)Online publication date: 19-Nov-2014
https://dl.acm.org/doi/10.1145/2661229.2661251
O'Toole MHeide FXiao LHullin MHeidrich WKutulakos K(2014)Temporal frequency probing for 5D transient analysis of global light transportACM Transactions on Graphics10.1145/2601097.260110333:4(1-11)Online publication date: 27-Jul-2014
https://dl.acm.org/doi/10.1145/2601097.2601103
Ament MBergmann CWeiskopf D(2014)Refractive radiative transfer equationACM Transactions on Graphics10.1145/255760533:2(1-22)Online publication date: 8-Apr-2014
https://dl.acm.org/doi/10.1145/2557605
Velten AWu DJarabo AMasia BBarsi CJoshi CLawson EBawendi MGutierrez DRaskar R(2013)Femto-photographyACM Transactions on Graphics10.1145/2461912.246192832:4(1-8)Online publication date: 21-Jul-2013
https://dl.acm.org/doi/10.1145/2461912.2461928
O'Toole MRaskar RKutulakos K(2012)Primal-dual coding to probe light transportACM Transactions on Graphics10.1145/2185520.218553531:4(1-11)Online publication date: 1-Jul-2012
https://dl.acm.org/doi/10.1145/2185520.2185535

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents