research-article

Photo-Consistent Reconstruction of Semitransparent Scenes by Density-Sheet Decomposition

Authors:

Samuel W. Hasinoff,

Kiriakos N. KutulakosAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 29, Issue 5

Pages 870 - 885

https://doi.org/10.1109/TPAMI.2007.1056

Published: 01 May 2007 Publication History

Abstract

This paper considers the problem of reconstructing visually realistic 3D models of dynamic semitransparent scenes, such as fire, from a very small set of simultaneous views (even two). We show that this problem is equivalent to a severely underconstrained computerized tomography problem, for which traditional methods break down. Our approach is based on the observation that every pair of photographs of a semitransparent scene defines a unique density field, called a Density Sheet, that 1) concentrates all its density on one connected, semitransparent surface, 2) reproduces the two photos exactly, and 3) is the most spatially compact density field that does so. From this observation, we reduce reconstruction to the convex combination of sheet-like density fields, each of which is derived from the Density Sheet of two input views. We have applied this method specifically to the problem of reconstructing 3D models of fire. Experimental results suggest that this method enables high-quality view synthesis without overfitting artifacts.

References

[1]

D. Nguyen, R. Fedkiw, and H. Jensen, “Physically Based Modeling and Animation of Fire,” Proc. ACM SIGGRAPH, pp.721-728, 2002.

Digital Library

[2]

J. Stam and E. Fiume, “Depicting Fire and Other Gaseous Phenomena Using Diffusion Processes,” Proc. ACM SIGGRAPH, pp. 129-136, 1995.

Digital Library

[3]

R. Fedkiw, J. Stam, and H.W. Jensen, “Visual Simulation of Smoke,” Proc. ACM SIGGRAPH, pp. 15-22, 2001.

Digital Library

[4]

T. Hawkins, P. Einarsson, and P. Debevec, “Acquisition of Time-Varying Participating Media,” Proc. ACM SIGGRAPH, pp. 812-815, 2005.

Digital Library

[5]

N. Foster and D.N. Metaxas, “Modeling the Motion of a Hot, Turbulent Gas,” Proc. ACM SIGGRAPH, pp. 181-188, 1997.

Digital Library

[6]

A. Lamorlette and N. Foster, “Structural Modeling of Flames for a Production Environment,” Proc. ACM SIGGRAPH, pp. 729-735, 2002.

Digital Library

[7]

N. Chiba, K. Muraoka, H. Takahashi, and M. Miura, “Two-Dimensional Visual Simulation of Flames, Smoke and the Spread of Fire,” J. Visualizuation and Computer Animation, vol. 5, no. 1, pp.37-54, Jan.-Mar. 1994.

[8]

Y. Takai, K. Ecchu, and N.K. Takai, “A Cellular Automaton Model of Particle Motions and Its Applications,” The Visual Computer, vol. 11, no. 5, pp. 240-252, 1995.

[9]

R.T. Baum, K.B. McGrattan, and M. Nyden, “An Examination of the Applicability of Computed Tomography for the Measurement of Component Concentrations in Fire-Generated Plumes,” Combustion and Flame, vol. 113, pp. 358-372, 1998.

[10]

D.P. Correia, P. Ferrão, and A. Caldeira-Pires, “Advanced 3D Emission Tomography Flame Temperature Sensor,” Combustion Science and Technology, vol. 163, pp. 1-24, 2001.

[11]

A. Schwarz, “Multi-Tomographic Flame Analysis with a Schlieren Apparatus,” Measurement Science & Technology, vol. 7, pp. 406-413, 1996.

[12]

W. Xue, W. Donglou, and P. Gongpei, “Use of Moire Tomography to Measure the Temperature Field of the Flame of a Pyrotechnical Composition from Its Infrared Radiation,” Combustion, Explosion, and Shock Waves, vol. 37, no. 4, pp. 440-442, 2001.

[13]

G.W. Faris and R.L. Byer, “Beam-Deflection Optical Tomography,” Optics Letters, vol. 12, no. 3, pp. 72-74, 1987.

[14]

A.K. Agrawal, N. Butuk, S.R. Gollahalli, and D. Griffin, “Three-Dimensional Rainbow Schlieren Tomography of a Temperature Field in Gas Flows,” Applied Optics, vol. 37, no. 3, pp. 479-485, 1998.

[15]

B.W. Albers and A.K. Agrawal, “Schlieren Analysis of an Oscillating Gas-Jet Diffusion Flame,” Combustion and Flame, vol. 199, pp. 84-94, 1999.

[16]

T. Otsuka and P. Wolanski, “Particle Image Velocimetry (PIV) Analysis of Flame Structure,” J. Loss Prevention in the Process Industries, vol. 14, no. 6, pp. 503-507, 2001.

[17]

A. Schödl, R. Szeliski, D.H. Salesin, and I. Essa, “Video Textures,” Proc. ACM SIGGRAPH, pp. 489-498, 2000.

Digital Library

[18]

D. Drysdale, An Introduction to Fire Dynamics, second ed. John Wiley and Sons, 1998.

[19]

P.J. Narayanan, P.W. Rander, and T. Kanade, “Constructing Virtual Worlds Using Dense Stereo,” Proc. Int'l Conf. Computer Vision, pp. 3-10, 1998.

Digital Library

[20]

A.C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging. IEEE Press, 1988.

[21]

T. Frese, C.A. Bouman, and K. Sauer, “Adaptive Wavelet Graph Model for Bayesian Tomographic Reconstruction,” IEEE Trans. Image Processing, vol. 11, no. 7, pp. 756-770, July 2002.

Digital Library

[22]

R. Szeliski and P. Golland, “Stereo Matching with Transparency and Matting,” Proc. Int'l Conf. Computer Vision, pp. 517-524, 1998.

Digital Library

[23]

I. Ihrke and M. Magnor, “Image-Based Tomographic Reconstruction of Flames,” Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation, pp. 367-375, 2004.

Digital Library

[24]

G. Wolberg, Digital Image Warping. IEEE CS Press, 1990.

Digital Library

[25]

A. Mohammad-Djafari, “Image Reconstruction of a Compact Object from a Few Number of Projections,” Proc. Int'l Conf. Signal and Image Processing, pp. 325-329, 1996.

[26]

V. Elangovan and R.T. Whittaker, “From Sinograms to Surfaces: A Direct Approach to the Segmentation of Tomographic Data,” Proc. Int'l Conf. Medical Image Computing and Computer-Assisted Intervention, pp. 213-223, 2001.

Digital Library

[27]

G.T. Herman and A. Kuba, Eds., Discrete Tomography: Foundations, Algorithms, and Applications. Birkhäuser, 1999.

[28]

F. Dachille IX, K. Mueller, and A. Kaufman, “Volumetric Backprojection,” Proc. IEEE Symp. Volume Visualization, pp. 109-117, 2000.

Digital Library

[29]

J.D. Bonet and P. Viola, “Roxels: Responsibility Weighted 3D Volume Reconstruction,” Proc. Int'l Conf. Computer Vision, pp. 418-425, 1999.

[30]

R. Bhotika, D.J. Fleet, and K.N. Kutulakos, “A Probabilistic Theory of Occupancy and Emptiness,” Proc. European Conf. Computer Vision, vol. 3, pp. 112-132, 2002.

Digital Library

[31]

A. Reche, I. Martin, and G. Drettakis, “Volumetric Reconstruction and Interactive Rendering of Trees from Photographs,” Proc. ACM SIGGRAPH, pp. 720-727, 2004.

Digital Library

[32]

G.W. Faris and R.L. Byer, “Beam-Deflection Optical Tomography of a Flame,” Optics Letters, vol. 12, no. 3, pp. 155-157, 1987.

[33]

V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, “Graphcut Textures: Image and Video Synthesis Using Graph Cuts,” Proc. ACM SIGGRAPH, pp. 277-286, 2003.

Digital Library

[34]

S. Soatto, G. Doretto, and Y. Wu, “Dynamic Textures,” Proc. Int'l Conf. Computer Vision, pp. 439-446, 2001.

[35]

G. Doretto and S. Soatto, “Editable Dynamic Textures,” Proc. Computer Vision and Pattern Recognition, vol. 2, pp. 137-142, 2003.

[36]

K. Bhat, S. Seitz, J. Hodgins, and P. Khosala, “Flow-Based Video Synthesis and Editing,” Proc. ACM SIGGRAPH, pp. 360-363, 2004.

Digital Library

[37]

J. Duderstadt and W. Martin, Transport Theory. John Wiley and Sons, 1979.

[38]

C. Silva and J. Santos-Victor, “Intrinsic Images for Dense Stereo Matching with Occlusions,” Proc. European Conf. Computer Vision, vol. 1, pp. 100-114, 2000.

Digital Library

[39]

J. Nocedal and S.J. Wright, Numerical Optimization. Springer, 1999.

[40]

J.-Y. Bouguet, “Camera Calibration Toolbox for Matlab,” Sept. 2002, http://www.vision.caltech.edu/bouguetj/calib_doc/.

[41]

S.W. Hasinoff, “Three-Dimensional Reconstruction of Fire from Images,” master's thesis, Dept. of Computer Science, Univ. of Toronto, 2002.

[42]

I. Ihrke and M. Magnor, “GrOVis-Fire: A Multi-Video Sequence for Volumetric Reconstruction and Rendering Research,” http://www.mpi-sb.mpg.de/~ihrke/Projects/Fire/GrOVis_Fire/, 2005.

[43]

S. Seitz and C. Dyer, “View Morphing,” Proc. ACM SIGGRAPH, pp. 21-30, 1996.

Digital Library

[44]

Digital Air Incorporated, http://www.virtualcamera.com, 2003.

[45]

G.W. Stewart, Introduction to Matrix Computations. Academic Press, 1973.

Cited By

Deng YYu HZhang DWu JZhu B(2023)Fluid Simulation on Neural Flow MapsACM Transactions on Graphics10.1145/361839242:6(1-21)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618392
Kang KBi ZFeng XDong YZhou KWu H(2023)Differentiable Dynamic Visible-Light TomographySIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618166(1-12)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618166
Ding YLi ZChen ZJi YYu JYe J(2023)Full-Volume 3D Fluid Flow Reconstruction With Light Field PIVIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.323634445:7(8405-8418)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3236344
Show More Cited By

Index Terms

Photo-Consistent Reconstruction of Semitransparent Scenes by Density-Sheet Decomposition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Reconstruction
      2. Image and video acquisition
        3D imaging
  2. Computer graphics
    1. Animation
    2. Shape modeling
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Computational geometry

Recommendations

Synthesizing Realistic Shadows with Bumpy Surface and Object Model Estimated by Shadows in Single Image
GRAPP 2015: Proceedings of the 10th International Conference on Computer Graphics Theory and Applications

Shadows play an important role in the human perception of 3D geometry in photographs, and properly synthesized shadows produce a realistic rendering result. Shadows, especially hard shadows, define powerful constraints between a light source, occluders, ...
Image-based rendering for scenes with reflections

We present a system for image-based modeling and rendering of real-world scenes containing reflective and glossy surfaces. Previous approaches to image-based rendering assume that the scene can be approximated by 3D proxies that enable view ...
Modeling and rendering of walkthrough environments with panoramic images
VRST '04: Proceedings of the ACM symposium on Virtual reality software and technology

An important, potential application of image-based techniques is to create photo-realistic image-based environments for interactive walkthrough. However, existing image-based studies are based on different assumptions with different focuses. There is a ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 29, Issue 5

May 2007

160 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © 2007.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 May 2007

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Deng YYu HZhang DWu JZhu B(2023)Fluid Simulation on Neural Flow MapsACM Transactions on Graphics10.1145/361839242:6(1-21)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618392
Kang KBi ZFeng XDong YZhou KWu H(2023)Differentiable Dynamic Visible-Light TomographySIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618166(1-12)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618166
Ding YLi ZChen ZJi YYu JYe J(2023)Full-Volume 3D Fluid Flow Reconstruction With Light Field PIVIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.323634445:7(8405-8418)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3236344
Zheng QSingh GSeidel HRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Neural relightable participating media renderingProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3541426(15203-15215)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3541426
Guo JLi MZong ZLiu YHe JGuo YYan L(2021)Volumetric appearance stylization with stylizing kernel prediction networkACM Transactions on Graphics10.1145/3450626.345979940:4(1-15)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459799
Wang YIdoughi RHeidrich W(2020)Stereo Event-Based Particle Tracking Velocimetry for 3D Fluid Flow ReconstructionComputer Vision – ECCV 202010.1007/978-3-030-58526-6_3(36-53)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1007/978-3-030-58526-6_3
Li ZJi YYu JYe J(2020)3D Fluid Flow Reconstruction Using Compact Light Field PIVComputer Vision – ECCV 202010.1007/978-3-030-58517-4_8(120-136)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1007/978-3-030-58517-4_8
Takahashi TLin M(2019)Video-guided real-to-virtual parameter transfer for viscous fluidsACM Transactions on Graphics10.1145/3355089.335655138:6(1-12)Online publication date: 8-Nov-2019
https://dl.acm.org/doi/10.1145/3355089.3356551
Eckert MUm KThuerey N(2019)ScalarFlowACM Transactions on Graphics10.1145/3355089.335654538:6(1-16)Online publication date: 8-Nov-2019
https://dl.acm.org/doi/10.1145/3355089.3356545
Zang GIdoughi RTao RLubineau GWonka PHeidrich W(2019)Warp-and-project tomography for rapidly deforming objectsACM Transactions on Graphics10.1145/3306346.332296538:4(1-13)Online publication date: 12-Jul-2019
https://dl.acm.org/doi/10.1145/3306346.3322965
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents