Article

Free access

Environment matting and compositing

Authors:

Douglas E. Zongker,

Dawn M. Werner,

David H. SalesinAuthors Info & Claims

SIGGRAPH '99: Proceedings of the 26th annual conference on Computer graphics and interactive techniques

Pages 205 - 214

https://doi.org/10.1145/311535.311558

Published: 01 July 1999 Publication History

Abstract

This paper introduces a new process, environment matting, which captures not just a foreground object and its traditional opacity matte from a real-world scene, but also a description of how that object refracts and reflects light, which we call an environment matte. The foreground object can then be placed in a new environment, using environment compositing, where it will refract and reflect light from that scene. Objects captured in this way exhibit not only specular but glossy and translucent effects, as well as selective attenuation and scattering of light according to wavelength. Moreover, the environment compositing process, which can be performed largely with texture mapping operations, is fast enough to run at interactive speeds on a desktop PC. We compare our results to photos of the same objects in real scenes. Applications of this work include the relighting of objects for virtual and augmented reality, more realistic 3D clip art, and interactive lighting design.

References

[1]

Paul Besl. Active optical range imaging sensors. In Jorge L.C. Sanz, editor, Advances in Machine Vision, chapter 1, pages 1-63. Springer-Verlag, 1989.

Digital Library

[2]

J. E Blinn and M. E. Newell. Texture and reflection in computer generated images. Communications of the ACM, 19:542-546, 1976.

Digital Library

[3]

Normand Bri6re and Pierre Poulin. Hierarchical view-dependent structures for interactive scene manipulation. In Proceedings of SIGGRAPH 96, pages 83-90, August 1996.

Digital Library

[4]

Peter J. Burt. Fast filter transforms for image processing. Computer Graphics and Image P~vcessing, 16:20-51, 1981.

[5]

Brian Carrihill and Robert Hummel. Experiments with the intensity ratio depth sensor. In Computer Vision, Graphics, and Image P~vcessing, volume 32, pages 337-358, 1985.

[6]

G. Chazan and N. Kiryati. Pyramidal intensity ratio depth sensor. Technical Report 121, Center for Communication and Information Technologies, Department of Electrical Engineering, Technion, Haifa, Israel, October 1995.

[7]

Robert L. Cook, Thomas Porter, and Loren Carpenter. Distributed ray tracing. Computer Graphics, 18(3):137-145, July 1984.

Digital Library

[8]

Franklin C. Crow. Summed-area tables for texture mapping. In P~vceedings of SIGGRAPtt 84, volume 18, pages 207-212, July 1984.

Digital Library

[9]

Paul E. Debevec and Jitendra Malik. Recovering high dynamic range radiance maps from photographs. In P1vceedings of SIGGRAPH 97, pages 369-378, August 1997.

Digital Library

[10]

Julie Dorsey, James Arvo, and Donald Greenberg. Interactive design of complex time dependent lighting. IEEE Computer Graphics and Applications, 15(2):26- 36, March 1995.

Digital Library

[11]

R. Fielding. The Technique of Special Effects Cinematography, pages 220-243. Focal/Hastings House, London, third edition, 1972.

[12]

Reid Gershbein. Cinematic Lighting in Computer Graphics. PhD thesis, Princeton University, 1999. Expected.

[13]

Andrew S. Glassner. An overview of ray tracing. In Andrew S. Glassner, editor, An Introduction to Ray Tracing, chapter 1. Academic Press, 1989.

Digital Library

[14]

Ned Greene. Environment mapping and other applications of world projections. IEEE Computer Graphics and Applications, 6(11), November 1986.

Digital Library

[15]

Ned Greene and Paul S. Heckbert. Creating raster omnimax images from multiple perspective views using the elliptical weighted average filter. IEEE Computer Graphics and Applications, 6(6):21-27, June 1986.

Digital Library

[16]

Eli Horn and Nahum Kiryati. Toward optimal structured light patterns. In P1vceedings of the International Conference on Recent Advances in Three- Dimensional Digital Imaging and Modeling, pages 28-35, 1997.

Digital Library

[17]

Douglas S. Kay and Donald R Greenberg. Transparency for computer synthesized images. In P1vceedings of SIGGRAPH 79, volume 13, pages 158-164, August 1979.

Digital Library

[18]

Gavin Miller and Marc Mondesir. Rendering hyper-sprites in real time. In G. Drettakis and N. Max, editors, P~vceedings 1998 Emvgraphics Workshop on Rendering, pages 193-198, June 1998.

[19]

Jeffry S. Nimeroff, Eero Simoncelli, and Julie Dorsey. Efficient re-rendering of naturally illuminated environments. In Fifth Emvgraphics Workshop on Rendering, pages 359-373, June 1994.

[20]

Thomas Porter and Tom Duff. Compositing digital images. In P~vceedings of SIGGRAPH 84, volume 18, pages 253-259, July 1984.

Digital Library

[21]

K. Sato and S. Inokuchi. Three-dimensional surface measurement by space encoding range imaging. Journal of Robotic Systems, 2:27-39, 1985.

[22]

Steven M. Seitz and Charles R. Dyer. View morphing: Synthesizing 3D metamorphoses using image transforms. In P1vceedings of SIGGRAPH 96, pages 21-30, August 1996.

Digital Library

[23]

Carlo H. S6quin and Eliot K. Smyrl. Parameterized ray tracing. In P~vceedings of SIGGRAPtt 89, volume 23, pages 307-314, July 1989.

Digital Library

[24]

Jean Paul Serra. Image Analysis and Mathematical Morphology. Academic Press, 1982.

Digital Library

[25]

Alvy Ray Smith and James F. Blinn. Blue screen matting. In P1vceedings of SIGGRAPtt 96, pages 259-268, August 1996.

Digital Library

[26]

Douglas Voorhies and Jim Foran. Reflection vector shading hardware. In Andrew Glassner, editor, P1vceedings of SIGGRAPH 94, pages 163-166, July 1994.

Digital Library

[27]

Gregory J. Ward. Measuring and modeling anisotropic reflection. In Edwin E. Catmull, editor, Computer Graphics (SIGGRAPH '92 P1vceedings), volume 26, pages 265-272, July 1992.

Digital Library

Cited By

Deng WCampbell DSun CKanitkar SShaffer MGould S(2024)Ray Deformation Networks for Novel View Synthesis of Refractive Objects2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00309(3106-3116)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00309
Huang KZhang FZhao JLi YDodgson N(2024)360 Stereo Image Composition With Depth AdaptionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332794330:9(6177-6191)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3327943
Lin ALin YGhosh A(2023)Practical Acquisition of Shape and Plausible Appearance of Reflective and Translucent ObjectsComputer Graphics Forum10.1111/cgf.1488942:4Online publication date: 26-Jul-2023
https://doi.org/10.1111/cgf.14889
Show More Cited By

Index Terms

Environment matting and compositing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Reconstruction
  2. Computer graphics
    1. Image manipulation
      1. Texturing
    2. Rendering

Recommendations

Environment Matting and Compositing
Seminal Graphics Papers: Pushing the Boundaries, Volume 2

This paper introduces a new process, environment matting, which captures not just a foreground object and its traditional opacity matte from a real-world scene, but also a description of how that object refracts and reflects light, which we call an ...
Environment matting extensions: towards higher accuracy and real-time capture
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques

Environment matting is a generalization of traditional bluescreen matting. By photographing an object in front of a sequence of structured light backdrops, a set of approximate light-transport paths through the object can be computed. The original ...
Shadow matting and compositing
SIGGRAPH '03: ACM SIGGRAPH 2003 Papers

In this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '99: Proceedings of the 26th annual conference on Computer graphics and interactive techniques

July 1999

463 pages

ISBN:0201485605

Chairman:
Warren Waggenspack
Louisiana State Univ., Baton Rouge

Seminal Graphics Papers: Pushing the Boundaries, Volume 2
August 2023
893 pages
ISBN:9798400708978
DOI:10.1145/3596711
Editor:
Mary C. Whitton
Department of Computer Science, UNC Chapel Hill, USA

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

ACM Press/Addison-Wesley Publishing Co.

United States

Publication History

Published: 01 July 1999

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Seminal Paper

Author Tags

Qualifiers

Article

Conference

SIGGRAPH99

Sponsor:

SIGGRAPH

SIGGRAPH99: 26th International Conference on Computer Graphics and Interactive Techniques

Acceptance Rates

SIGGRAPH '99 Paper Acceptance Rate 52 of 320 submissions, 16%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

167
Total Citations
View Citations
1,624
Total Downloads

Downloads (Last 12 months)129
Downloads (Last 6 weeks)16

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Deng WCampbell DSun CKanitkar SShaffer MGould S(2024)Ray Deformation Networks for Novel View Synthesis of Refractive Objects2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00309(3106-3116)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00309
Huang KZhang FZhao JLi YDodgson N(2024)360 Stereo Image Composition With Depth AdaptionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332794330:9(6177-6191)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3327943
Lin ALin YGhosh A(2023)Practical Acquisition of Shape and Plausible Appearance of Reflective and Translucent ObjectsComputer Graphics Forum10.1111/cgf.1488942:4Online publication date: 26-Jul-2023
https://doi.org/10.1111/cgf.14889
Li ZLong XWang YCao TWang WLuo FXiao C(2023)NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01700(18501-18511)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.01700
Wang DZhang TSüsstrunk S(2023)NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00036(317-327)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.00036
Chen XZhou ZDeng ZGhasemalizadeh OSun MKuo CSen A(2023)Tabletop Transparent Scene Reconstruction via Epipolar-Guided Optical Flow with Monocular Depth Completion Prior2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids)10.1109/Humanoids57100.2023.10375156(1-8)Online publication date: 12-Dec-2023
https://doi.org/10.1109/Humanoids57100.2023.10375156
Liu QLv XYu WGuo CZhang S(2023)Dual-context aggregation for universal image mattingMultimedia Tools and Applications10.1007/s11042-023-17517-wOnline publication date: 15-Nov-2023
https://doi.org/10.1007/s11042-023-17517-w
Sheinin MSchechner YKutulakos K(2022)Computational Imaging on the Electric GridIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2019.290303544:12(8728-8739)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TPAMI.2019.2903035
Zou ZZhao RShi TQiu SShi Z(2022)Castle in the Sky: Dynamic Sky Replacement and Harmonization in VideosIEEE Transactions on Image Processing10.1109/TIP.2022.319271731(5067-5078)Online publication date: 2022
https://doi.org/10.1109/TIP.2022.3192717
Hang ZYang Y(2022)Learning Super-Resolution of Environment Matting of Transparent Objects From a Single ImageIEEE Access10.1109/ACCESS.2022.314046610(3548-3558)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3140466
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents