Article

Acquisition and rendering of transparent and refractive objects

Authors:

Wojciech Matusik,

Hanspeter Pfister,

Leonard McMillanAuthors Info & Claims

EGRW '02: Proceedings of the 13th Eurographics workshop on Rendering

Pages 267 - 278

Published: 26 July 2002 Publication History

Abstract

This paper introduces a new image-based approach to capturing and modeling highly specular, transparent, or translucent objects. We have built a system for automatically acquiring high quality graphical models of objects that are extremely difficult to scan with traditional 3D scanners. The system consists of turntables, a set of cameras and lights, and monitors to project colored backdrops. We use multi-background matting techniques to acquire alpha and environment mattes of the object from multiple viewpoints. Using the alpha mattes we reconstruct an approximate 3D shape of the object. We use the environment mattes to compute a high-resolution surface reflectance field. We also acquire a low-resolution surface reflectance field using the overhead array of lights. Both surface reflectance fields are used to relight the objects and to place them into arbitrary environments. Our system is the first to acquire and render transparent and translucent 3D objects, such as a glass of beer, from arbitrary viewpoints under novel illumination.

References

[1]

Active Optical Range Imaging Sensors. Advances in Machine Vision. Springer Verlag, 1989.

[2]

J. Blinn and M. Newell. Texture and Reflection in Computer Generated Images. Communications of the ACM (SIGGRAPH '76 Proceedings), 19(10):542--547, October 1976.

Digital Library

[3]

C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen. Unstructured Lumigraph Rendering. In Computer Graphics, SIGGRAPH 2001 Proceedings, pages 425--432. Los Angeles, CA, 2001.

Digital Library

[4]

B. Cabral, M. Olano, and P. Nemec. Reflection Space Image Based Rendering. In Computer Graphics, SIGGRAPH '99 Proceedings, pages 165--170. Los Angeles, CA, August 1999.

Digital Library

[5]

W.-C. Chen, R. Grzeszczuk, and J.-Y. Bouguet. Light Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields. In Computer Graphics, To appear in the SIGGRAPH 2002 Proceedings. San Antonio, TX, July 2002.

Digital Library

[6]

Y.-Y. Chuang, D. Zongker, J. Hindorff, B. Curless, D. Salesin, and R. Szeliski. Environment Matting Extensions: Towards Higher Accuracy and Real-Time Capture. In Computer Graphics, SIGGRAPH 2000 Proceedings, pages 121--130. 2000.

Digital Library

[7]

K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink. Reflectance and Texture of Real World Surfaces. ACM Transactions on Graphics, 1(18):1--34, 1999.

Digital Library

[8]

P. Debeyec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar. Acquiring the Reflectance Field of a Human Face. In Computer Graphics, SIGGRAPH 2000 Proceedings, pages 145--156. July 2000.

Digital Library

[9]

P. Debevec and J. Malik. Recovering High Dynamic Range Radiance Maps From Photographs. In Computer Graphics, SIGGRAPH 97 Proceedings, pages 369--378. Los Angeles, CA, 1997.

Digital Library

[10]

P. Debevec, Y. Yu, and G. Borshukov. Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping. In Proceedings of the 9th Eurographics Workshop on Rendering, pages 105--116. Vienna, Austria, June 1998.

[11]

J. Dorsey, A. Edelman, J. Legakis, H. W. Jensen, and H. K. Pedersen. Modeling and Rendering of Weathered Stone. In Computer Graphics, SIGGRAPH '99 Proceedings, pages 225--234. Los Angeles, CA, August 1999.

Digital Library

[12]

S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen. The Lumigraph. In Computer Graphics, SIGGRAPH 96 Proceedings, pages 43--54. New Orleans, LS, August 1996.

Digital Library

[13]

Z. Hakura and J. Snyder. Realistic Reflections and Refractions on Graphics Hardware with Hybrid Rendering and Layered Environment Maps. In Proceedings of the 12th Eurographics Workshop on Rendering. London, UK, June 2001.

Digital Library

[14]

Z. Hakura, J. Snyder, and J. Lengyel. Parameterized Environment Maps. In Symposium on Interactive 3D Graphics, pages 203--208. March 2001.

Digital Library

[15]

T. Hawkins, J. Cohen, and P. Debevec. A Photometric Approach to Digitizing Cultural Artifacts. In 2nd International Symposium on Virtual Reality, Archaeology, and Cultural Heritage. Glyfada, Greece, November 2001.

Digital Library

[16]

W. Heidrich, H. Lensch, M. Cohen, and H. Seidel. Light Field Techniques for Reflections and Refractions. In Rendering Techniques '99, pages 187--196. Springer, Wien, Granda, Spain, June 1999.

[17]

W. Heidrich and H.-P. Seidel. Realistic, Hardware-Accelerated Shading and Lighting. In Computer Graphics, SIGGRAPH '99 Proceedings, pages 171--178. Los Angeles, CA, August 1999.

Digital Library

[18]

H. W. Jensen. Realistic Image Synthesis Using Photon Mapping. AK Peters, 2001. ISBN: 1568811470.

Digital Library

[19]

H. W. Jensen, S. R. Marschner, M. Levoy, and P. Hanrahan. A Practical Model for Subsurface Light Transport. In Computer Graphics, SIGGRAPH 2001 Proceedings, pages 511--518. Los Angeles, CA, August 2001.

Digital Library

[20]

A. Laurentini. The Visual hull Concept for Silhouette-Based Image Understanding. PAMI, 16(2):150--162, February 1994.

Digital Library

[21]

H. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel. Image-Based Reconstruction of Spatially Varying Materials. In Proceedings of the 12th Eurographics Workshop on Rendering. June 2001.

Digital Library

[22]

M. Levoy and P. Hanrahan. Light Field Rendering. In Computer Graphics, SIGGRAPH 96 Proceedings, pages 31--42. New Orleans, LS, August 1996.

Digital Library

[23]

S. Marschner, S. Westin, E. Lafortune, K. Torrance, and D. Greenberg. Image-based BRDF Measurement Including Human Skin. In Proceedings of the 10th Eurographics Workshop on Rendering, pages 139--152. Granada, Spain, June 1999.

Digital Library

[24]

W. Matusik, C. Buehler, R. Raskar, S. Gortler, and L. McMillan. Image-Based Visual Hulls. In Computer Graphics, SIGGRAPH 2000 Proceedings, pages 369--374. Los Angeles, CA, July 2000.

Digital Library

[25]

W. Matusik, H. Pfister, A. Ngan, P. Beardsley, R. Ziegler, and L. McMillan. Image-Based 3D Photography using Opacity Hulls. In Computer Graphics, To appear in the SIGGRAPH 2002 Proceedings. San Antonio, TX, July 2002. Available as MERL TR after April 15.

Digital Library

[26]

F. Nicodemus, J. Richmond, J. Hsia, I. Ginsberg, and T. Limperis. Geometric Considerations and Nomenclature for Reflectance. National Bureau of Standards (US) Monograph 161, 1977.

[27]

R. Grzeszczuk (organizer). Acquisition and Visualization of Surface Light Fields. In Course Notes, SIGGRAPH 2001. 2001.

[28]

E. Paquette, P. Poulin, and G. Drettakis. Surface Aging by Impacts. In Graphics Interface 2001, pages 175--182. June 2001.

Digital Library

[29]

H. Pfister, M. Zwicker, J. van Baar, and M Gross. Surfels: Surface Elements as Rendering Primitives. In Computer Graphics, SIGGRAPH 2000 Proceedings, pages 335--342. Los Angeles, CA, July 2000.

Digital Library

[30]

M. Pharr and P. Hanrahan. Monte Carlo Evaluation of Non-Linear Scattering Equations for Subsurface Reflection. In Computer Graphics, SIGGRAPH 2000 Proceedings, pages 75--84. New Orleans, LS, July 2000.

Digital Library

[31]

K. Pulli, M. Cohen, T. Duchamp, H. Hoppe, L. Shapiro, and W. Stuetzle. View-Based Rendering: Visualizing Real Objects from Scanned Range and Color Data. In Eurographics Rendering Workshop 1997, pages 23--34. June 1997.

Digital Library

[32]

H. Rushmeier. 3D Capture for Computer Graphics. In Third International Conference on 3D Digital Imaging and Modeling. Quebec City, Canada, May 28 -- June 1 2001.

[33]

H. E. Rushmeier. Realistic Image Synthesis for Scenes with Radiatively Participating Media. Ph.d. thesis, Program of Computer Graphics, Cornell University, Ithaca, NY, 1988.

Digital Library

[34]

Y. Sato, M. D. Wheeler, and K. Ikeuchi. Object Shape and Reflectance Modeling from Observation. In Computer Graphics, SIGGRAPH 97 Proceedings, pages 379--387. 1997.

Digital Library

[35]

P. Shirley, H. Hu, B. Smits, and E. Lafortune. A Practitioners' Assessment of Light Reflection Models. In Proceedings of Pacific Graphics, pages 40--49. October 1997.

Digital Library

[36]

A. R. Smith and J. F. Blinn. Blue Screen Matting. In Computer Graphics, volume 30 of SIGGRAPH 96 Proceedings, pages 259--268. 1996.

Digital Library

[37]

J. Stam. Multiple Scattering as a Diffusion Process. In Proceedings of the 6th Eurographics Workshop on Rendering, pages 51--58. Dublin, Ireland, June 1995.

[38]

T. Whitted. An Improved Illumination Model for Shaded Display. Communications of the ACM, 6(23):343--349, June 1980.

Digital Library

[39]

D. Wood, D. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle. Surface Light Fields for 3D Photography. In Computer Graphics, SIGGRAPH 2000 Proceedings, pages 287--296. Los Angeles, CA, July 2000.

Digital Library

[40]

Y. Yu, P. Debevec, J. Malik, and T. Hawkins. Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs. In Computer Graphics, SIGGRAPH 99 Proceedings, pages 215--224. Los Angeles, CA, August 1999.

Digital Library

[41]

D. Zongker, D. Werner, B. Curless, and D. Salesin. Environment Matting and Compositing. In Computer Graphics, SIGGRAPH 99 Proceedings, pages 205--214. August 1999.

Digital Library

[42]

M. Zwicker, H. Pfister., J. Van Baar, and M. Gross. Surface Splatting. In Computer Graphics, SIGGRAPH 2001 Proceedings, pages 371--378. Los Angeles, CA, July 2001.

Digital Library

Cited By

Pan JSu JHsiao KYen TChu H(2022)Sampling Neural Radiance Fields for Refractive ObjectsSIGGRAPH Asia 2022 Technical Communications10.1145/3550340.3564234(1-4)Online publication date: 6-Dec-2022
https://dl.acm.org/doi/10.1145/3550340.3564234
Weinmann MKlein R(2015)Advances in geometry and reflectance acquisition (course notes)SIGGRAPH Asia 2015 Courses10.1145/2818143.2818165(1-71)Online publication date: 2-Nov-2015
https://dl.acm.org/doi/10.1145/2818143.2818165
Yeung STang CBrown MKang S(2011)Matting and compositing of transparent and refractive objectsACM Transactions on Graphics10.1145/1899404.189940630:1(1-13)Online publication date: 2-Feb-2011
https://dl.acm.org/doi/10.1145/1899404.1899406
Show More Cited By

Index Terms

Acquisition and rendering of transparent and refractive objects
1. Computer systems organization
  1. Architectures
    1. Other architectures
      1. Special purpose systems
2. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Graphics file formats

Recommendations

Matting and compositing of transparent and refractive objects

This article introduces a new approach for matting and compositing transparent and refractive objects in photographs. The key to our work is an image-based matting model, termed the Attenuation-Refraction Matte (ARM), that encodes plausible refractive ...
Reflective and Refractive Objects for Mixed Reality

In this paper, we present a novel rendering method which integrates reflective or refractive objects into a differential instant radiosity (DIR) framework usable for mixed-reality (MR) applications. This kind of objects are very special from the light ...
Interactive relighting of dynamic refractive objects

We present a new technique for interactive relighting of dynamic refractive objects with complex material properties. We describe our technique in terms of a rendering pipeline in which each stage runs entirely on the GPU. The rendering pipeline ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EGRW '02: Proceedings of the 13th Eurographics workshop on Rendering

July 2002

336 pages

ISBN:1581135343

Editors:
Simon Gibson
University of Manchester
,
Paul Debevec
USC Institute for Creative Technologies

Sponsors

EUROGRAPHICS: The European Association for Computer Graphics

Publisher

Eurographics Association

Goslar, Germany

Publication History

Published: 26 July 2002

Check for updates

Qualifiers

Article

Conference

EGRW02

Sponsor:

EUROGRAPHICS

EGRW02: Eurographics Workshop on Rendering

June 26 - 28, 2002

Pisa, Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
779
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pan JSu JHsiao KYen TChu H(2022)Sampling Neural Radiance Fields for Refractive ObjectsSIGGRAPH Asia 2022 Technical Communications10.1145/3550340.3564234(1-4)Online publication date: 6-Dec-2022
https://dl.acm.org/doi/10.1145/3550340.3564234
Weinmann MKlein R(2015)Advances in geometry and reflectance acquisition (course notes)SIGGRAPH Asia 2015 Courses10.1145/2818143.2818165(1-71)Online publication date: 2-Nov-2015
https://dl.acm.org/doi/10.1145/2818143.2818165
Yeung STang CBrown MKang S(2011)Matting and compositing of transparent and refractive objectsACM Transactions on Graphics10.1145/1899404.189940630:1(1-13)Online publication date: 2-Feb-2011
https://dl.acm.org/doi/10.1145/1899404.1899406
O'Toole MKutulakos K(2010)Optical computing for fast light transport analysisACM Transactions on Graphics10.1145/1882261.186616529:6(1-12)Online publication date: 15-Dec-2010
https://dl.acm.org/doi/10.1145/1882261.1866165
O'Toole MKutulakos KDrettakis G(2010)Optical computing for fast light transport analysisACM SIGGRAPH Asia 2010 papers10.1145/1866158.1866165(1-12)Online publication date: 15-Dec-2010
https://dl.acm.org/doi/10.1145/1866158.1866165
Yamazaki MIwata SXu G(2007)Dense 3D reconstruction of specular and transparent objects using stereo cameras and phase-shift methodProceedings of the 8th Asian conference on Computer vision - Volume Part II10.5555/1775728.1775796(570-579)Online publication date: 18-Nov-2007
https://dl.acm.org/doi/10.5555/1775728.1775796
Lehtinen J(2007)A framework for precomputed and captured light transportACM Transactions on Graphics10.1145/1289603.128960426:4(13-es)Online publication date: 1-Oct-2007
https://dl.acm.org/doi/10.1145/1289603.1289604
Einarsson PChabert CJones AMa WLamond BHawkins TBolas MSylwan SDebevec P(2006)Relighting human locomotion with flowed reflectance fieldsProceedings of the 17th Eurographics conference on Rendering Techniques10.5555/2383894.2383917(183-194)Online publication date: 26-Jun-2006
https://dl.acm.org/doi/10.5555/2383894.2383917
Trifonov BBradley DHeidrich W(2006)Tomographic reconstruction of transparent objectsProceedings of the 17th Eurographics conference on Rendering Techniques10.5555/2383894.2383902(51-60)Online publication date: 26-Jun-2006
https://dl.acm.org/doi/10.5555/2383894.2383902
Green PKautz JMatusik WDurand FOlano MSéquin C(2006)View-dependent precomputed light transport using nonlinear Gaussian function approximationsProceedings of the 2006 symposium on Interactive 3D graphics and games10.1145/1111411.1111413(7-14)Online publication date: 14-Mar-2006
https://dl.acm.org/doi/10.1145/1111411.1111413
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

Media

Figures

Other

Tables

View Table of Contents