article

Image-based reconstruction of spatial appearance and geometric detail

Authors:

Hendrik P. A. Lensch,

Michael Goesele,

Wolfgang Heidrich,

Hans-Peter SeidelAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 22, Issue 2

Pages 234 - 257

https://doi.org/10.1145/636886.636891

Published: 01 April 2003 Publication History

Abstract

Real-world objects are usually composed of a number of different materials that often show subtle changes even within a single material. Photorealistic rendering of such objects requires accurate measurements of the reflection properties of each material, as well as the spatially varying effects. We present an image-based measuring method that robustly detects the different materials of real objects and fits an average bidirectional reflectance distribution function (BRDF) to each of them. In order to model local changes as well, we project the measured data for each surface point into a basis formed by the recovered BRDFs leading to a truly spatially varying BRDF representation. Real-world objects often also have fine geometric detail that is not represented in an acquired mesh. To increase the detail, we derive normal maps even for non-Lambertian surfaces using our measured BRDFs. A high quality model of a real object can be generated with relatively little input data. The generated model allows for rendering under arbitrary viewing and lighting conditions and realistically reproduces the appearance of the original object.

References

[1]

Bernardini, F., Martin, I. M., and Rushmeier, H. 2001. High-quality texture reconstruction from multiple scans. IEEE Trans. Visual. Comput. Graph. 7, 4 (October-November), 318--332. ISSN 1077-2626.

[2]

Boivin, S. and Gagalowicz, A. 2001. Image-based rendering of diffuse, specular and glossy surfaces from a single image. In Proceedings of SIGGRAPH 2001, E. Fiume, Ed. Computer Graphics Proceedings, Annual Conference Series. ACM Press/ACM SIGGRAPH, 107--116. ISBN 1-58113-292-1.

[3]

Dana, K., van Ginneken, B., Nayar, S., and Koenderink, J. 1999. Reflectance and texture of real-world surfaces. ACM Trans. Graph. 18, 1 (January), 1--34.

[4]

Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the Reflectance Field of a Human Face. In Proceedings of SIGGRAPH. 145--156. ISBN 1-58113-208-5.

[5]

Debevec, P. and Malik, J. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In Proceedings of SIGGRAPH. 369--378.

[6]

Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Proceedings of SIGGRAPH. 11--20.

[7]

Forsyth, D. A. and Zisserman, A. 1989. Mutual Illumination. In Proceedings of Computer Vision and Pattern Recognition (CVPR '89). IEEE Computer Society Press, 466--473.

[8]

Furukawa, R., Kawasaki, H., Ikeuchi, K., and Sakauchi, M. 2002. Appearance based object modeling using texture database: Acquisition compression and rendering. In Thirteenth Eurographics Workshop on Rendering. 267--276.

[9]

Garland, M. and Heckbert, P. 1997. Surface Simplification Using Quadric Error Metrics. In Proceedings of SIGGRAPH (August). 209--216.

[10]

Gersho, A. and Gray, R. 1992. Vector Quantization and Signal Compression. Kluwer Academic Publishers.

[11]

Gibson, S., Howard, T., and Hubbold, R. 2001. Flexible image-based photometric reconstruction using virtual light sources. Comput. Graph. Forum 20, 3. ISSN 1067--7055.

[12]

Goesele, M., Heidrich, W., Lensch, H. P., and Seidel, H.-P. 2000. Building a photo studio for measurement purposes. In Proceedings of Vision, Modeling, and Visualization. 231--238.

[13]

Goesele, M., Heidrich, W., and Seidel, H.-P. 2001. Entropy based dark frame subtraction. In Image Processing, Image Quality, Image Capture Systems Conference (PICS). 293--298.

[14]

Gortler, S., Grzeszczuk, R., Szelinski, R., and Cohen, M. 1996. The Lumigraph. In Proceedings of SIGGRAPH. 43--54.

[15]

Kautz, J. and Seidel, H.-P. August 2000. Towards Interactive Bump Mapping with Anisotropic Shift-Variant BRDFs. In Eurographics/SIGGRAPH Hardware Workshop. 51--58.

[16]

Kobbelt, L. 1996. Discrete fairing. In Proceedings of the 7th IMA Conference on the Mathematics of Surfaces. 101--131.

[17]

Koenderink, J., van Doorn, A., and Stavridi, M. 1996. Bidirectional Reflection Distribution Function expressed in terms of surface scattering modes. In Proceedings of the 4th Europ. Conf. on Computer Vision. 28--39.

[18]

Lafortune, E., Foo, S., Torrance, K., and Greenberg, D. 1997. Non-Linear Approximation of Reflectance Functions. In Proceedings of SIGGRAPH. 117--126.

[19]

Lensch, H., Heidrich, W., and Seidel, H.-P. 2000. Automated Texture Registration and Stitching for Real World Models. In Pacific Graphics '00. 317--326.

[20]

Lensch, H., Heidrich, W., and Seidel, H.-P. 2001a. Silhouette-based algorithm for texture registration and stitching. Graphical Models 63, 4 (July), 245--262.

[21]

Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2001b. Image-based reconstruction of spatially varying materials. In Rendering Techniques 2001: 12th Eurographics Workshop on Rendering. Eurographics, 103--114. ISBN 3-211-83709-4.

[22]

Levoy, M. and Hanrahan, P. 1996. Light Field Rendering. In Proceedings of SIGGRAPH. 31--42.

[23]

Lewis, R. 1993. Making shaders more physically plausible. In 4th Eurographics Workshop on Rendering. 47--62.

[24]

Liu, X., Yu, Y., and Shum, H.-Y. 2001. Synthesizing bidirectional texture functions for real-world surfaces. In Proceedings of ACM SIGGRAPH 2001. Computer Graphics Proceedings, Annual Conference Series. ACM Press/ACM SIGGRAPH, 97--106. ISBN 1-58113-292-1.

[25]

Lloyd, S. 1982. Least squares quantization in PCM. IEEE Trans. Inform. Theory IT-28, 129--137.

[26]

Lu, R., Koenderink, J., and Kappers, A. 1998. Optical Properties (bidirectional reflectance distribution functions) of velvet. Applied Optics 37, 25 (Sept.), 5974--5984.

[27]

MacQueen, J. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability. Vol. 1.

[28]

Malzbender, T., Gelb, D., and Wolters, H. 2001. Polynomial texture maps. In Proceedings of ACM SIGGRAPH 2001. Computer Graphics Proceedings, Annual Conference Series. ACM Press/ACM SIGGRAPH, 519--528. ISBN 1-58113-292-1.

[29]

Marschner, S. 1998. Inverse rendering for computer graphics. Ph.D. thesis, Cornell University.

[30]

Marschner, S., Westin, S., Lafortune, E., Torrance, K., and Greenberg, D. 1999. Image-based BRDF Measurement Including Human Skin. In 10th Eurographics Workshop on Rendering. 131--144.

[31]

Matusik, W., Pfister, H., Ngan, A., Beardsley, P., and McMillan, L. 2002a. Image-Based 3D Photography Using Opacity Hulls. In Proceedings of SIGGRAPH. 427--437.

[32]

Matusik, W., Pfister, H., Ziegler, R., Ngan, A., and McMillan, L. 2002b. Acquisition and Rendering of Transparent and Refractive Objects. In Thirteenth Eurographics Workshop on Rendering. 277--288.

[33]

McAllister, D. 2002. A Generalized Representation of Surface Appearance. Ph.D. thesis, University of North Carolina.

[34]

McAllister, D., Lastra, A., and Heidrich, W. 2002. Efficient Rendering of Spatial Bi-directional Reflectance Distribution Functions. In Proceedings of Conference on Graphics Hardware.

[35]

Miller, G., Rubin, S., and Ponceleon, D. 1998. Lazy decompression of surface light fields for precomputed global illumination. In 9th Eurographics Workshop on Rendering. 281--292.

[36]

Nayar, S., Ikeuchi, K., and Kanade, T. 1991. Surface reflection: physical and geometrical perspectives. IEEE Trans. PAMI 13, 7, 611--634.

[37]

Nayar, S. K., Ikeuchi, K., and Kanade, T. 1990. Shape from Interreflections. In International Conference on Computer Vision (ICCV '90). Osaka, Japan, 2--11.

[38]

Press, W., Teukolsky, S., Vetterling, W., and Flannery, B. 1992. Numerical Recipes in C: The Art of Scientific Computing (2nd ed.). Cambridge University Press. ISBN 0-521-43108-5.

[39]

Ramamoorthi, R. and Hanrahan, P. 2001. A signal-processing framework for inverse rendering. In Proceedings of SIGGRAPH 2001, E. Fiume, Ed. Computer Graphics Proceedings, Annual Conference Series. ACM Press/ACM SIGGRAPH, 117--128. ISBN 1-58113-292-1.

[40]

Robertson, M. A., Borman, S., and Stevenson, R. L. 1999. Dynamic Range Improvement Through Multiple Exposures. In Proceedings of the International Conference on Image Processing (ICIP'99). IEEE, 159--163.

[41]

Rushmeier, H., Bernardini, F., Mittleman, J., and Taubin, G. 1998. Acquiring input for rendering at appropriate levels of detail: Digitizing a piet&abrave;. Eurographics Rendering Workshop 1998, 81--92. ISBN 3-211-83213-0. Held in Vienna, Austria.

[42]

Rushmeier, H., Taubin, G., and Guéziec, A. 1997. Applying Shape from Lighting Variation to Bump Map Capture. In 8th Eurographics Workshop on Rendering Workshop. 35--44.

[43]

Sato, Y., Wheeler, M., and Ikeuchi, K. 1997. Object Shape and Reflectance Modeling from Observation. In Proceedings of SIGGRAPH (August). 379--388.

[44]

Schirmacher, H., Heidrich, W., Rubick, M., Schiron, D., and Seidel, H.-P. 1999. Image-Based BRDF Reconstruction. In Proceedings of the 4th VMV Conference. 285--292.

[45]

Torrance, K. and Sparrow, E. 1967. Theory for off-specular reflection from roughened surfaces. J. Optic. Soc. Amer. 57, 9.

[46]

Ward Larson, G. 1992. Measuring and Modeling Anisotropic Reflection. In Proceedings of SIGGRAPH. 265--272.

[47]

Westin, S., Arvo, J., and Torrance, K. 1992. Predicting Reflectance Functions From Complex Surfaces. In Proceedings of SIGGRAPH. 255--264.

[48]

Wood, D., Azuma, D., Aldinger, K., Curless, B., Duchamp, T., Salesin, D., and Stuetzle, W. 2000. Surface Light Fields for 3D Photography. In Proceedings of SIGGRAPH. 287--296.

[49]

Yu, Y., Debevec, P., Malik, J., and Hawkins, T. 1999. Inverse Global Illumination: Recovering Reflectance Models of Real Scenes From Photographs. In Proceedings of SIGGRAPH. 215--224.

[50]

Yu, Y. and Malik, J. July 1998. Recovering Photometric Properties of Architectural Scenes from Photographs. In Proceedings of SIGGRAPH. 207--218.

[51]

Zhang, R., Tsai, P.-S., Cryer, J., and Shah, M. 1999. Shape from Shading: A Survey. IEEE Trans. Pattern Analy. Mach. Intelli. 21, 8 (August), 690--706.

[52]

Zhang, Z. 1999. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. In Proceedings of the International Conference on Computer Vision. 666--673.

Cited By

Luo XScandolo LBousseau AEisemann E(2024)Single‐Image SVBRDF Estimation with Learned Gradient DescentComputer Graphics Forum10.1111/cgf.1501843:2Online publication date: 23-Apr-2024
https://doi.org/10.1111/cgf.15018
Ma XYu YWu HZhou K(2024)Efficient Reflectance Capture With a Deep Gated Mixture-of-ExpertsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.326187230:7(4246-4256)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3261872
Tetzlaff M(2024)High-Fidelity Specular SVBRDF Acquisition From Flash PhotographsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.323527730:4(1885-1896)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3235277
Show More Cited By

Index Terms

Image-based reconstruction of spatial appearance and geometric detail
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
  2. Computer graphics
    1. Image manipulation
      1. Texturing

Recommendations

An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods

This paper provides an introduction to photometric methods for image-based 3D shape reconstruction and a survey of photometric stereo techniques. We begin with taxonomy of active and passive shape acquisition techniques. Then we describe the methodical ...
Reduction of rank-reduced orientation-from-color problem with many unknown lights to two-image known-illuminant photometric stereo
ISCV '95: Proceedings of the International Symposium on Computer Vision

How does shape from shading proceed when two or more differently colored light sources are present? For a uniformly colored Lambertian surface illuminated by a collection of point or extended light sources or inter reflections, with unknown directions ...
Generalization of the Lambertian model and implications for machine vision

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 22, Issue 2

April 2003

128 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/636886

Issue’s Table of Contents

Copyright © 2003 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2003

Published in TOG Volume 22, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

263
Total Citations
View Citations
2,885
Total Downloads

Downloads (Last 12 months)68
Downloads (Last 6 weeks)7

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Luo XScandolo LBousseau AEisemann E(2024)Single‐Image SVBRDF Estimation with Learned Gradient DescentComputer Graphics Forum10.1111/cgf.1501843:2Online publication date: 23-Apr-2024
https://doi.org/10.1111/cgf.15018
Ma XYu YWu HZhou K(2024)Efficient Reflectance Capture With a Deep Gated Mixture-of-ExpertsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.326187230:7(4246-4256)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3261872
Tetzlaff M(2024)High-Fidelity Specular SVBRDF Acquisition From Flash PhotographsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.323527730:4(1885-1896)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3235277
Cheng TMa WGuan KTorralba AWang SOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Structure from duplicatesProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666874(17195-17207)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666874
Ma XXu XZhang LZhou KWu H(2023)OpenSVBRDF: A Database of Measured Spatially-Varying ReflectanceACM Transactions on Graphics10.1145/361835842:6(1-14)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618358
Wang LZhang LGao FZhang J(2023)DeepBasis: Hand-Held Single-Image SVBRDF Capture via Two-Level Basis Material ModelSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618239(1-11)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618239
Liu CFischer MRitschel T(2023)Learning to Learn and Sample BRDFsComputer Graphics Forum10.1111/cgf.1475442:2(201-211)Online publication date: 23-May-2023
https://doi.org/10.1111/cgf.14754
Kang KGu MXie CYang XWu HZhou K(2023)Neural Reflectance Capture in the View-Illumination DomainIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.311737029:2(1450-1462)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TVCG.2021.3117370
Schmitt CAntić BNeculai ALee JGeiger A(2023)Towards Scalable Multi-View Reconstruction of Geometry and MaterialsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.331434845:12(15850-15869)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3314348
Zhang YXu TYu JYe YJing YWang JYu JYang W(2023)NeMF: Inverse Volume Rendering with Neural Microflake Field2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.02095(22862-22872)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.02095
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents