research-article

Appearance capture and modeling of human teeth

Authors:

Zdravko Velinov,

Parsa Mirdehghan,

Steve Marschner,

Thabo BeelerAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 37, Issue 6

Article No.: 207, Pages 1 - 13

https://doi.org/10.1145/3272127.3275098

Published: 04 December 2018 Publication History

Abstract

Recreating the appearance of humans in virtual environments for the purpose of movie, video game, or other types of production involves the acquisition of a geometric representation of the human body and its scattering parameters which express the interaction between the geometry and light propagated throughout the scene. Teeth appearance is defined not only by the light and surface interaction, but also by its internal geometry and the intra-oral environment, posing its own unique set of challenges. Therefore, we present a system specifically designed for capturing the optical properties of live human teeth such that they can be realistically re-rendered in computer graphics. We acquire our data in vivo in a conventional multiple camera and light source setup and use exact geometry segmented from intra-oral scans. To simulate the complex interaction of light in the oral cavity during inverse rendering we employ a novel pipeline based on derivative path tracing with respect to both optical properties and geometry of the inner dentin surface. The resulting estimates of the global derivatives are used to extract parameters in a joint numerical optimization. The final appearance faithfully recreates the acquired data and can be directly used in conventional path tracing frameworks for rendering virtual humans.

Supplementary Material

ZIP File (a207-velinov.zip)

Supplemental files.

Download
113.71 MB

MP4 File (a207-velinov.mp4)

Download
113.97 MB

References

[1]

Jae Sung Ahn, Anjin Park, Ju Wan Kim, Byeong Ha Lee, and Joo Beom Eom. 2017. Development of Three-Dimensional Dental Scanning Apparatus Using Structured Illumination. In Sensors.

[2]

F. O. Bartell, E. L. Dereniak, and W. L. Wolfe. 1981. The theory and measurement of bidirectional reflectance distribution function BRDF and bidirectional transmittance distribution function BTDF. In Radiation scattering in optical systems, G. H. Hunt (Ed.), Vol. 257. 154--160.

[3]

Thabo Beeler. 2012. Passive Spatio-Temporal Geometry Reconstruction of Human Faces at Very High Fidelity. Ph.D. Dissertation. ETH Zurich / Disney Research Zurich. http://graphics.ethz.ch/~dbeeler/research/bee12c/PassiveSpatioTemporalGeometryReconstructionOfHumanFacesAtVeryHighFidelity.pdf

[4]

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. 2004. High accuracy optical flow estimation based on a theory for warping. In ECCV. 25--36.

[5]

Stéphanie I. Buchaillard, S. H. Ong, Yohan Payan, and Kelvin Foong. 2007. 3D Statistical Models for Tooth Surface Reconstruction. Comput. Biol. Med. 37, 10 (2007), 1461--1471.

Digital Library

[6]

C.N. Carter, R.J. Pusateri, Dongqing Chen, A.H. Ahmed, and A.A. Farag. 2010. Shape from shading for hybrid surfaces as applied to tooth reconstruction. In IEEE ICIP. 4049--4052.

[7]

S. Chandrasekhar. 1960. Radiative Transfer. Dover Publications.

[8]

Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. 2000. Acquiring the Reflectance Field of a Human Face. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '00). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 145--156.

Digital Library

[9]

Eugene d'Eon and Geoffrey Irving. 2011. A quantized-diffusion model for rendering translucent materials. 30, 4 (July 2011), 56:1--56:14.

Digital Library

[10]

Eustace L. Dereniak, Langford G. Brod, and John E. Hubbs. 1982. Bidirectional transmittance distribution function measurements on ZnSe. Appl. Opt. 21, 24 (Dec 1982), 4421--4425.

[11]

Craig Donner and Henrik Wann Jensen. 2005. Light Diffusion in Multi-layered Translucent Materials. In ACM SIGGRAPH 2005 Papers (SIGGRAPH '05). ACM, New York, NY, USA, 1032--1039.

Digital Library

[12]

Craig Donner and Henrik Wann Jensen. 2008. Rendering Translucent Materials Using Photon Diffusion. In ACM SIGGRAPH 2008 Classes (SIGGRAPH '08). ACM, New York, NY, USA, Article 4, 9 pages.

Digital Library

[13]

Craig Donner, Tim Weyrich, Eugene d'Eon, Ravi Ramamoorthi, and Szymon Rusinkiewicz. 2008. A Layered, Heterogeneous Reflectance Model for Acquiring and Rendering Human Skin. In ACM SIGGRAPH Asia 2008 Papers (SIGGRAPH Asia '08). ACM, New York, NY, USA, Article 140, 12 pages.

Digital Library

[14]

David Eberly. 2015. Distance Between Point and Triangle in 3D. Technical Report. https://www.geometrictools.com/Documentation/DistancePoint3Triangle3.pdf

[15]

Aly Farag, Shireen Elhabian, Aly Abdelrehim, Wael Aboelmaaty, Allan Farman, and David Tasman. 2013. Model-Based Human Teeth Shape Recovery from a Single Optical Image with Unknown Illumination. In Medical Computer Vision: Recognition Techniques and Applications in Medical Imaging (MCV '12). 263--272.

Digital Library

[16]

Roald Frederickx and Philip Dutré. 2017. A Forward Scattering Dipole Model from a Functional Integral Approximation. ACM Trans. Graph. 36, 4, Article 109 (July 2017), 13 pages.

Digital Library

[17]

Daniel Fried, Richard E. Glena, John D. B. Featherstone, and Wolf Seka. 1995. Nature of light scattering in dental enamel and dentin at visible and near-infrared wavelengths. Journal of Applied Optics 34, 7 (1995), 1278--1285.

[18]

Jeppe Revall Frisvad, Toshiya Hachisuka, and Thomas Kim Kjeldsen. 2014. Directional Dipole Model for Subsurface Scattering. ACM Trans. Graph. 34, 1, Article 5 (Dec. 2014), 12 pages.

Digital Library

[19]

S. Garrido-Jurado, R. Mu noz Salinas, F.J. Madrid-Cuevas, and M.J. Marín-Jiménez. 2014. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47, 6 (2014), 2280 -- 2292.

Digital Library

[20]

Abhijeet Ghosh, Tim Hawkins, Pieter Peers, Sune Frederiksen, and Paul Debevec. 2008. Practical modeling and acquisition of layered facial reflectance. In ACM Transactions on Graphics (TOG), Vol. 27. ACM, 139.

Digital Library

[21]

Ioannis Gkioulekas, Anat Levin, and Todd Zickler. 2016. An Evaluation of Computational Imaging Techniques for Heterogeneous Inverse Scattering., 685--701 pages.

[22]

Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse Volume Rendering with Material Dictionaries. ACM Trans. Graph. 32, 6, Article 162 (Nov. 2013), 13 pages.

Digital Library

[23]

D. Guarnera, G.C. Guarnera, A. Ghosh, C. Denk, and M. Glencross. 2016. BRDF Representation and Acquisition. Comput. Graph. Forum 35, 2 (May 2016), 625--650.

[24]

Ralf Habel, Per H. Christensen, and Wojciech Jarosz. 2013. Photon Beam Diffusion: A Hybrid Monte Carlo Method for Subsurface Scattering. 32, 4 (June 2013).

[25]

Milovš Hašan and Ravi Ramamoorthi. 2013. Interactive Albedo Editing in Path-traced Volumetric Materials. ACM Trans. Graph. 32, 2, Article 11 (April 2013), 11 pages.

Digital Library

[26]

L. G. Henyey and J. L. Greenstein. 1941. Diffuse radiation in the Galaxy. Astrophys. J. 93 (1941), 70--83.

[27]

Nicolas Holzschuch. 2015. Accurate computation of single scattering in participating media with refractive boundaries. Computer Graphics Forum 34, 6 (Sept. 2015), 48--59.

Digital Library

[28]

A. Ilie and G. Welch. 2005. Ensuring color consistency across multiple cameras. In Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, Vol. 2. 1268--1275 Vol. 2.

Digital Library

[29]

Wenzel Jakob. 2010. Mitsuba renderer. http://mitsuba-renderer.org.

[30]

Wenzel Jakob, Adam Arbree, Jonathan T. Moon, Kavita Bala, and Steve Marschner. 2010. A Radiative Transfer Framework for Rendering Materials with Anisotropic Structure. In ACM SIGGRAPH 2010 Papers (SIGGRAPH '10). ACM, New York, NY, USA, Article 53, 13 pages.

Digital Library

[31]

Wenzel Jakob and Steve Marschner. 2012. Manifold exploration: a Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport. 31, 4 (July 2012), 58:1--58:13.

Digital Library

[32]

Henrik Wann Jensen and Per H. Christensen. 1998. Efficient simulation of light transport in scenes with participating media using photon maps. In 98 (Annual Conference Series). ACM, New York, NY, USA, 311--320.

Digital Library

[33]

Henrik Wann Jensen, Stephen R Marschner, Marc Levoy, and Pat Hanrahan. 2001. A practical model for subsurface light transport. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, 511--518.

Digital Library

[34]

James T. Kajiya and Brian P Von Herzen. 1984. Ray tracing volume densities. 18, 3 (Jan. 1984), 165--174.

Digital Library

[35]

Masahide Kawai, Tomoyori Iwao, Akinobu Maejima, and Shigeo Morishima. 2014. Automatic Photorealistic 3D Inner Mouth Restoration from Frontal Images., 51--62 pages.

[36]

Pramook Khungurn, Daniel Schroeder, Shuang Zhao, Kavita Bala, and Steve Marschner. 2015. Matching Real Fabrics with Micro-Appearance Models. ACM Trans. Graph. 35, 1, Article 1 (Dec. 2015), 26 pages.

Digital Library

[37]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980

[38]

Oliver Klehm, Fabrice Rousselle, Marios Papas, Derek Bradley, Christophe Hery, Bernd Bickel, Wojciech Jarosz, and Thabo Beeler. 2015. Recent Advances in Facial Appearance Capture. Comput. Graph. Forum 34, 2 (May 2015), 709--733.

Digital Library

[39]

Eric P. Lafortune and Yves D. Willems. 1996. Rendering participating media with bidirectional path tracing. Springer-Verlag, London, UK, 91--100.

Digital Library

[40]

Christian Thode Larsen, Jeppe Revall Frisvad, Peter Dahl Ejby Jensen, and Jakob Andreas Bærentzen. 2012. Real-Time Rendering of Teeth with No Preprocessing. In Advances in Visual Computing, George Bebis, Richard Boyle, Bahram Parvin, Darko Koracin, Charless Fowlkes, Sen Wang, Min-Hyung Choi, Stephan Mantler, Jürgen Schulze, Daniel Acevedo, Klaus Mueller, and Michael Papka (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 334--345.

[41]

Hongsong Li, Fabio Pellacini, and Kenneth E. Torrance. 2005. A Hybrid Monte Carlo Method for Accurate and Efficient Subsurface Scattering. In Proceedings of the Sixteenth Eurographics Conference on Rendering Techniques (EGSR '05). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 283--290.

Digital Library

[42]

Satya P Mallick, Todd E Zickler, David J Kriegman, and Peter N Belhumeur. 2005. Beyond Lambert: Reconstructing specular surfaces using color. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 2. Ieee, 619--626.

Digital Library

[43]

Francesco Guido Mangano, Andrea Gandolfi, Giuseppe Luongo, and Silvia Logozzo. 2017. Intraoral scanners in dentistry: a review of the current literature. In BMC oral health.

[44]

Albert Mehl and Volker Blanz. 2005. A new approach for automatic reconstruction of occlusal surfaces with the biogeneric tooth model. Int. J. Comput. Dent. 8 (2005), 13--25.

[45]

Zhuo Meng, Steve Yao, Hui Yao, Yan Liang, Tiegen Liu, Yanni Li, Guanhua Wang, and Shoufeng Lan. 2009. Measurement of the refractive index of human teeth by optical coherence tomography. Journal of Biomedical Optics 14 (2009), 14 - 14 - 4.

[46]

E. Mostafa, S. Elhabian, A. Abdelrahim, S. Elshazly, and A. Farag. 2014. Statistical morphable model for human teeth restoration. In IEEE ICIP. 4285--4288.

[47]

Adolfo Munoz, Jose I. Echevarria, Francisco J. Seron, Jorge Lopez-Moreno, Mashhuda Glencross, and Diego Gutierrez. 2011. BSSRDF Estimation from Single Images. Computer Graphics Forum 30, 2 (2011), 455--464. 2011.01873.x

[48]

S. Omachi, K. Saito, H. Aso, S. Kasahara, S. Yamada, and K. Kimura. 2007. Tooth shape reconstruction from ct images using spline Curves. In Wavelet Analysis and Pattern Recognition, Vol. 1. 393--396.

[49]

Marios Papas, Christian Regg, Wojciech Jarosz, Bernd Bickel, Philip Jackson, Wojciech Matusik, Steve Marschner, and Markus Gross. 2013. Fabricating Translucent Materials Using Continuous Pigment Mixtures. ACM Trans. Graph. 32, 4, Article 146 (July 2013), 12 pages.

Digital Library

[50]

Mark Pauly, Thomas Kollig, and Alexander Keller. 2000. Metropolis light transport for participating media. Springer-Verlag, London, UK, 11--22.

Digital Library

[51]

Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically Based Rendering: From Theory to Implementation (3rd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Digital Library

[52]

Ioana-Sofia Pop-Ciutrila, Razvan Ghinea, Maria del Mar Perez Gomez, Horatiu Alexandru Colosi, Diana Dudea, and Mandra Badea. 2015. Dentine scattering, absorption, transmittance and light reflectivity in human incisors, canines and molars. Journal of Dentistry 43 (2015), 1116--1124.

[53]

D. Spitzer and J. J. Ten Bosch. 1975. The absorption and scattering of light in bovine and human dental enamel. Calcified Tissue Research 17, 2 (1975), 129--137.

[54]

Jos Stam. 1995. Multiple scattering as a diffusion process. (1995), 41--50.

[55]

Sarah Tariq, Andrew Gardner, Ignacio Llamas, Andrew Jones, Paul Debevec, and Greg Turk. 2006. Efficient estimation of spatially varying subsurface scattering parameters. Vision, Modeling, and Visualization (VMV2006) (2006), 129--136.

[56]

Xin Tong, Jiaping Wang, Stephen Lin, Baining Guo, and Heung-Yeung Shum. 2005. Modeling and Rendering of Quasi-homogeneous Materials. ACM Trans. Graph. 24, 3 (July 2005), 1054--1061.

Digital Library

[57]

Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction Through Rough Surfaces. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (EGSR'07). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 195--206.

Digital Library

[58]

Bruce Walter, Shuang Zhao, Nicolas Holzschuch, and Kavita Bala. 2009. Single Scattering in Refractive Media with Triangle Mesh Boundaries. ACM Transactions on Graphics 28, 3 (Aug. 2009), 92:1--8.

Digital Library

[59]

Jiaping Wang, Shuang Zhao, Xin Tong, Stephen Lin, Zhouchen Lin, Yue Dong, Baining Guo, and Heung-Yeung Shum. 2008. Modeling and Rendering of Heterogeneous Translucent Materials Using the Diffusion Equation. ACM Trans. Graph. 27, 1, Article 9 (March 2008), 18 pages.

Digital Library

[60]

Tim Weyrich, Wojciech Matusik, Hanspeter Pfister, Bernd Bickel, Craig Donner, Chien Tu, Janet McAndless, Jinho Lee, Addy Ngan, Henrik Wann Jensen, et al. 2006. Analysis of human faces using a measurement-based skin reflectance model. In ACM Transactions on Graphics (TOG), Vol. 25. ACM, 1013--1024.

Digital Library

[61]

Chenglei Wu, Derek Bradley, Pablo Garrido, Michael Zollhöfer, Christian Theobalt, Markus Gross, and Thabo Beeler. 2016. Model-based Teeth Reconstruction. ACM Trans. Graph. 35, 6, Article 220 (2016), 220:1--220:13 pages.

Digital Library

[62]

R Yanagisawa, Y Sugaya, S Kasahara, and S Omachi. 2014. Tooth shape reconstruction from dental CT images with the region-growing method. Dentomaxillofacial Radiology 43, 6 (2014), 20140080.

[63]

M. D. Zeiler. 2012. ADADELTA: An Adaptive Learning Rate Method. ArXiv e-prints (Dec. 2012). arXiv:cs.LG/1212.5701

[64]

Shu-Xian Zheng, Jia Li, and Qing-Feng Sun. 2011. A Novel 3D Morphing Approach for Tooth Occlusal Surface Reconstruction. Comput. Aided Des. 43, 3 (2011), 293--302.

Digital Library

Cited By

Tran DJensen MSantafé-Gabarda PKällberg SFerrero AHannemose MFrisvad J(2024)Digitizing translucent object appearance by validating computed optical propertiesApplied Optics10.1364/AO.52197463:16(4317)Online publication date: 22-May-2024
https://doi.org/10.1364/AO.521974
Kerbiriou GAvril QMarchal M(2024)3D Reconstruction and Semantic Modeling of EyelashesComputer Graphics Forum10.1111/cgf.1504043:2Online publication date: 24-Apr-2024
https://doi.org/10.1111/cgf.15040
Li CJin LZheng YYu YHan X(2023)EMS: 3D Eyebrow Modeling from Single-View ImagesACM Transactions on Graphics10.1145/361832342:6(1-19)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618323
Show More Cited By

Index Terms

Appearance capture and modeling of human teeth
1. Computing methodologies
  1. Computer graphics
    1. Rendering
      1. Ray tracing
      2. Reflectance modeling
    2. Shape modeling

Recommendations

Differentiable Monte Carlo ray tracing through edge sampling

Gradient-based methods are becoming increasingly important for computer graphics, machine learning, and computer vision. The ability to compute gradients is crucial to optimization, inverse problems, and deep learning. In rendering, the gradient is ...
Single-shot high-quality facial geometry and skin appearance capture

We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not require ...
Practical dynamic facial appearance modeling and acquisition

We present a method to acquire dynamic properties of facial skin appearance, including dynamic diffuse albedo encoding blood flow, dynamic specular intensity, and per-frame high resolution normal maps for a facial performance sequence. The method ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 37, Issue 6

December 2018

1401 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3272127

Editor:
Takeo Igarashi
The University of Tokyo, Japan

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 December 2018

Published in TOG Volume 37, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
639
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)3

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tran DJensen MSantafé-Gabarda PKällberg SFerrero AHannemose MFrisvad J(2024)Digitizing translucent object appearance by validating computed optical propertiesApplied Optics10.1364/AO.52197463:16(4317)Online publication date: 22-May-2024
https://doi.org/10.1364/AO.521974
Kerbiriou GAvril QMarchal M(2024)3D Reconstruction and Semantic Modeling of EyelashesComputer Graphics Forum10.1111/cgf.1504043:2Online publication date: 24-Apr-2024
https://doi.org/10.1111/cgf.15040
Li CJin LZheng YYu YHan X(2023)EMS: 3D Eyebrow Modeling from Single-View ImagesACM Transactions on Graphics10.1145/361832342:6(1-19)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618323
Alterman MSaiko ELevin A(2022)Direct acquisition of volumetric scattering phase function using speckle correlationsSIGGRAPH Asia 2022 Conference Papers10.1145/3550469.3555379(1-9)Online publication date: 29-Nov-2022
https://dl.acm.org/doi/10.1145/3550469.3555379
Zhang CElgharib MFox GGu MTheobalt CWang W(2022)An Implicit Parametric Morphable Dental ModelACM Transactions on Graphics10.1145/3550454.355546941:6(1-13)Online publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1145/3550454.3555469
Deng XLuan FWalter BBala KMarschner S(2022)Reconstructing Translucent Objects using Differentiable RenderingACM SIGGRAPH 2022 Conference Proceedings10.1145/3528233.3530714(1-10)Online publication date: 27-Jul-2022
https://dl.acm.org/doi/10.1145/3528233.3530714
Ploumpis SMoschoglou STriantafyllou VZafeiriou S(2022)3D human tongue reconstruction from single “in-the-wild” images2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00279(2761-2770)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00279
Wang YSong LWu WQian CHe RLoy C(2022)Talking Faces: Audio-to-Video Face GenerationHandbook of Digital Face Manipulation and Detection10.1007/978-3-030-87664-7_8(163-188)Online publication date: 31-Jan-2022
https://doi.org/10.1007/978-3-030-87664-7_8
Chandran PWinberg SZoss GRiviere JGross MGotardo PBradley D(2021)Rendering with styleACM Transactions on Graphics10.1145/3478513.348050940:6(1-14)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480509
Bao LLin XChen YZhang HWang SZhe XKang DHuang HJiang XWang JYu DZhang Z(2021)High-Fidelity 3D Digital Human Head Creation from RGB-D SelfiesACM Transactions on Graphics10.1145/347295441:1(1-21)Online publication date: 9-Nov-2021
https://dl.acm.org/doi/10.1145/3472954
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents