Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Open access

CT2Hair: High-Fidelity 3D Hair Modeling using Computed Tomography

Published: 26 July 2023 Publication History
  • Get Citation Alerts
  • Abstract

    We introduce CT2Hair, a fully automatic framework for creating high-fidelity 3D hair models that are suitable for use in downstream graphics applications. Our approach utilizes real-world hair wigs as input, and is able to reconstruct hair strands for a wide range of hair styles. Our method leverages computed tomography (CT) to create density volumes of the hair regions, allowing us to see through the hair unlike image-based approaches which are limited to reconstructing the visible surface. To address the noise and limited resolution of the input density volumes, we employ a coarse-to-fine approach. This process first recovers guide strands with estimated 3D orientation fields, and then populates dense strands through a novel neural interpolation of the guide strands. The generated strands are then refined to conform to the input density volumes. We demonstrate the robustness of our approach by presenting results on a wide variety of hair styles and conducting thorough evaluations on both real-world and synthetic datasets. Code and data for this paper are at github.com/facebookresearch/CT2Hair.

    Supplementary Material

    ZIP File (papers_209-supplemental.zip)
    supplemental material
    MP4 File (papers_209_VOD.mp4)
    presentation

    References

    [1]
    Khaled AbuJbara, Ramzi Idoughi, and Wolfgang Heidrich. 2021. Non-Linear Anisotropic Diffusion for Memory-Efficient Computed Tomography Super-Resolution Reconstruction. In 2021 International Conference on 3D Vision (3DV). IEEE.
    [2]
    Yongtang Bao and Yue Qi. 2018. A survey of image-based techniques for hair modeling. IEEE Access 6 (2018), 18670--18684.
    [3]
    Thabo Beeler, Bernd Bickel, Gioacchino Noris, Paul Beardsley, Steve Marschner, Robert W Sumner, and Markus Gross. 2012. Coupled 3D reconstruction of sparse facial hair and skin. ACM Transactions on Graphics 31, 4 (2012), 1--10.
    [4]
    Florence Bertails, Basile Audoly, Marie-Paule Cani, Bernard Querleux, Frédéric Leroy, and Jean-Luc Lévëque. 2006. Super-Helices for Predicting the Dynamics of Natural Hair. ACM Transactions on Graphics 25, 3 (2006), 1180--1187.
    [5]
    Paul J Besl and Neil D McKay. 1992. Method for registration of 3-D shapes. In Sensor Fusion IV: Control Paradigms and Data Structures, Vol. 1611. Spie, 586--606.
    [6]
    Menglei Chai, Linjie Luo, Kalyan Sunkavalli, Nathan Carr, Sunil Hadap, and Kun Zhou. 2015. High-quality hair modeling from a single portrait photo. ACM Transactions on Graphics 34, 6 (2015), 1--10.
    [7]
    Menglei Chai, Tianjia Shao, Hongzhi Wu, Yanlin Weng, and Kun Zhou. 2016. Autohair: Fully automatic hair modeling from a single image. ACM Transactions on Graphics 35, 4 (2016).
    [8]
    Menglei Chai, Lvdi Wang, Yanlin Weng, Xiaogang Jin, and Kun Zhou. 2013. Dynamic hair manipulation in images and videos. ACM Transactions on Graphics 32, 4 (2013), 1--8.
    [9]
    Menglei Chai, Lvdi Wang, Yanlin Weng, Yizhou Yu, Baining Guo, and Kun Zhou. 2012. Single-view hair modeling for portrait manipulation. ACM Transactions on Graphics 31, 4 (2012), 1--8.
    [10]
    Matt Jen-Yuan Chiang, Benedikt Bitterli, Chuck Tappan, and Brent Burley. 2016. A Practical and Controllable Hair and Fur Model for Production Path Tracing. In Computer Graphics Forum, Vol. 2. 275--283.
    [11]
    Jose I Echevarria, Derek Bradley, Diego Gutierrez, and Thabo Beeler. 2014. Capturing and stylizing hair for 3D fabrication. ACM Transactions on Graphics 33, 4 (2014), 1--11.
    [12]
    Hongbo Fu, Yichen Wei, Chiew-Lan Tai, and Long Quan. 2007. Sketching hairstyles. In Proceedings of the 4th Eurographics Workshop on Sketch-based Interfaces and Modeling. 31--36.
    [13]
    Tomas Lay Herrera, Arno Zinke, and Andreas Weber. 2012. Lighting hair from the inside: A thermal approach to hair reconstruction. ACM Transactions on Graphics 31, 6 (2012), 1--9.
    [14]
    Liwen Hu, Derek Bradley, Hao Li, and Thabo Beeler. 2017a. Simulation-ready hair capture. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 281--294.
    [15]
    Liwen Hu, Chongyang Ma, Linjie Luo, and Hao Li. 2014a. Robust hair capture using simulated examples. ACM Transactions on Graphics 33, 4 (2014), 1--10.
    [16]
    Liwen Hu, Chongyang Ma, Linjie Luo, and Hao Li. 2015. Single-view hair modeling using a hairstyle database. ACM Transactions on Graphics 34, 4 (2015), 1--9.
    [17]
    Liwen Hu, Chongyang Ma, Linjie Luo, Li-Yi Wei, and Hao Li. 2014b. Capturing braided hairstyles. ACM Transactions on Graphics 33, 6 (2014), 1--9.
    [18]
    Liwen Hu, Shunsuke Saito, Lingyu Wei, Koki Nagano, Jaewoo Seo, Jens Fursund, Iman Sadeghi, Carrie Sun, Yen-Chun Chen, and Hao Li. 2017b. Avatar digitization from a single image for real-time rendering. ACM Transactions on Graphics 36, 6 (2017), 1--14.
    [19]
    Wenzel Jakob, Jonathan T Moon, and Steve Marschner. 2009. Capturing hair assemblies fiber by fiber. ACM Transactions on Graphics 28, 5 (2009), 1--9.
    [20]
    Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction. ACM Transactions on Graphics 32, 3 (2013), 1--13.
    [21]
    Marilyn Keller, Silvia Zuffi, Michael J Black, and Sergi Pujades. 2022. OSSO: Obtaining Skeletal Shape from Outside. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20492--20501.
    [22]
    Pramook Khungurn, Daniel Schroeder, Shuang Zhao, S Marschner, and K Bala. 2015. Matching micro-appearance models to real fabrics. ACM Transactions on Graphics 3, 10.1145, Article 1 (2015).
    [23]
    Pramook Khungurn, Rundong Wu, James Noeckel, Steve Marschner, and Kavita Bala. 2017. Fast rendering of fabric micro-appearance models under directional and spherical gaussian lights. ACM Transactions on Graphics 36, 6 (2017), 1--15.
    [24]
    Tae-Yong Kim and Ulrich Neumann. 2002. Interactive multiresolution hair modeling and editing. ACM Transactions on Graphics 21, 3 (2002), 620--629.
    [25]
    Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
    [26]
    Waiming Kong and Masayuki Nakajima. 1998. Generation of 3D Hair Model from Multiple Pictures. The Journal of the Institute of Image Information and Television Engineers 52, 9 (1998), 1351--1356.
    [27]
    Zhiyi Kuang, Yiyang Chen, Hongbo Fu, Kun Zhou, and Youyi Zheng. 2022. Deep-MVSHair: Deep Hair Modeling from Sparse Views. In SIGGRAPH Asia 2022 Conference Papers. 1--8.
    [28]
    Alexander LC Kwan, John M Boone, Kai Yang, and Shih-Ying Huang. 2007. Evaluation of the spatial resolution characteristics of a cone-beam breast CT scanner. Medical Physics 34, 1 (2007), 275--281.
    [29]
    Yuwei Li, Minye Wu, Yuyao Zhang, Lan Xu, and Jingyi Yu. 2021. PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. 816--822.
    [30]
    Yuwei Li, Longwen Zhang, Zesong Qiu, Yingwenqi Jiang, Nianyi Li, Yuexin Ma, Yuyao Zhang, Lan Xu, and Jingyi Yu. 2022. NIMBLE: a non-rigid hand model with bones and muscles. ACM Transactions on Graphics 41, 4 (2022), 1--16.
    [31]
    Shu Liang, Xiufeng Huang, Xianyu Meng, Kunyao Chen, Linda G Shapiro, and Ira Kemelmacher-Shlizerman. 2018. Video to fully automatic 3d hair model. ACM Transactions on Graphics 37, 6 (2018), 1--14.
    [32]
    Linjie Luo, Hao Li, and Szymon Rusinkiewicz. 2013a. Structure-aware hair capture. ACM Transactions on Graphics 32, 4 (2013), 1--12.
    [33]
    Linjie Luo, Cha Zhang, Zhengyou Zhang, and Szymon Rusinkiewicz. 2013b. Wide-baseline hair capture using strand-based refinement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 265--272.
    [34]
    Miles Macklin, Matthias Müller, and Nuttapong Chentanez. 2016. XPBD: position-based simulation of compliant constrained dynamics. In Proceedings of the 9th International Conference on Motion in Games. 49--54.
    [35]
    Ishit Mehta, Michaël Gharbi, Connelly Barnes, Eli Shechtman, Ravi Ramamoorthi, and Manmohan Chandraker. 2021. Modulated periodic activations for generalizable local functional representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14214--14223.
    [36]
    Giljoo Nam, Chenglei Wu, Min H Kim, and Yaser Sheikh. 2019. Strand-accurate multiview hair capture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 155--164.
    [37]
    Sylvain Paris, Hector M Briceno, and François X Sillion. 2004. Capture of hair geometry from multiple images. ACM Transactions on Graphics 23, 3 (2004), 712--719.
    [38]
    Sylvain Paris, Will Chang, Oleg I Kozhushnyan, Wojciech Jarosz, Wojciech Matusik, Matthias Zwicker, and Frédo Durand. 2008. Hair photobooth: geometric and photometric acquisition of real hairstyles. ACM Transactions on Graphics 27, 3 (2008), 30.
    [39]
    Emmanuel Piuze, Paul G Kry, and Kaleem Siddiqi. 2011. Generalized helicoids for modeling hair geometry. In Computer Graphics Forum, Vol. 30. Wiley Online Library, 247--256.
    [40]
    Radu Alexandru Rosu, Shunsuke Saito, Ziyan Wang, Chenglei Wu, Sven Behnke, and Giljoo Nam. 2022. Neural strands: Learning hair geometry and appearance from multi-view images. In European Conference on Computer Vision. Springer, 73--89.
    [41]
    Darius Rückert, Yuanhao Wang, Rui Li, Ramzi Idoughi, and Wolfgang Heidrich. 2022. NeAT: Neural Adaptive Tomography. ACM Transactions on Graphics 41, 4 (2022).
    [42]
    Shunsuke Saito, Liwen Hu, Chongyang Ma, Hikaru Ibayashi, Linjie Luo, and Hao Li. 2018. 3D hair synthesis using volumetric variational autoencoders. ACM Transactions on Graphics 37, 6 (2018), 1--12.
    [43]
    Yuefan Shen, Changgeng Zhang, Hongbo Fu, Kun Zhou, and Youyi Zheng. 2020. Deepsketchhair: Deep sketch-based 3d hair modeling. IEEE Transactions on Visualization and Computer Graphics 27, 7 (2020), 3250--3263.
    [44]
    Tiancheng Sun, Giljoo Nam, Carlos Aliaga, Christophe Hery, and Ravi Ramamoorthi. 2021. Human Hair Inverse Rendering using Multi-View Photometric data. In Eurographics Symposium on Rendering - DL-only Track, Adrien Bousseau and Morgan McGuire (Eds.). The Eurographics Association.
    [45]
    Lvdi Wang, Yizhou Yu, Kun Zhou, and Baining Guo. 2009. Example-based hair geometry synthesis. ACM Transactions on Graphics 28, 3, Article 56 (2009).
    [46]
    Ziyan Wang, Giljoo Nam, Tuur Stuyck, Stephen Lombardi, Michael Zollhöfer, Jessica Hodgins, and Christoph Lassner. 2022. HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair Performance Capture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6143--6154.
    [47]
    Kelly Ward, Florence Bertails, Tae-Yong Kim, Stephen R Marschner, Marie-Paule Cani, and Ming C Lin. 2007a. A survey on hair modeling: Styling, simulation, and rendering. IEEE Transactions on Visualization and Computer Graphics 13, 2 (2007), 213--234.
    [48]
    Kelly Ward, Nico Galoppo, and Ming Lin. 2007b. Interactive virtual hair salon. Presence 16, 3 (2007), 237--251.
    [49]
    Kelly Ward, Ming C Lin, Lee Joohi, Susan Fisher, and Dean Macri. 2003. Modeling hair using level-of-detail representations. In Proceedings 11th IEEE International Workshop on Program Comprehension. IEEE, 41--47.
    [50]
    Sebastian Winberg, Gaspard Zoss, Prashanth Chandran, Paulo Gotardo, and Derek Bradley. 2022. Facial hair tracking for high fidelity performance capture. ACM Transactions on Graphics 41, 4 (2022), 1--12.
    [51]
    Keyu Wu, Yifan Ye, Lingchen Yang, Hongbo Fu, Kun Zhou, and Youyi Zheng. 2022. NeuralHDHair: Automatic High-fidelity Hair Modeling from a Single Image Using Implicit Neural Representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1526--1535.
    [52]
    Jun Xing, Koki Nagano, Weikai Chen, Haotian Xu, Li-yi Wei, Yajie Zhao, Jingwan Lu, Byungmoon Kim, and Hao Li. 2019. Hairbrush for immersive data-driven hair modeling. In Proceedings of the 32Nd Annual ACM Symposium on User Interface Software and Technology. 263--279.
    [53]
    Zexiang Xu, Hsiang-Tao Wu, Lvdi Wang, Changxi Zheng, Xin Tong, and Yue Qi. 2014. Dynamic hair capture using spacetime optimization. ACM Transactions on Graphics 33, 6 (2014), 1--11.
    [54]
    Lingchen Yang, Zefeng Shi, Youyi Zheng, and Kun Zhou. 2019. Dynamic hair modeling from monocular videos using deep neural networks. ACM Transactions on Graphics 38, 6 (2019), 1--12.
    [55]
    Yizhou Yu. 2001. Modeling realistic virtual hairstyles. In Proceedings Ninth Pacific Conference on Computer Graphics and Applications. Pacific Graphics 2001. IEEE, 295--304.
    [56]
    Cem Yuksel, Scott Schaefer, and John Keyser. 2009. Hair meshes. ACM Transactions on Graphics 28, 5 (2009), 1--7.
    [57]
    Guangming Zang, Mohamed Aly, Ramzi Idoughi, Peter Wonka, and Wolfgang Heidrich. 2018a. Super-resolution and sparse view CT reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV). 137--153.
    [58]
    Guangming Zang, Ramzi Idoughi, Ran Tao, Gilles Lubineau, Peter Wonka, and Wolfgang Heidrich. 2018b. Space-time tomography for continuously deforming objects. ACM Transactions on Graphics 37, 4 (2018), 1--14.
    [59]
    Guangming Zang, Ramzi Idoughi, Ran Tao, Gilles Lubineau, Peter Wonka, and Wolfgang Heidrich. 2019. Warp-and-Project Tomography for Rapidly Deforming Objects. ACM Transactions on Graphics 38, 4 (2019).
    [60]
    Meng Zhang, Pan Wu, Hongzhi Wu, Yanlin Weng, Youyi Zheng, and Kun Zhou. 2018. Modeling hair from an rgb-d camera. ACM Transactions on Graphics 37, 6 (2018), 1--10.
    [61]
    Meng Zhang and Youyi Zheng. 2019. Hair-GAN: Recovering 3D hair structure from a single image using generative adversarial networks. Visual Informatics 3, 2 (2019), 102--112.
    [62]
    Shuang Zhao, Wenzel Jakob, Steve Marschner, and Kavita Bala. 2011. Building volumetric appearance models of fabric using micro CT imaging. ACM Transactions on Graphics 30, 4 (2011), 1--10.
    [63]
    Shuang Zhao, Fujun Luan, and Kavita Bala. 2016. Fitting procedural yarn models for realistic cloth rendering. ACM Transactions on Graphics 35, 4 (2016), 1--11.
    [64]
    Mianlun Zheng, Bohan Wang, Jingtao Huang, and Jernej Barbič. 2022. Simulation of Hand Anatomy Using Medical Imaging. ACM Transactions on Graphics 41, 6 (2022), 1--20.
    [65]
    Yi Zhou, Liwen Hu, Jun Xing, Weikai Chen, Han-Wei Kung, Xin Tong, and Hao Li. 2018. Hairnet: Single-view hair reconstruction using convolutional neural networks. In Proceedings of the European Conference on Computer Vision (ECCV). 235--251.

    Cited By

    View all

    Index Terms

    1. CT2Hair: High-Fidelity 3D Hair Modeling using Computed Tomography

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Transactions on Graphics
        ACM Transactions on Graphics  Volume 42, Issue 4
        August 2023
        1912 pages
        ISSN:0730-0301
        EISSN:1557-7368
        DOI:10.1145/3609020
        Issue’s Table of Contents
        This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 26 July 2023
        Published in TOG Volume 42, Issue 4

        Check for updates

        Author Tags

        1. 3D modeling
        2. hair modeling
        3. computed tomography

        Qualifiers

        • Research-article

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)1,917
        • Downloads (Last 6 weeks)108
        Reflects downloads up to 26 Jul 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Modeling Hair Strands with Roving CapsulesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657450(1-9)Online publication date: 13-Jul-2024
        • (2024)Reach for the Arcs: Reconstructing Surfaces from SDFs via Tangent PointsACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657419(1-12)Online publication date: 13-Jul-2024
        • (2024)3D Reconstruction and Semantic Modeling of EyelashesComputer Graphics Forum10.1111/cgf.1504043:2Online publication date: 24-Apr-2024
        • (2023)Automatic Aorta Segmentation with Heavily Augmented, High-Resolution 3-D ResUNet: Contribution to the SEG.A ChallengeSegmentation of the Aorta. Towards the Automatic Segmentation, Modeling, and Meshing of the Aortic Vessel Tree from Multicenter Acquisition10.1007/978-3-031-53241-2_4(42-54)Online publication date: 8-Oct-2023

        View Options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Get Access

        Login options

        Full Access

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media