Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3489849.3489922acmconferencesArticle/Chapter ViewAbstractPublication PagesvrstConference Proceedingsconference-collections
abstract

IMAGEimate - An End-to-End Pipeline to Create Realistic Animatable 3D Avatars from a Single Image Using Neural Networks

Published: 08 December 2021 Publication History
  • Get Citation Alerts
  • Abstract

    Current advances in image based 3D human shape estimation and parametric human models enable creating realistic 3D virtual humans. We present a pipeline which takes advantage of these models and takes a single input image to create realistic 3D animatable avatars. The pipeline extracts shape and pose parameters from the input image and builds an implicit surface representation, which is then fitted onto a parametric human model. This fitted human model is animated to new and novel poses extracting pose parameters from a motion capture dataset. We extend the pipeline showcasing realism and interaction by texture painting it using Substance Painter and embedding it in an AR scene using Adobe Aero respectively.

    References

    [1]
    Bharat Lal Bhatnagar, Cristian Sminchisescu, Christian Theobalt, and Gerard Pons-Moll. 2020. Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction. In European Conference on Computer Vision (ECCV). Springer.
    [2]
    Levon Khachatryan. 2020. Tex-An Mesh: Textured and animatable human body mesh reconstruction from a single image. https://github.com/lev1khachatryan/Tex-An_Mesh.
    [3]
    Ruilong Li, Shan Yang, David A. Ross, and Angjoo Kanazawa. 2021. Learn to Dance with AIST++: Music Conditioned 3D Dance Generation. arxiv:2101.08779 [cs.CV]
    [4]
    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-Person Linear Model. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 34, 6 (Oct. 2015), 248:1–248:16.
    [5]
    Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. 2019. AMASS: Archive of Motion Capture as Surface Shapes. In International Conference on Computer Vision. 5442–5451.
    [6]
    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. 2019. Expressive Body Capture: 3D Hands, Face, and Body from a Single Image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). 10975–10985.
    [7]
    Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. 2020. PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
    [8]
    Zhan Xu, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh. 2020. RigNet: Neural Rigging for Articulated Characters. ACM Trans. on Graphics 39 (2020).

    Cited By

    View all
    • (2024)Using Artmaking Generative AIs to Support Augmented Reality Learning Designs With Adobe Aero AppInquiries of Pedagogical Shifts and Critical Mindsets Among Educators10.4018/979-8-3693-1078-6.ch006(132-152)Online publication date: 8-Mar-2024

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    VRST '21: Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology
    December 2021
    563 pages
    ISBN:9781450390927
    DOI:10.1145/3489849
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 December 2021

    Check for updates

    Qualifiers

    • Abstract
    • Research
    • Refereed limited

    Conference

    VRST '21

    Acceptance Rates

    Overall Acceptance Rate 66 of 254 submissions, 26%

    Upcoming Conference

    VRST '24

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)0

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Using Artmaking Generative AIs to Support Augmented Reality Learning Designs With Adobe Aero AppInquiries of Pedagogical Shifts and Critical Mindsets Among Educators10.4018/979-8-3693-1078-6.ch006(132-152)Online publication date: 8-Mar-2024

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media