invited-talk

Masquerade: fine-scale details for head-mounted camera motion capture data

Authors:

Lucio Moser,

Darren Hendler,

Doug RobleAuthors Info & Claims

SIGGRAPH '17: ACM SIGGRAPH 2017 Talks

Article No.: 18, Pages 1 - 2

https://doi.org/10.1145/3084363.3085086

Published: 30 July 2017 Publication History

Get Access

Abstract

We present Masquerade, a novel modular and expandable tool for adding fine-scale details to facial motion capture data from head-mounted cameras. After studying two important related works we developed a framework to reproduce the original approaches as well as to test equally promising alternatives. This framework has been vital for understanding the limitations of previous approaches and to explore ways to improve the results. Our final solution was a combination of algorithms and data representations that produced better results than previous works when tested with our evaluation data. Since then, Masquerade is being actively used in production for enhancing marker data with fine-scale details.

References

[1]

Amit H. Bermano, Derek Bradley, Thabo Beeler, Fabio Zund, Derek Nowrouzezahrai, Ilya Baran, Olga Sorkine-Hornung, Hanspeter Pfister, Robert W. Sumner, Bernd Bickel, and Markus Gross. 2014. Facial Performance Enhancement Using Dynamic Shape Space Analysis. ACM Trans. Graph. 33, 2, Article 13 (April 2014), 12 pages.

Digital Library

Google Scholar

[2]

Bernd Bickel, Manuel Lang, Mario Botsch, Miguel A. Otaduy, and Markus Gross. 2008. Pose-space Animation and Transfer of Facial Details. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '08). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 57--66. http://dl.acm.org/citation.cfm?id=1632592.1632602

Digital Library

Google Scholar

[3]

Steven McDonagh, Martin Klaudiny, Derek Bradley, Thabo Beeler, Iain Matthews, Kenny Mitchell, undefined, undefined, undefined, and undefined. 2016. Synthetic Prior Design for Real-Time Face Tracking. 2016 Fourth International Conference on 3D Vision (3DV) 00 (2016), 639--648.

Crossref

Google Scholar

[4]

Chenglei Wu, Derek Bradley, Markus Gross, and Thabo Beeler. 2016. An Anatomically-constrained Local Deformation Model for Monocular Face Capture. ACM Trans. Graph. 35, 4, Article 115 (July 2016), 12 pages.

Digital Library

Google Scholar

Cited By

View all

Cong MLan LFedkiw R(2024)Local Geometric Indexing of High Resolution Data for Facial Reconstruction From Sparse MarkersIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328949530:8(5289-5298)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3289495
Vilchis CMendez-Ruiz MPerez-Guerrero CGonzalez-Mendoza M(2024)Action unit intensity regression for facial MoCap aimed towards digital humansMultimedia Tools and Applications10.1007/s11042-024-19400-8Online publication date: 25-May-2024
https://doi.org/10.1007/s11042-024-19400-8
Tian ZWeng DFang HShen TZhang W(2024)Robust facial marker tracking based on a synthetic analysis of optical flows and the YOLO networkThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02931-w40:4(2471-2489)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s00371-023-02931-w
Show More Cited By

Index Terms

Masquerade: fine-scale details for head-mounted camera motion capture data
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        Motion capture

Recommendations

High-quality, cost-effective facial motion capture pipeline with 3D regression
SIGGRAPH '18: ACM SIGGRAPH 2018 Talks

We present our improved marker-based facial motion capture pipeline that leverages on 3D regression from head-mounted camera (HMC) images to speed up and reduce the cost of high quality 3D marker tracking. We use machine learning to boost productivity ...
EgoCap: egocentric marker-less motion capture with two fisheye cameras

Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center. They often create discomfort with marker suits, and their recording volume is ...
Motion capture from body-mounted cameras

Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large ...

Comments

Information & Contributors

Information

Published In

SIGGRAPH '17: ACM SIGGRAPH 2017 Talks

July 2017

158 pages

ISBN:9781450350082

DOI:10.1145/3084363

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 July 2017

Check for updates

Author Tags

Qualifiers

Invited-talk

Conference

SIGGRAPH '17

Sponsor:

SIGGRAPH

SIGGRAPH '17: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 30 - August 3, 2017

California, Los Angeles

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
601
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cong MLan LFedkiw R(2024)Local Geometric Indexing of High Resolution Data for Facial Reconstruction From Sparse MarkersIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328949530:8(5289-5298)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3289495
Vilchis CMendez-Ruiz MPerez-Guerrero CGonzalez-Mendoza M(2024)Action unit intensity regression for facial MoCap aimed towards digital humansMultimedia Tools and Applications10.1007/s11042-024-19400-8Online publication date: 25-May-2024
https://doi.org/10.1007/s11042-024-19400-8
Tian ZWeng DFang HShen TZhang W(2024)Robust facial marker tracking based on a synthetic analysis of optical flows and the YOLO networkThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02931-w40:4(2471-2489)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s00371-023-02931-w
Bailey SRiviere JMikkelsen MO'Brien J(2023)Monocular Facial Performance Capture Via Deep Expression MatchingComputer Graphics Forum10.1111/cgf.1463941:8(243-254)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14639
Tian ZWeng DFang HBao Y(2023)Deep Detector and Optical Flow-based Tracking Approach of Facial Markers for Animation Capture2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct60411.2023.00134(625-630)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00134
Vilchis CPerez-Guerrero CMendez-Ruiz MGonzalez-Mendoza M(2023)A survey on the pipeline evolution of facial capture and tracking for digital humansMultimedia Systems10.1007/s00530-023-01081-229:4(1917-1940)Online publication date: 1-Apr-2023
https://doi.org/10.1007/s00530-023-01081-2
Moser LChien CWilliams MSerra JHendler DRoble D(2021)Semi-supervised video-driven facial animation transfer for productionACM Transactions on Graphics10.1145/3478513.348051540:6(1-18)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480515
Pontes HDuarte JPinheiro P(2018)A software library for sign recognition through natural interface devices2018 13th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI.2018.8399280(1-6)Online publication date: Jun-2018
https://doi.org/10.23919/CISTI.2018.8399280
Hendler DMoser LBattulwar RCorral DCramer PMiller RCloudsdale RRoble D(2018)AvengersACM SIGGRAPH 2018 Talks10.1145/3214745.3214766(1-2)Online publication date: 12-Aug-2018
https://dl.acm.org/doi/10.1145/3214745.3214766
Moser LWilliams MHendler DRoble D(2018)High-quality, cost-effective facial motion capture pipeline with 3D regressionACM SIGGRAPH 2018 Talks10.1145/3214745.3214755(1-2)Online publication date: 12-Aug-2018
https://dl.acm.org/doi/10.1145/3214745.3214755

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

High-quality, cost-effective facial motion capture pipeline with 3D regression

EgoCap: egocentric marker-less motion capture with two fisheye cameras

Motion capture from body-mounted cameras

Comments

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

High-quality, cost-effective facial motion capture pipeline with 3D regression

EgoCap: egocentric marker-less motion capture with two fisheye cameras

Motion capture from body-mounted cameras

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations