abstract

FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms

Authors:

Maki SugimotoAuthors Info & Claims

SA '19: SIGGRAPH Asia 2019 XR

Pages 9 - 10

https://doi.org/10.1145/3355355.3361888

Published: 17 November 2019 Publication History

Get Access

Abstract

Supernumerary Robotic Arms (SRAs) can make physical activities easier, but require cooperation with the operator. To improve cooperation, we predict the operator’s intentions by using his/her Facial Expressions (FEs). We propose to map FEs to SRAs commands (e.g. grab, release). To measure FEs, we used a optical sensor-based approach (here inside a HMD), the sensors data are fed to a SVM classifying them in FEs. The SRAs can then carry out commands by predicting the operator’s FEs (and arguably, the operator’s intention). We made a Virtual reality Environment (VE) with SRAs and synchronizable avatar to investigate the most suitable mapping between FEs and SRAs. In SIGGRAPH Asia 2019, the user can manipulate virtual SRAs using his/her FEs.

Supplementary Material

p9-fukuoka-supplement (sa19xr-14.mov)

video

Download
134.50 MB

References

[1]

M.Niwa, H.Iizuka, H.Ando, and T.Maeda. 2012. Tumori Control : Manipulate Robot with detection and transmission of an Archetype of Behavioral Intention. Transactions of the Virtual Reality Society of Japan 17, 1 (2012), 3–10. https://doi.org/10.18974/tvrsj.17.1_3

Crossref

Google Scholar

[2]

K. Suzuki, F. Nakamura, J. Otsuka, K. Masai, Y. Itoh, Y. Sugiura, and M. Sugimoto. 2017. Recognition and mapping of facial expressions to avatar by embedded photo reflective sensors in head mounted display. In 2017 IEEE Virtual Reality (VR). 177–185. https://doi.org/10.1109/VR.2017.7892245

Crossref

Google Scholar

Cited By

View all

Fukuoka MNakamura FVerhulst AInami MKitazaki MSugimoto M(2024)Sensory Attenuation With a Virtual Robotic Arm Controlled Using Facial MovementsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324609230:7(4023-4038)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2023.3246092
Nakamura FSugimoto MWard JMcGill MMarky K(2023)Exploring the Effect of Transfer Learning on Facial Expression Recognition using Photo-Reflective Sensors embedded into a Head-Mounted DisplayProceedings of the Augmented Humans International Conference 202310.1145/3582700.3583705(317-319)Online publication date: 12-Mar-2023
https://dl.acm.org/doi/10.1145/3582700.3583705
Yamamura NUriu DMuramatsu MKamiyama YKashino ZSakamoto STanaka NTanigawa TOnishi AYoshida SYamanaka SInami M(2023)Social Digital Cyborgs: The Collaborative Design Process of JIZAI ARMSProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581169(1-19)Online publication date: 19-Apr-2023
https://doi.org/10.1145/3544548.3581169
Show More Cited By

Recommendations

Enhanced Pressure-Based Multimodal Immersive Experiences
AH '18: Proceedings of the 9th Augmented Human International Conference

Haptic feedback to the feet has been explored in the context of ground surface simulation, but existing solutions have generally not taken into account the effects of variable foot pressure during naturalistic stepping movements. We present here our ...
Augmented Human: Augmented Reality and Beyond
AltMM'18: Proceedings of the 3rd International Workshop on Multimedia Alternate Realities

Will Augmented Reality (AR) allow us to access digital information, experience others' stories, and thus explore alternate realities? AR has recently attracted attention again due to the rapid advances in related multimedia technologies as well as ...
Real-time facial tracking in virtual reality
SA '16: SIGGRAPH ASIA 2016 VR Showcase

Virtual reality (VR) emerges as the next social computing platform. For realizing immersive social interactions, projecting facial expressions onto the virtual avatar a crucial component. This is a challenge in VR as it requires capturing the facial ...

Comments

Information & Contributors

Information

Published In

SA '19: SIGGRAPH Asia 2019 XR

November 2019

47 pages

ISBN:9781450369473

DOI:10.1145/3355355

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 November 2019

Check for updates

Author Tags

Qualifiers

Abstract
Research
Refereed limited

Conference

SA '19

Sponsor:

SIGGRAPH

SA '19: SIGGRAPH Asia 2019

November 17 - 20, 2019

QLD, Brisbane, Australia

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
299
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Fukuoka MNakamura FVerhulst AInami MKitazaki MSugimoto M(2024)Sensory Attenuation With a Virtual Robotic Arm Controlled Using Facial MovementsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324609230:7(4023-4038)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2023.3246092
Nakamura FSugimoto MWard JMcGill MMarky K(2023)Exploring the Effect of Transfer Learning on Facial Expression Recognition using Photo-Reflective Sensors embedded into a Head-Mounted DisplayProceedings of the Augmented Humans International Conference 202310.1145/3582700.3583705(317-319)Online publication date: 12-Mar-2023
https://dl.acm.org/doi/10.1145/3582700.3583705
Yamamura NUriu DMuramatsu MKamiyama YKashino ZSakamoto STanaka NTanigawa TOnishi AYoshida SYamanaka SInami M(2023)Social Digital Cyborgs: The Collaborative Design Process of JIZAI ARMSProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581169(1-19)Online publication date: 19-Apr-2023
https://doi.org/10.1145/3544548.3581169
Nakamura FMurakami MSuzuki KFukuoka MMasai KSugimoto M(2023)Analyzing the Effect of Diverse Gaze and Head Direction on Facial Expression Recognition With Photo-Reflective Sensors Embedded in a Head-Mounted DisplayIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317976629:10(4124-4139)Online publication date: 1-Oct-2023
https://doi.org/10.1109/TVCG.2022.3179766
Nakabayashi YNakamura FSugimoto M(2023)Facial Expression Recognition by Photo-Reflective Sensors Considering Time Series and Head Posture2023 28th Asia Pacific Conference on Communications (APCC)10.1109/APCC60132.2023.10460652(358-363)Online publication date: 19-Nov-2023
https://doi.org/10.1109/APCC60132.2023.10460652
Nakanishi YFukuoka MKasahara SSugimoto M(2022)Synchronous and Asynchronous Manipulation Switching of Multiple Robotic Embodiment Using EMG and Eye GazeProceedings of the Augmented Humans International Conference 202210.1145/3519391.3522753(94-103)Online publication date: 13-Mar-2022
https://dl.acm.org/doi/10.1145/3519391.3522753
Inami MUriu DKashino ZYoshida SSaito HMaekawa AKitazaki M(2022)Cyborgs, Human Augmentation, Cybernetics, and JIZAI BodyProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519401(230-242)Online publication date: 13-Mar-2022
https://dl.acm.org/doi/10.1145/3519391.3519401
Teo TSakurada KFukuoka MSugimoto M(2022)Techniques for using VRChat to Replace On-site Experiments2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00055(249-253)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00055
Liao ZChen BChang TZheng QLiu KLv J(2022)A human augmentation device design review: supernumerary robotic limbsIndustrial Robot: the international journal of robotics research and application10.1108/IR-03-2022-007950:2(256-274)Online publication date: 27-Oct-2022
https://doi.org/10.1108/IR-03-2022-0079
Eden JBräcklein MIbáñez JBarsakcioglu DDi Pino GFarina DBurdet EMehring C(2022)Principles of human movement augmentation and the challenges in making it a realityNature Communications10.1038/s41467-022-28725-713:1Online publication date: 15-Mar-2022
https://doi.org/10.1038/s41467-022-28725-7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

Supplementary Material

References

Cited By

Recommendations

Enhanced Pressure-Based Multimodal Immersive Experiences

Augmented Human: Augmented Reality and Beyond

Real-time facial tracking in virtual reality

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations