poster

Deep learning for action recognition in augmented reality assistance systems

Authors:

Matthias Schröder,

Helge RitterAuthors Info & Claims

SIGGRAPH '17: ACM SIGGRAPH 2017 Posters

Article No.: 75, Pages 1 - 2

https://doi.org/10.1145/3102163.3102191

Published: 30 July 2017 Publication History

Get Access

Abstract

Recent advances in the development of optical head-mounted displays (HMDs), such as the Microsoft HoloLens, Google Glass, or Epson Moverio, which overlay visual information directly in the user's field of vision, have opened up new possibilities for augmented reality (AR) applications. We propose a system that uses such an optical HMD to assist the user during goal-oriented activities (e.g. manufacturing work) in an intuitive and unobtrusive way (Essig et al. 2016). To this end, our system observes and recognizes the user's actions and generates context-sensitive feedback. Figure 1 shows an overview of our approach, exemplified with the task of assembling a bird house.

Supplementary Material

ZIP File (a75-schroder.zip)

Supplemental files.

Download
21.27 MB

References

[1]

Kai Essig, Benjamin Strenge, and Thomas Schack. 2016. ADAMAAS: Towards Smart Glasses for Mobile and Personalized Action Assistance. In International Conference on Pervasive Technologies Related to Assistive Environments. 46:1--46:4.

Digital Library

Google Scholar

[2]

Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Conference on Computer Vision and Pattern Recognition. 3431--3440.

Crossref

Google Scholar

[3]

Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556

Google Scholar

[4]

Andrea Tagliasacchi, Matthias Schröder, Anastasia Tkach, Sofien Bouaziz, Mario Botsch, and Mark Pauly. 2015. Robust Articulated-ICP for Real-Time Hand Tracking. Comput. Graph. Forum 34, 5 (2015), 101--114.

Crossref

Google Scholar

Cited By

View all

Fink LKostolani DTrautner TSchlund S(2024)Make some Noise: Acoustic Classification of Manual Work Steps Towards Adaptive Assistance SystemsProcedia CIRP10.1016/j.procir.2024.07.024127(135-140)Online publication date: 2024
https://doi.org/10.1016/j.procir.2024.07.024
Ali SAbdullah Armand TAthar AHussain AAli MYaseen MJoo MKim H(2023)Metaverse in Healthcare Integrated with Explainable AI and Blockchain: Enabling Immersiveness, Ensuring Trust, and Providing Patient Data SecuritySensors10.3390/s2302056523:2(565)Online publication date: 4-Jan-2023
https://doi.org/10.3390/s23020565
Strenge BSchack T(2023)Cognitive assistance for action selection: Challenges and approachesFrontiers in Psychology10.3389/fpsyg.2022.103185813Online publication date: 4-Jan-2023
https://doi.org/10.3389/fpsyg.2022.1031858
Show More Cited By

Index Terms

Deep learning for action recognition in augmented reality assistance systems
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Activity recognition and understanding
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality

Recommendations

Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Augmented reality as perceptual reality
VSMM'06: Proceedings of the 12th international conference on Interactive Technologies and Sociotechnical Systems

As shown in Paul Milgram et al’s Reality-Virtuality Continuum (1994), Augmented Reality occupies a very unique status in the spectrum of Mixed Reality. Unlike Virtual Reality, which is completely made up of the virtual and has been the most important ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...

Comments

Information & Contributors

Information

Published In

SIGGRAPH '17: ACM SIGGRAPH 2017 Posters

July 2017

173 pages

ISBN:9781450350150

DOI:10.1145/3102163

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 July 2017

Check for updates

Author Tags

Qualifiers

Poster

Conference

SIGGRAPH '17

Sponsor:

SIGGRAPH

SIGGRAPH '17: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 30 - August 3, 2017

California, Los Angeles

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
561
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)1

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Fink LKostolani DTrautner TSchlund S(2024)Make some Noise: Acoustic Classification of Manual Work Steps Towards Adaptive Assistance SystemsProcedia CIRP10.1016/j.procir.2024.07.024127(135-140)Online publication date: 2024
https://doi.org/10.1016/j.procir.2024.07.024
Ali SAbdullah Armand TAthar AHussain AAli MYaseen MJoo MKim H(2023)Metaverse in Healthcare Integrated with Explainable AI and Blockchain: Enabling Immersiveness, Ensuring Trust, and Providing Patient Data SecuritySensors10.3390/s2302056523:2(565)Online publication date: 4-Jan-2023
https://doi.org/10.3390/s23020565
Strenge BSchack T(2023)Cognitive assistance for action selection: Challenges and approachesFrontiers in Psychology10.3389/fpsyg.2022.103185813Online publication date: 4-Jan-2023
https://doi.org/10.3389/fpsyg.2022.1031858
Xu XYu AJonker TTodi KLu FQian XEvangelista Belo JWang TLi MMun AWu TShen JZhang TKokhlikyan NWang FSorenson PKim SBenko H(2023)XAIR: A Framework of Explainable AI in Augmented RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581500(1-30)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581500
Wang CYan J(2023)A Comprehensive Survey of RGB-Based and Skeleton-Based Human Action RecognitionIEEE Access10.1109/ACCESS.2023.328231111(53880-53898)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3282311
Feng MMeunier J(2022)Skeleton Graph-Neural-Network-Based Human Action Recognition: A SurveySensors10.3390/s2206209122:6(2091)Online publication date: 8-Mar-2022
https://doi.org/10.3390/s22062091
Zhabokrytskyi I(2022)Current State and Prospects of Increasing the Functionality of Augmented Reality Using Neural NetworksÈlektronnoe modelirovanie10.15407/emodel.44.05.07344:5(73-89)Online publication date: 10-Sep-2022
https://doi.org/10.15407/emodel.44.05.073
Voillemin TWannous HVandeborre J(2022)FirstPiano: A New Egocentric Hand Action Dataset Oriented Towards Augmented Reality ApplicationsImage Analysis and Processing – ICIAP 202210.1007/978-3-031-06433-3_15(170-181)Online publication date: 15-May-2022
https://doi.org/10.1007/978-3-031-06433-3_15
Schack TStrenge BEssig K(2021)Approaches for Cognitive Assistance in Industry 4.0 AssemblyArtificial Intelligence in Industry 4.010.1007/978-3-030-61045-6_4(45-54)Online publication date: 28-Feb-2021
https://doi.org/10.1007/978-3-030-61045-6_4
Lapointe JMolyneaux HAllili M(2020)A Literature Review of AR-Based Remote Guidance Tasks with User StudiesVirtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications10.1007/978-3-030-49698-2_8(111-120)Online publication date: 10-Jul-2020
https://doi.org/10.1007/978-3-030-49698-2_8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

Supplementary Material

References

Cited By

Index Terms

Recommendations

Haptics in Augmented Reality

Augmented reality as perceptual reality

Multimodal augmented reality: the norm rather than the exception

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations