demonstration

A Web-Based Remote Assistance System with Gravity-Aware 3D Hand Gesture Visualization

Authors:

Chelhwon Kim,

Patrick Chiu,

Yulius TjahjadiAuthors Info & Claims

ISS '19: Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces

Pages 319 - 322

https://doi.org/10.1145/3343055.3360742

Published: 10 November 2019 Publication History

Get Access

Abstract

We present a remote assistance system that enables a remotely located expert to provide guidance using hand gestures to a customer who performs a physical task in a different location. The system is built on top of a web-based real-time media communication framework which allows the customer to use a commodity smartphone to send a live video feed to the expert, from which the expert can see the view of the customer's workspace and can show his/her hand gestures over the video in real-time. The expert's hand gesture is captured with a hand tracking device and visualized with a rigged 3D hand model on the live video feed. The system can be accessed via a web browser, and it does not require any app software to be installed on the customer's device. Our system supports various types of devices including smartphone, tablet, desktop PC, and smart glass. To improve the collaboration experience, the system provides a novel gravity-aware hand visualization technique.

Supplementary Material

MP4 File (issde07.mp4)

Supplemental video

Download
29.44 MB

References

[1]

Leila Alem, Franco Tecchia, and Weidong Huang. 2011. Remote tele-assistance system for maintenance operators in mines. (2011).

Google Scholar

[2]

Judith Amores, Xavier Benavides, and Pattie Maes. 2015. Showme: A remote collaboration system that supports immersive gestural communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1343--1348.

Digital Library

Google Scholar

[3]

Weidong Huang and Leila Alem. 2013. HandsinAir: a wearable system for remote collaboration on physical tasks. In Proceedings of the 2013 conference on Computer supported cooperative work companion. ACM, 153--156.

Digital Library

Google Scholar

[4]

Seungwon Kim, Gun Lee, Nobuchika Sakata, and Mark Billinghurst. 2014. Improving co-presence with augmented visual communication cues for sharing experience through video conference. In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 83--92.

Crossref

Google Scholar

[5]

David S Kirk and Danaë Stanton Fraser. 2005. The effects of remote gesturing on distance instruction. In Proceedings of th 2005 conference on Computer support for collaborative learning: learning 2005: the next 10 years! International Society of the Learning Sciences, 301--310.

Digital Library

Google Scholar

[6]

Rajinder S Sodhi, Brett R Jones, David Forsyth, Brian P Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 179--188.

Digital Library

Google Scholar

[7]

Hongling Sun, Yue Liu, Zhenliang Zhang, Xiaoxu Liu, and Yongtian Wang. 2018. Employing Different Viewpoints for Remote Guidance in a Collaborative Augmented Environment. In ChineseCHI '18. ACM, 64--70.

Digital Library

Google Scholar

Cited By

View all

Batch AButcher PRitsos PElmqvist N(2024)Wizualization: A “Hard Magic” Visualization System for Immersive and Ubiquitous AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332658030:1(507-517)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326580
Mizuno KKamimura KEbihara TWakatsuki NZempo K(2021)2D-3D Transformation of Pointing Objects Based on Skeletal Body Surface Models for Remote Collaboration2021 IEEE 10th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE53005.2021.9621900(259-262)Online publication date: 12-Oct-2021
https://doi.org/10.1109/GCCE53005.2021.9621900
Calandra DCannavò ALamberti F(2021)Improving AR-powered remote assistance: a new approach aimed to foster operator’s autonomy and optimize the use of skilled resourcesThe International Journal of Advanced Manufacturing Technology10.1007/s00170-021-06871-4114:9-10(3147-3164)Online publication date: 21-Apr-2021
https://doi.org/10.1007/s00170-021-06871-4

Index Terms

A Web-Based Remote Assistance System with Gravity-Aware 3D Hand Gesture Visualization
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Web-based interaction
    2. Interaction techniques
      1. Gestural input

Recommendations

The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments
MuC '22: Proceedings of Mensch und Computer 2022

Natural user interfaces are on the rise. Manufacturers for Augmented, Virtual, and Mixed Reality head mounted displays are increasingly integrating new sensors into their consumer grade products, allowing gesture recognition without additional hardware. ...
Shape-Based Features for Optimized Hand Gesture Recognition

Gesture recognition is the most intuitive form of human-computer interface. Hand gestures provide a natural way for humans to interact with computers to perform a variety of different applications. However, factors such as complexity of hand gesture ...
Characteristics of hand gesture navigation: a case study using a wearable device (MYO)
British HCI '15: Proceedings of the 2015 British HCI Conference

The understanding of characteristics of gestures, such as ergonomic, cognitive and social aspects could potentially contribute to an interface design mediated by hand gestures. In this research, a questionnaire model is implemented to suggest some ...

Comments

Information & Contributors

Information

Published In

ISS '19: Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces

November 2019

450 pages

ISBN:9781450368919

DOI:10.1145/3343055

General Chairs:
Bongshin Lee
Microsoft Research, USA
,
Geehyuk Lee
KAIST, South Korea
,
Program Chairs:
Stacey Scott
University of Guelph, Canada
,
Melanie Tory
Tableau Software, USA
,
Jeonghyun Kim
Korea University, South Korea

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2019

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

ISS '19

Sponsor:

SIGCHI

ISS '19: Interactive Surfaces and Spaces

November 10 - 13, 2019

Daejeon, Republic of Korea

Acceptance Rates

ISS '19 Paper Acceptance Rate 26 of 85 submissions, 31%;

Overall Acceptance Rate 147 of 533 submissions, 28%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
143
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)4

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Batch AButcher PRitsos PElmqvist N(2024)Wizualization: A “Hard Magic” Visualization System for Immersive and Ubiquitous AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332658030:1(507-517)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326580
Mizuno KKamimura KEbihara TWakatsuki NZempo K(2021)2D-3D Transformation of Pointing Objects Based on Skeletal Body Surface Models for Remote Collaboration2021 IEEE 10th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE53005.2021.9621900(259-262)Online publication date: 12-Oct-2021
https://doi.org/10.1109/GCCE53005.2021.9621900
Calandra DCannavò ALamberti F(2021)Improving AR-powered remote assistance: a new approach aimed to foster operator’s autonomy and optimize the use of skilled resourcesThe International Journal of Advanced Manufacturing Technology10.1007/s00170-021-06871-4114:9-10(3147-3164)Online publication date: 21-Apr-2021
https://doi.org/10.1007/s00170-021-06871-4

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments

Shape-Based Features for Optimized Hand Gesture Recognition

Characteristics of hand gesture navigation: a case study using a wearable device (MYO)