research-article

Exploring Audio, Visual, and Tactile Cues for Synchronous Remote Assistance

Authors:

Sebastian Günther,

Daniel Avrahami,

Max MühlhäuserAuthors Info & Claims

PETRA '18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

Pages 339 - 344

https://doi.org/10.1145/3197768.3201568

Published: 26 June 2018 Publication History

Abstract

Today, remote collaboration techniques between field workers and remotely located experts mainly focus on traditional communication channels, such as voice- or video-conferencing. Those systems may not be suitable in every situation or the communication gets cumbersome if both parties do not share a common ground. In this paper, we explore three supporting communication channels based on audio, visual, and tactile cues. We built a prototypical application implementing those cues and evaluated them in a user study. Based on the user feedback, we report first insights for building remote assistance systems utilizing additional cues.

References

[1]

L. Alem and J. Li. 2011. A Study of Gestures in a Video-mediated Collaborative Assembly Task. Adv. in Hum.-Comp. Int. 2011, Article 1 (Jan. 2011), 7 pages.

Digital Library

[2]

J. Amores, X. Benavides, and P. Maes. 2015. ShowMe: A Remote Collaboration System That Supports Immersive Gestural Communication. In Proc. CHI EA '15. ACM, New York, NY, USA, 1343--1348.

Digital Library

[3]

X. Benavides, J. Amores, and P. Maes. 2015. Remot-IO: A System for Reaching into the Environment of a Remote Collaborator. In Adj. Proc. UIST '15. ACM, New York, NY, USA, 99--100.

Digital Library

[4]

M. Funk, J. Heusler, E. Akcay, K. Weiland, and A. Schmidt. 2016. Haptic, Auditory, or Visual?: Towards Optimal Error Feedback at Manual Assembly Workplaces. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments - PETRA '16. ACM Press, New York, New York, USA, 1--6.

Digital Library

[5]

S. Fussell and R. Krauss. 1989. The effects of intended audience on message production and comprehension: Reference in a common ground framework. Journal of Experimental Social Psychology 25, 3 (1989), 203--219.

[6]

S. Fussell, L. Setlock, and R. Kraut. 2003. Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks. In Proc. CHI '03. ACM Press, New York, New York, USA, 513.

Digital Library

[7]

S. Fussell, L. Setlock, J. Yang, J. Ou, E. Mauer, and A. Kramer. 2004. Gestures over Video Streams to Support Remote Collaboration on Physical Tasks. Hum.-Comput. Interact. 19, 3 (Sept. 2004), 273--309.

Digital Library

[8]

S. Gauglitz, B. Nuernberger, M. Turk, and T. Höllerer. 2014. In Touch with the Remote World: Remote Collaboration with Augmented Reality Drawings and Virtual Navigation. In Proc. of VRST '14. ACM, New York, NY, USA, 197--205.

Digital Library

[9]

S. Gauglitz, B. Nuernberger, M. Turk, and T. Höllerer. 2014. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proc. UIST '14. ACM Press, New York, New York, USA, 449--459.

Digital Library

[10]

P. Gurevich, J. Lanir, B. Cohen, and R. Stone. 2012. TeleAdvisor: A Versatile Augmented Reality Tool for Remote Assistance. In Proc. CHI '12. ACM, New York, NY, USA, 619--622.

Digital Library

[11]

S. Günther, F. Müller, M. Funk, J. Kirchner, N. Dezfuli, and M. Mühlhäuser. 2018. TactileGlove: Assistive Spatial Guidance in 3D Space through Vibrotactile Navigation. In Proceedings of the 11th ACM International Conference on PErvasive Technologies Related to Assistive Environments - PETRA '18. ACM.

Digital Library

[12]

W. Heuten, N. Henze, S. Boll, and M. Pielot. 2008. Tactile Wayfinder. In Proc. NordiCHI '08. ACM Press, New York, New York, USA, 172.

[13]

S. Holland, R. D. Morse, and H. Gedenryd. 2002. AudioGPS: Spatial Audio Navigation with a Minimal Attention Interface. Personal and Ubiquitous Computing 6, 4 (2002), 253--259.

Digital Library

[14]

W. Huang and L. Alem. 2011. Supporting Hand Gestures in Mobile Remote Collaboration: A Usability Evaluation. In Proc. BCS-HCI '11. BCS, Swinton, UK, 211--216. http://dl.acm.org/citation.cfm?id=2305316.2305356

Digital Library

[15]

S. Kasahara and J. Rekimoto. 2015. JackIn Head: Immersive Visual Telepresence System with Omnidirectional Wearable Camera for Remote Collaboration. In VRST '15. ACM, NY, USA, 217--225.

Digital Library

[16]

B. Katz, S. Kammoun, G. Parseihian, O. Gutierrez, A. Brilhault, M. Auvray, P. Truillet, M. Denis, S. Thorpe, and C. Jouffrais. 2012. NAVIG: augmented reality guidance system for the visually impaired. Virtual Reality 16, 4 (2012), 253--269.

Digital Library

[17]

O. B. Kaul and M. Rohs. 2017. HapticHead: A Spherical Vibrotactile Grid Around the Head for 3D Guidance in Virtual and Augmented Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3729--3740.

Digital Library

[18]

D. Kirk and D. Stanton Fraser. 2006. Comparing remote gesture technologies for supporting collaborative physical tasks. In CHI '06. ACM Press, New York, NY, USA, 1191.

Digital Library

[19]

T. Kosch, R. Kettner, M. Funk, and A. Schmidt. 2016. Comparing Tactile, Auditory, and Visual Assembly Error-Feedback for Workers with Cognitive Impairments. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 53--60.

Digital Library

[20]

R. Kraut, S. Fussell, and J. Siegel. 2003. Visual Information As a Conversational Resource in Collaborative Physical Tasks. Hum.-Comput. Interact. 18, 1 (2003), 13--49.

Digital Library

[21]

H. Kuzuoka. 1992. Spatial workspace collaboration. In Proc. of CHI '92. ACM Press, New York, New York, USA, 533--540.

Digital Library

[22]

S. Lee and T. Starner. 2010. BuzzWear. In Proc. of CHI '10. ACM Press, New York, New York, USA, 433.

[23]

R. Stalnaker. 2002. Common Ground. Linguistics and Philosophy 25, 5--6 (2002), 701--721.

[24]

V. Sundareswaran, K. Wang, S. Chen, R. Behringer, J. McGee, C. Tam, and P. Zahorik. 2003. 3D Audio Augmented Reality: Implementation and Experiments. In Proc. of ISMAR '03. IEEE Computer Society, Washington, DC, USA, 296-. http://dl.acm.org/citation.cfm?id=946248.946841

Digital Library

[25]

F. Tecchia, L. Alem, and W. Huang. 2012. 3D helping hands. In VRCAI '12. ACM Press, New York, NY, USA, 323.

[26]

K. Tsukada and M. Yasumura. 2004. ActiveBelt: Belt-Type Wearable Tactile Display for Directional Navigation. Springer Berlin Heidelberg, Berlin, Heidelberg, 384--399.

[27]

J. Zelek. 2005. Seeing by touch (haptics) for wayfinding. International Congress Series 1282 (2005), 1108 - 1112.

[28]

J. Zelek, S. Bromley, D. Asmar, and D. Thompson. 2003. A haptic glove as a tactile-vision sensory substitution for wayfinding. Journal of Visual Impairment and Blindness 97, 10 (2003), 621--632. http://www.afb.org/jvib/

Cited By

Tanprasert TDai JMcGrenere J(2024)HelpCall: Designing Informal Technology Assistance for Older Adults via VideoconferencingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642938(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642938
Fang WChen LZhang THu HBi J(2024)Co2iAR: Co-located audio-visual enabled mobile collaborative industrial AR wiring harness assemblyRobotics and Computer-Integrated Manufacturing10.1016/j.rcim.2024.10279590(102795)Online publication date: Dec-2024
https://doi.org/10.1016/j.rcim.2024.102795
Memmesheimer VLöber JEbert A(2024)AWARESCUES: Awareness Cues Scaling with Group Size and Extended Reality DevicesVirtual, Augmented and Mixed Reality10.1007/978-3-031-61041-7_4(44-59)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-61041-7_4
Show More Cited By

Index Terms

Exploring Audio, Visual, and Tactile Cues for Synchronous Remote Assistance
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. HCI theory, concepts and models

Recommendations

A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Supporting natural communication cues is critical for people to work together remotely and face-to-face. In this paper we present a Mixed Reality (MR) remote collaboration system that enables a local worker to share a live 3D panorama of his/her ...
Which notification is better? Comparing Visual, Audio and Tactile Cues for Asynchronous Mixed Reality (MR) Remote Collaboration: A User Study
MUM '22: Proceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia

Although Mixed Reality (MR) has been explored to support scenarios of remote collaboration, the majority of studies reported in literature have focused on synchronous use-cases, where all team-members can interact in real-time. Hence, an opportunity ...
The Effect of Collaboration Styles and View Independence on Video-Mediated Remote Collaboration

This paper investigates how different collaboration styles and view independence affect remote collaboration. Our remote collaboration system shares a live video of a local user's real-world task space with a remote user. The remote user can have an ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PETRA '18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference

June 2018

591 pages

ISBN:9781450363907

DOI:10.1145/3197768

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

NSF: National Science Foundation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

PETRA '18

PETRA '18: The 11th PErvasive Technologies Related to Assistive Environments Conference

June 26 - 29, 2018

Corfu, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
439
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)5

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tanprasert TDai JMcGrenere J(2024)HelpCall: Designing Informal Technology Assistance for Older Adults via VideoconferencingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642938(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642938
Fang WChen LZhang THu HBi J(2024)Co2iAR: Co-located audio-visual enabled mobile collaborative industrial AR wiring harness assemblyRobotics and Computer-Integrated Manufacturing10.1016/j.rcim.2024.10279590(102795)Online publication date: Dec-2024
https://doi.org/10.1016/j.rcim.2024.102795
Memmesheimer VLöber JEbert A(2024)AWARESCUES: Awareness Cues Scaling with Group Size and Extended Reality DevicesVirtual, Augmented and Mixed Reality10.1007/978-3-031-61041-7_4(44-59)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-61041-7_4
Huang WWakefield MRasmussen TKim SBillinghurst MHuang WBillinghurst MAlem LXiao CRasmussen T(2024)Communication Cues for Remote GuidanceComputer‐Supported Collaboration10.1002/9781119719830.ch4(81-114)Online publication date: 24-May-2024
https://doi.org/10.1002/9781119719830.ch4
Yan YBai XHe WWang SZhang XWang PLiu LZhang B(2023)Can you notice my attention? A novel information vision enhancement method in MR remote collaborative assemblyThe International Journal of Advanced Manufacturing Technology10.1007/s00170-023-11652-2127:3-4(1835-1857)Online publication date: 2-Jun-2023
https://doi.org/10.1007/s00170-023-11652-2
Aoyama KMaeda KUeoka RMakioka SSakura NNakashima KHirose MAmemiya T(2022)Improving Remote Spatial Understanding by Transmitting Spherical Images via Video Chat ApplicationsInternational Journal of Automation Technology10.20965/ijat.2022.p028616:3(286-295)Online publication date: 5-May-2022
https://doi.org/10.20965/ijat.2022.p0286
Günther SRasch JSchön DMüller FSchmitz MRiemann JMatviienko AMühlhäuser M(2022)Smooth as Steel Wool: Effects of Visual Stimuli on the Haptic Perception of Roughness in Virtual RealityProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517454(1-17)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517454
Rasmussen TFeuchtner THuang WGrønbæk K(2022)Supporting workspace awareness in remote assistance through a flexible multi-camera system and Augmented Reality awareness cuesJournal of Visual Communication and Image Representation10.1016/j.jvcir.2022.10365589:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.jvcir.2022.103655
Huang WWakefield MRasmussen TKim SBillinghurst M(2022)A review on communication cues for augmented reality based remote guidanceJournal on Multimodal User Interfaces10.1007/s12193-022-00387-116:2(239-256)Online publication date: 12-Apr-2022
https://doi.org/10.1007/s12193-022-00387-1
Wang YWang PLuo ZYan Y(2022)A novel AR remote collaborative platform for sharing 2.5D gestures and gazeThe International Journal of Advanced Manufacturing Technology10.1007/s00170-022-08747-7119:9-10(6413-6421)Online publication date: 21-Jan-2022
https://doi.org/10.1007/s00170-022-08747-7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents