Towards the Design of Effective Haptic and Audio Displays for Augmented Reality and Mixed Reality Applications
Authors: 
Gang Ren, Side Wei, Eamonn O’Neill, Fenfang Chen Academic Editor: Hoshang KolivandAuthors Info & Claims
Gang Ren, Side Wei, Eamonn O’Neill, Fenfang Chen Academic Editor: Hoshang KolivandAuthors Info & ClaimsAbstract
Augmented Reality (AR) and Mixed Reality (MR) technology has potential for supporting mobile applications. However, nonvisual interaction modalities are undervalued and underused in AR/MR applications. Visual displays can be ineffective or inappropriate in some situations such as walking or driving. Meanwhile, nonvisual modalities are becoming increasingly important in mobile user experiences. In this paper, we report two studies investigating nonvisual interaction modalities such as audio and haptic displays with mobile AR/MR applications. In the first study, we investigate a range of design factors for haptic and audio displays, including rhythm, amplitude, and their combination in representing tourism information to users with a mobile phone. The results show a main effect for Interaction modality, with identification rates highest for information represented in a combined Haptic-Audio display. In the second study, we investigate target location tasks in 3D space using spatial audio feedback and a head-mounted display. We evaluate several design factors including audio feedback device, volume, rhythm, and the target’s horizontal and vertical position. Results show that the vertical positions are very difficult to locate, and overall our participants prefer audio cues with loud volume and fast rhythm. Finally, we propose practical audio and haptic display design guidelines for AR/MR applications.
References
[1]
C. D. Kounavis, A. E. Kasimati, and E. D. Zamani, “Enhancing the tourism experience through mobile augmented reality: Challenges and prospects,” International Journal of Engineering Business Management, vol. 4, no. 1, pp. 1–6, 2012.
[2]
S. Wei, G. Ren, and E. O'Neill, “Haptic and audio displays for augmented reality tourism applications,” in Proceedings of the 2014 IEEE Haptics Symposium, HAPTICS 2014, pp. 485–488, usa, February 2014.
[3]
Y. Vazquez-Alvarez, I. Oakley, and S. A. Brewster, “Auditory display design for exploration in mobile audio-augmented reality,” Personal and Ubiquitous Computing, vol. 16, no. 8, pp. 987–999, 2012.
[4]
A. Poon, “The 'new tourism' revolution,” Tourism Management, vol. 15, no. 2, pp. 91–92, 1994.
[5]
J. Krippendorf, “The new tourist - turning point for leisure and travel,” Tourism Management, vol. 7, no. 2, pp. 131–135, 1986.
[6]
D. Park, T.-J. Nam, and C.-K. Shi, “Designing an immersive tour experience system for cultural tour sites,” in Proceedings of the Conference on Human Factors in Computing Systems, CHI EA 2006, pp. 1193–1198, can, April 2006.
[7]
E. C. Kansa and E. Wilde, “Tourism, peer production, and location-based service design,” in Proceedings of the 2008 IEEE International Conference on Services Computing, SCC 2008, pp. 629–636, usa, July 2008.
[8]
T. Gleue and P. Dähne, “Design and Implementation of a Mobile Device for Outdoor Augmented Reality in the ARCHEOGUIDE Project,” in Proceedings of the VAST 2001 Virtual Reality, Archeology, and Cultural Heritage, pp. 161–168, November 2001.
[9]
S. C. Yuen, G. Yaoyuneyong, and E. Johnson, “Augmented Reality: An Overview and Five Directions for AR in Education,” Journal of Educational Technology Development and Exchange (JETDE), vol. 4, no. 1, pp. 119–140, 2011.
[10]
J. Ryu, J. Jung, G. Park, and S. Choi, “Psychophysical model for vibrotactile rendering in mobile devices,” Presence: Teleoperators and Virtual Environments, vol. 19, no. 4, pp. 364–387, 2010.
[11]
C. Giachritsis, G. Randall, and S. Roselier, “Development of tactile navigational patterns,” in Proceedings of the 7th International Workshop on Haptic and Audio Interaction Design, p. 27, Lund, Sweden, 2012.
[12]
M. Srikulwong and O. E. Neill, “Towards the development of a tactilebased pedestrian navigation display for urban environments,” in Proceedings of the IEEE World Haptics Conference, pp. 503–508, 2013.
[13]
D. K. Mcgookin and S. A. Brewster, “Understanding Concurrent Earcons: Applying Auditory Scene Analysis Principles to Concurrent Earcon Recognition,” ACM Transactions on Applied Perception, vol. 1, no. 2, pp. 130–155, 2004.
[14]
D. Ternes and K. E. Maclean, “Designing large sets of haptic icons with rhythm,” in Inproceedings of the International Conference on Human Haptic Sensing and Touch Enabled, pp. 199–208, Springer, 2008.
[15]
H.-Y. Chen, J. Park, H. Z. Tan, and S. Dai, “Redundant coding of simulated tactile key clicks with audio signals,” in Proceedings of the 2010 IEEE Haptics Symposium, HAPTICS 2010, pp. 29–34, usa, March 2010.
[16]
F. Bork, B. Fuers, A.-K. Schneider, F. Pinto, C. Graumann, and N. Navab, “Auditory and visio-temporal distance coding for 3-dimensional perception in medical augmented reality,” in Proceedings of the 14th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2015, pp. 7–12, jpn, October 2015.
[17]
D. J. Finnegan, E. O'Neill, and M. J. Proulx, “Compensating for distance compression in audiovisual virtual environments using incongruence,” in Proceedings of the 34th Annual Conference on Human Factors in Computing Systems, CHI 2016, pp. 200–212, usa, May 2016.
[18]
T. Chatzidimitris, D. Gavalas, and D. Michael, “SoundPacman: Audio augmented reality in location-based games,” in Proceedings of the 18th Mediterranean Electrotechnical Conference, MELECON 2016, cyp, April 2016.
[19]
D. R. Begault, E. M. Wenzel, M. Godfroy, J. D. Miller, and M. R. Anderson, “Applying spatial audio to human interfaces: 25 years of NASA experience,” in Proceedings of the AES 40th International Conference on Spatial Audio: Sense the Sound of Space, jpn, October 2010.
[20]
L. M. Brown, S. A. Brewster, and H. C. Purchase, “Multidimensional tactons for non-visual information presentation in mobile devices,” in Proceedings of the 8th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2006, pp. 231–238, fin, September 2006.
Index Terms
- Towards the Design of Effective Haptic and Audio Displays for Augmented Reality and Mixed Reality Applications
Index terms have been assigned to the content through auto-classification.
Recommendations
Haptic interaction in augmented reality
MM '09: Proceedings of the 17th ACM international conference on MultimediaIn an interactive augmented reality (AR) system, people are, most of the time, immersed in an environment rich of visual and audio feedback. The sense of touch is lacking in most of the AR environment. We present an augmented reality system where people ...
Haptic augmented reality: Taxonomy and an example of stiffness modulation
Haptic augmented reality (AR) enables the user to feel a real environment augmented with synthetic haptic stimuli. This article addresses two important topics in haptic AR. First, a new taxonomy for haptic AR is established based on a composite visuo-...
Mixed Reality MIDI Keyboard Demonstration
AM '17: Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music ExperiencesThe Mixed Reality MIDI Keyboard is a prototype designed to augment virtual reality experiences through the inclusion of a physical interface which aligns the user's senses with the virtual environment. It also serves as a platform on which the uses of ...
Comments
Information & Contributors
Information
Published In
Copyright © 2018 Gang Ren et al.
This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Publisher
Hindawi Limited
London, United Kingdom
Publication History
Published: 01 January 2018
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Feb 2025