Article

A Review of Augmented Reality-Based Human-Computer Interaction Applications of Gesture-Based Interaction

Author:

Chutisant KerdvibulvechAuthors Info & Claims

HCI International 2019 – Late Breaking Papers: 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings

Pages 233 - 242

https://doi.org/10.1007/978-3-030-30033-3_18

Published: 26 July 2019 Publication History

Abstract

In recent years, augmented reality (AR) is an extremely growing field in information technology, computer science, and computer engineering. Although there are many recent works that use augmented reality for different purposes, most of the existing works do not focus on reviewing recent augmented reality-based human-computer interaction applications regarding gesture-based interaction. Therefore, we focus on a different goal from them. In this paper, we study robust methodologies that researchers have recently achieved gesture-based interaction for using in augmented reality-based human-computer interaction (HCI) applications. To begin with, we explore the recognitions of hand gestures using augmented reality. Next, we explore the possibilities of utilizing augmented reality for gesture-based interaction. We also give a suggestion and present a future scenario for gesture-based interaction and augmented reality. We believe that this would help the interactions that humans would have with modern innovations in an integrated cross-disciplinary area in the near future of human-computer interaction.

References

[1]

Asadi-Aghbolaghi, M., et al.: A survey on deep learning based approaches for action and gesture recognition in image sequences. In: 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG), pp. 476–483 (2017)

[2]

Kerdvibulvech C Miesenberger K, Bühler C, and Penaz P A review of computer-based gesture interaction methods for supporting disabled people with special needs Computers Helping People with Special Needs 2016 Cham Springer 503-506

[3]

Al-Shamayleh AS, Ahmad RB, Abushariah MA, Alam KA, and Jomhari N A systematic literature review on vision based gesture recognition techniques Multimedia Tools Appl. 2018 77 21 28121-28184

[4]

Reski, N., Alissandrakis, A.: Using an augmented reality cube-like interface and 3D gesture-based interaction to navigate and manipulate data. In: 11th International Symposium on Visual Information Communication and Interaction (VINCI), pp. 92–96 (2018)

[5]

Jahani H, Alyamani HJ, Kavakli M, Dey A, and Billinghurst M Maedche A, vom Brocke J, and Hevner A User evaluation of hand gestures for designing an intelligent in-vehicle interface Designing the Digital Transformation 2017 Cham Springer 104-121

[6]

Hürst W and van Wezel C Gesture-based interaction via finger tracking for mobile augmented reality Multimedia Tools Appl. 2013 62 1 233-258

[7]

Chutisant, K.: A methodology for hand and finger motion analysis using adaptive probabilistic models. EURASIP J. Embed. Syst. Number 18 2014

[8]

Mueller, F., Mehta, D., Sotnychenko, O., Sridhar, S., Casas, D., Theobalt, C.: Real-time hand tracking under occlusion from an egocentric RGB-D sensor. In: International Conference on Computer Vision (ICCV), pp. 1163–1172 (2017)

[9]

Kerdvibulvech C Hand tracking by extending distance transform and hand model in real-time Pattern Recogn. Image Anal. 2015 25 3 437-441

[10]

Kerdvibulvech C De Paolis LT, Bourdot P, and Mongelli A An innovative real-time mobile augmented reality application in arts Augmented Reality, Virtual Reality, and Computer Graphics 2017 Cham Springer 251-260

[11]

Kim M and Lee JY Touch and hand gesture-based interactions for directly manipulating 3D virtual objects in mobile augmented reality Multimedia Tools Appl. 2016 75 23 16529-16550

[12]

Frikha R, Ejbali R, Zaied M, and Ben Amar C Jackowski K, Burduk R, Walkowiak K, Woźniak M, and Yin H Natural gesture based interaction with virtual heart in augmented reality Intelligent Data Engineering and Automated Learning – IDEAL 2015 2015 Cham Springer 457-465

[13]

Gül LF Studying gesture-based interaction on a mobile augmented reality application for co-design activity J. Multimodal User Interfaces 2018 12 2 109-124

[14]

Gül LF, Halıcı M, Uzun C, and Esengün M Luo Y Understanding the impact of mobile augmented reality on co-design cognition and co-modelling Cooperative Design, Visualization, and Engineering 2016 Cham Springer 362-370

[15]

Sorgalla, J., Fleck, J., Sachweh, S.: ARGI: augmented reality for gesture-based interaction in variable smart environments. In: 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP), HUCAPP, vol. 2, pp. 102–107 (2018)

[16]

Ens, B., Quigley, A., Yeo, H.-S., Irani, P., Piumsomboon, T., Billinghurst, M.: Counterpoint: exploring mixed-scale gesture interaction for AR applications. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI), 2018, Paper No. LBW120 (2018)

[17]

Simmons, H., Devi, R., Ens, B., Billinghurst, M.: Comparative evaluation of sensor devices for micro-gestures. In: International Conference on Artificial Reality and Telexistence & Eurographics Symposium on Virtual Environments (ICAT-EGVE), Posters and Demos, pp. 23–24 (2017)

[18]

Lugtenberg G et al. Multimodal augmented reality - augmenting auditory-tactile feedback to change the perception of thickness MultiMedia Model. (MMM) 2018 1 369-380

[19]

Lugtenberg, G., Sandor, C., Hürst, W., Plopski, A., Taketomi, T., Kato, H.: Changing perception of physical properties using multimodal augmented reality: position paper. In: Proceedings of the 2016 Workshop on Multimodal Virtual and Augmented Reality (MVAR), in Conjunction with International Conference on Multimodal Interaction (ICMI), pp. 8:1–8:4 (2016)

[20]

Xue, Y., Zhao, L., Xue, M., Fu, J.: Gesture interaction and augmented reality based hand rehabilitation supplementary system. In: 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), INSPEC Accession Number: 18311447, Chongqing, China, 12–14 October 2018

[21]

Zhen, L., Jing, C., Zixiang, Z., Qiushuo, T., Ningsheng, H.: An optical see-through augmented reality system with gesture-based interaction. In: International Conference on Virtual Reality and Visualization (ICVRV), INSPEC Accession Number: 16918263, Hangzhou, China, pp. 447–452, 24–26 September 2016

[22]

Aliprantis, J., Konstantakis, M., Nikopoulou, R., Mylonas, P., Caridakis, G.: Natural interaction in augmented reality context. Visual Pattern Extraction and Recognition for Cultural Heritage Understanding (VIPERC), Piza, Italy, 30 January 2019

[23]

Li Y, Huang J, Tian F, Wang H-A, and Dai G-Z Gesture interaction in virtual reality Virtual Reality Intell. Hardware 2019 1 1 84-112

[24]

Khan, H., Lee, G.A., Hoermann, S., Clifford, R.M.S., Billinghurst, M., Lindeman, R.W.: Evaluating the Effects of Hand-gesture-based Interaction with Virtual Content in a 360° Movie. In: ICAT-EGVE 63–70 (2017)

[25]

Céspedes-Hernández D, González-Calleros JM, Guerrero-García J, and Rodríguez-Vizzuett L Agredo-Delgado Vanessa and Ruiz Pablo H Gesture-based interaction for virtual reality environments through user-defined commands Human-Computer Interaction 2019 Cham Springer 143-157

[26]

Kerdvibulvech, C.: Hybrid model of human hand motion for cybernetics application. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), San Diego, pp. 2367–2372 (2014)

[27]

Tzorakoleftherakis E, Ansari A, Wilson AD, Schultz JA, and Murphey TD Model-based reactive control for hybrid and high-dimensional robotic systems IEEE Robot. Autom. Lett. 2016 1 1 431-438

[28]

Roo, J.S., Hachet, M.: Towards a hybrid space combining spatial augmented reality and virtual reality. In: IEEE Symposium on 3D User Interfaces (3DUI), Los Angeles, CA, pp. 195–198 (2017)

Cited By

Dash ABalaji KDogra DKim B(2024)Interactions with 3D virtual objects in augmented reality using natural gesturesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03175-440:9(6449-6462)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s00371-023-03175-4

Recommendations

Using an Augmented Reality Cube-Like Interface and 3D Gesture-Based Interaction to Navigate and Manipulate Data
VINCI '18: Proceedings of the 11th International Symposium on Visual Information Communication and Interaction

In this paper we describe our work-in-progress to create an interface that enables users to browse and select data within an Augmented Reality environment, using a virtual cube object that can be interacted with through 3D gestural input. We present the ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Occlusion based interaction methods for tangible augmented reality environments
VRCAI '04: Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry

Traditional Tangible Augmented Reality (Tangible AR) interfaces combine a mixture of tangible user interface and augmented reality technology, complementing each other for novel interaction methods and real world anchored visualization. However, well ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

HCI International 2019 – Late Breaking Papers: 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings

Jul 2019

616 pages

ISBN:978-3-030-30032-6

DOI:10.1007/978-3-030-30033-3

Editor:
Constantine Stephanidis
University of Crete and Foundation for Research and Technology - Hellas (Forth), Heraklion, Greece

© Springer Nature Switzerland AG 2019.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 26 July 2019

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dash ABalaji KDogra DKim B(2024)Interactions with 3D virtual objects in augmented reality using natural gesturesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03175-440:9(6449-6462)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s00371-023-03175-4

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents