Enriching Industrial Training Experience in Virtual Reality with Pseudo-Haptics and Vibrotactile Stimulation
Article No.: 24, Pages 1 - 11
Abstract
Virtual Reality (VR) technology facilitates effective, flexible, and safe industrial training for novice technicians when on-site training is not feasible. However, previous research has shown that training in VR may be less successful than traditional learning approaches in real-world settings, and haptic interaction may be the key to improving virtual training. In this study, we integrated pseudo-haptic feedback from motion delay with vibrotactile stimulation to enhance the sense of presence, enjoyment, and the perception of physical properties in VR, which may be crucial for achieving faithful simulations. The impact of combined haptic support was assessed in a complex industrial training procedure completing a variety of tasks such as component assembly and cleaning. The results indicate that vibrotactile cues are beneficial for presence and enjoyment, whereas pseudo-haptic illusions effectively enable kinesthetic sensations. Furthermore, multimodal haptic feedback that mixed the two yielded the most advantageous outcomes. Our findings highlight the potential of the pseudo-haptic and vibrotactile fusion in industrial training scenarios, presenting practical implications of the state-of-the-art haptic technologies for virtual learning.
References
[1]
Haslinda Abdullah. 2009. Major challenges to the effective management of human resource training and development activities.Journal of International Social Research 2, 8 (2009).
[2]
Ahmad Abiri, Jake Pensa, Anna Tao, Ji Ma, Yen-Yi Juo, Syed J Askari, James Bisley, Jacob Rosen, Erik P Dutson, and Warren S Grundfest. 2019. Multi-modal haptic feedback for grip force reduction in robotic surgery. Scientific reports 9, 1 (2019), 5016.
[3]
Shuhei Asano, Shogo Okamoto, and Yoji Yamada. 2014. Vibrotactile stimulation to increase and decrease texture roughness. IEEE Transactions on Human-Machine Systems 45, 3 (2014), 393–398.
[4]
Yuki Ban and Yusuke Ujitoko. 2021. Hit-Stop in VR: Combination of Pseudo-haptics and Vibration Enhances Impact Sensation. In 2021 IEEE World Haptics Conference (WHC). IEEE, 991–996.
[5]
Fahmi Bellalouna. 2020. Industrial use cases for augmented reality application. In 2020 11th IEEE International Conference on Cognitive Infocommunications (CogInfoCom). IEEE, 000011–000018.
[6]
Cyril Bossard, Gilles Kermarrec, Cédric Buche, and Jacques Tisseau. 2008. Transfer of learning in virtual environments: a new challenge?Virtual Reality 12, 3 (2008), 151–161.
[7]
Carola Botto, Alberto Cannavò, Daniele Cappuccio, Giada Morat, Amir Nematollahi Sarvestani, Paolo Ricci, Valentina Demarchi, and Alessandra Saturnino. 2020. Augmented reality for the manufacturing industry: the case of an assembly assistant. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 299–304.
[8]
Elodie Bouzbib, Claudio Pacchierotti, and Anatole Lécuyer. 2023. When Tangibles Become Deformable: Studying Pseudo-Stiffness Perceptual Thresholds in a VR Grasping Task. IEEE Transactions on Visualization and Computer Graphics 29, 5 (2023), 2743–2752.
[9]
Evren Bozgeyikli, Andrew Raij, Srinivas Katkoori, and Rajiv Dubey. 2016. Point & teleport locomotion technique for virtual reality. In Proceedings of the 2016 annual symposium on computer-human interaction in play. 205–216.
[10]
P Buń, D Grajewski, and F Górski. 2021. Using augmented reality devices for remote support in manufacturing: A case study and analysis.Advances in Production Engineering & Management 16, 4 (2021).
[11]
Marcus A Butavicius, Armando Vozzo, Helen Braithwaite, and George Galanis. 2012. Evaluation of a virtual reality parachute training simulator: Assessing learning in an off-course augmented feedback training schedule. The International Journal of Aviation Psychology 22, 3 (2012), 282–298.
[12]
Patrick Carlson, Anicia Peters, Stephen B Gilbert, Judy M Vance, and Andy Luse. 2015. Virtual training: Learning transfer of assembly tasks. IEEE transactions on visualization and computer graphics 21, 6 (2015), 770–782.
[13]
Amine Chellali, Cédric Dumas, and Isabelle Milleville-Pennel. 2011. Influences of haptic communication on a shared manual task. Interacting with Computers 23, 4 (2011), 317–328.
[14]
Doo-Hun Choi and Ghee-Young Noh. 2021. The impact of presence on learning transfer intention in virtual reality simulation game. SAGE Open 11, 3 (2021), 21582440211032178.
[15]
Inrak Choi, Heather Culbertson, Mark R Miller, Alex Olwal, and Sean Follmer. 2017. Grabity: A wearable haptic interface for simulating weight and grasping in virtual reality. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 119–130.
[16]
Inrak Choi, Yiwei Zhao, Eric J Gonzalez, and Sean Follmer. 2020. Augmenting perceived softness of haptic proxy objects through transient vibration and visuo-haptic illusion in virtual reality. IEEE Transactions on Visualization and Computer Graphics 27, 12 (2020), 4387–4400.
[17]
Seungmoon Choi and Katherine J Kuchenbecker. 2012. Vibrotactile display: Perception, technology, and applications. Proc. IEEE 101, 9 (2012), 2093–2104.
[18]
Natalia Cooper, Ferdinando Millela, Iain Cant, Mark D White, and Georg Meyer. 2021. Transfer of training—Virtual reality training with augmented multisensory cues improves user experience during training and task performance in the real world. PloS one 16, 3 (2021), e0248225.
[19]
Paweł Dobrowolski, Maciek Skorko, Grzegorz Pochwatko, Monika Myśliwiec, and Andrzej Grabowski. 2021. Immersive virtual reality and complex skill learning: transfer effects after training in younger and older adults. Frontiers in Virtual Reality 1 (2021), 604008.
[20]
Mohsen Ebrahimzadeh and Sepideh Alavi. 2017. Digital Video Games: E-learning Enjoyment as a Predictor of Vocabulary Learning.Electronic Journal of Foreign Language Teaching 14, 2 (2017).
[21]
Noureddine Elmqaddem. 2019. Augmented reality and virtual reality in education. Myth or reality?International journal of emerging technologies in learning 14, 3 (2019).
[22]
Martin Feick, André Zenner, Oscar Ariza, Anthony Tang, Cihan Biyikli, and Antonio Krüger. 2023. Turn-It-Up: Rendering Resistance for Knobs in Virtual Reality through Undetectable Pseudo-Haptics. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–10.
[23]
David Feygin, Madeleine Keehner, and R Tendick. 2002. Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill. In Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002. IEEE, 40–47.
[24]
Christos Giachritsis, Rachel Wright, and Alan Wing. 2010. The contribution of proprioceptive and cutaneous cues in weight perception: Early evidence for maximum-likelihood integration. In Haptics: Generating and Perceiving Tangible Sensations: International Conference, EuroHaptics 2010, Amsterdam, July 8-10, 2010. Proceedings, Part I. Springer, 11–16.
[25]
Janet K Gibbs, Marco Gillies, and Xueni Pan. 2022. A comparison of the effects of haptic and visual feedback on presence in virtual reality. International Journal of Human-Computer Studies 157 (2022), 102717.
[26]
Simone Grassini, Karin Laumann, and Martin Rasmussen Skogstad. 2020. The use of virtual reality alone does not promote training performance (but sense of presence does). Frontiers in psychology 11 (2020), 553693.
[27]
Joanna Hernik and Elżbieta Jaworska. 2018. The effect of enjoyment on learning. In INTED2018 proceedings. IATED, 508–514.
[28]
Yutaro Hirao, Tomohiro Amemiya, Takuji Narumi, Ferran Argelaguet, and Anatole Lécuyer. 2023. Leveraging Tendon Vibration to Enhance Pseudo-Haptic Perceptions in VR. IEEE Transactions on Visualization and Computer Graphics (2023).
[29]
Yutaro Hirao, Tuukka M Takala, and Anatole Lécuyer. 2020. Comparing motion-based versus controller-based pseudo-haptic weight sensations in VR. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 305–310.
[30]
Christian Hirt, Valentin Holzwarth, Joy Gisler, Johannes Schneider, and Andreas Kunz. 2019. Virtual learning environment for an industrial assembly task. In 2019 IEEE 9th International Conference on Consumer Electronics (ICCE-Berlin). IEEE, 337–342.
[31]
Thuong Hoang, Stefan Greuter, and Simeon Taylor. 2022. An evaluation of virtual reality maintenance training for industrial hydraulic machines. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 573–581.
[32]
Inwook Hwang, Hyungki Son, and Jin Ryong Kim. 2017. AirPiano: Enhancing music playing experience in virtual reality with mid-air haptic feedback. In 2017 IEEE world haptics conference (WHC). IEEE, 213–218.
[33]
Ali Israr and Freddy Abnousi. 2018. Towards pleasant touch: Vibrotactile grids for social touch interactions. In Extended abstracts of the 2018 CHI conference on human factors in computing systems. 1–6.
[34]
David Antonio Gomez Jauregui, Ferran Argelaguet, Anne-Hélène Olivier, Maud Marchal, Franck Multon, and Anatole Lecuyer. 2014. Toward" pseudo-haptic avatars": Modifying the visual animation of self-avatar can simulate the perception of weight lifting. IEEE transactions on visualization and computer graphics 20, 4 (2014), 654–661.
[35]
Ömür Kaya Kalkan, Şener Karabulut, and Gürhan Höke. 2021. Effect of virtual reality-based training on complex industrial assembly task performance. Arabian Journal for Science and Engineering 46, 12 (2021), 12697–12708.
[36]
Oliver Beren Kaul, Kevin Meier, and Michael Rohs. 2017. Increasing presence in virtual reality with a vibrotactile grid around the head. In Human-Computer Interaction–INTERACT 2017: 16th IFIP TC 13 International Conference, Mumbai, India, September 25-29, 2017, Proceedings, Part IV 16. Springer, 289–298.
[37]
Hooman Khosravi, Katayoon Etemad, and Faramarz F Samavati. 2022. Mass simulation in VR using vibrotactile feedback and a co-located physically-based virtual hand. Computers & Graphics 102 (2022), 120–132.
[38]
Jinwook Kim, Seonghyeon Kim, and Jeongmi Lee. 2022. The Effect of Multisensory Pseudo-Haptic Feedback on Perception of Virtual Weight. IEEE Access (2022).
[39]
Kishor Lakshminarayanan, Abigail W Lauer, Viswanathan Ramakrishnan, John G Webster, and Na Jin Seo. 2015. Application of vibration to wrist and hand skin affects fingertip tactile sensation. Physiological reports 3, 7 (2015), e12465.
[40]
Anatole Lécuyer. 2009. Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback. Presence: Teleoperators and Virtual Environments 18, 1 (2009), 39–53.
[41]
Min Li, Wenliang Guo, Bo He, Guanghua Xu, Hua Yuan, Jun Xie, and Tangfei Tao. 2019. Motion assistance and resistance using pseudo-haptic feedback for upper-limb rehabilitation. In 2019 16th International Conference on Ubiquitous Robots (UR). IEEE, 328–333.
[42]
JJ-W Lin, Henry Been-Lirn Duh, Donald E Parker, Habib Abi-Rached, and Thomas A Furness. 2002. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In Proceedings ieee virtual reality 2002. IEEE, 164–171.
[43]
Dorothy Lucardie. 2014. The impact of fun and enjoyment on adult’s learning. Procedia-Social and behavioral sciences 142 (2014), 439–446.
[44]
Andualem Tadesse Maereg, Atulya Nagar, David Reid, and Emanuele L Secco. 2017. Wearable vibrotactile haptic device for stiffness discrimination during virtual interactions. Frontiers in Robotics and AI 4 (2017), 42.
[45]
Guido Makransky and Richard E Mayer. 2022. Benefits of taking a virtual field trip in immersive virtual reality: Evidence for the immersion principle in multimedia learning. Educational Psychology Review 34, 3 (2022), 1771–1798.
[46]
Andrii Matviienko, Florian Müller, Martin Schmitz, Marco Fendrich, and Max Mühlhäuser. 2022. Skyport: Investigating 3d teleportation methods in virtual environments. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–11.
[47]
Fabio Marco Monetti, Andrea de Giorgio, Haisheng Yu, Antonio Maffei, and Mario Romero. 2022. An experimental study of the impact of virtual reality training on manufacturing operators on industrial robotic tasks. Procedia CIRP 106 (2022), 33–38.
[48]
Hye Sung Moon, Grady Orr, and Myounghoon Jeon. 2023. Hand tracking with vibrotactile feedback enhanced presence, engagement, usability, and performance in a virtual reality rhythm game. International Journal of Human–Computer Interaction 39, 14 (2023), 2840–2851.
[49]
Ian Oakley, Yeongmi Kim, Junhun Lee, and Jeha Ryu. 2006. Determining the feasibility of forearm mounted vibrotactile displays. In 2006 14th symposium on haptic interfaces for virtual environment and teleoperator systems. IEEE, 27–34.
[50]
Dmitry Pavlov, Igor Sosnovsky, Vyacheslav Dimitrov, Vasilii Melentyev, and Dmitry Korzun. 2020. Case study of using virtual and augmented reality in industrial system monitoring. In 2020 26th Conference of Open Innovations Association (FRUCT). IEEE, 367–375.
[51]
Evan Pezent, Ali Israr, Majed Samad, Shea Robinson, Priyanshu Agarwal, Hrvoje Benko, and Nick Colonnese. 2019. Tasbi: Multisensory squeeze and vibrotactile wrist haptics for augmented and virtual reality. In 2019 IEEE World Haptics Conference (WHC). IEEE, 1–6.
[52]
Evan Pezent, Alix Macklin, Jeffrey M Yau, Nicholas Colonnese, and Marcia K O’Malley. 2023. Multisensory Pseudo-Haptics for Rendering Manual Interactions with Virtual Objects. Advanced Intelligent Systems (2023), 2200303.
[53]
Poorang Piroozfar, Amer Essa, and Eric RP Farr. 2017. The application of Augmented Reality and Virtual Reality in the construction industry using wearable devices. In The 9th International Conference on Construction in the 21st Century (CITC-9). 0–0.
[54]
Andreas Pusch and Anatole Lécuyer. 2011. Pseudo-haptics: from the theoretical foundations to practical system design guidelines. In Proceedings of the 13th international conference on multimodal interfaces. 57–64.
[55]
Jussi Rantala, Päivi Majaranta, Jari Kangas, Poika Isokoski, Deepak Akkil, Oleg Špakov, and Roope Raisamo. 2020. Gaze interaction with vibrotactile feedback: Review and design guidelines. Human–Computer Interaction 35, 1 (2020), 1–39.
[56]
Regania Pasca Rassy, Nur Azlina Mohamed Mokmin, and Nur Effatul Fairuz Zainal Apandi. 2023. Examining Learning Enjoyment, Presence, and Achievement Using a Virtual Reality Pathology Laboratory by Applying the CAMIL Model in Health Education. (2023).
[57]
Michael Rietzler, Florian Geiselhart, Julian Frommel, and Enrico Rukzio. 2018. Conveying the perception of kinesthetic feedback in virtual reality using state-of-the-art hardware. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.
[58]
Michael Rietzler, Florian Geiselhart, Jan Gugenheimer, and Enrico Rukzio. 2018. Breaking the tracking: Enabling weight perception using perceivable tracking offsets. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.
[59]
Michael Rietzler, Gabriel Haas, Thomas Dreja, Florian Geiselhart, and Enrico Rukzio. 2019. Virtual muscle force: Communicating kinesthetic forces through pseudo-haptic feedback and muscle input. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 913–922.
[60]
Majed Samad, Elia Gatti, Anne Hermes, Hrvoje Benko, and Cesare Parise. 2019. Pseudo-haptic weight: Changing the perceived weight of virtual objects by manipulating control-display ratio. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
[61]
Ahm Shamsuzzoha, Rayko Toshev, Viet Vu Tuan, Timo Kankaanpaa, and Petri Helo. 2021. Digital factory–virtual reality environments for industrial training and maintenance. Interactive learning environments 29, 8 (2021), 1339–1362.
[62]
Jeffery W Smith and John L Salmon. 2017. Development and analysis of virtual reality technician-training platform and methods. In Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Vol. 6. 1–12.
[63]
Marco Speicher, Jan Ehrlich, Vito Gentile, Donald Degraen, Salvatore Sorce, and Antonio Krüger. 2019. Pseudo-haptic controls for mid-air finger-based menu interaction. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.
[64]
Carolin Stellmacher, Feri Irsanto Pujianto, Tanja Kojic, Jan-Niklas Voigt-Antons, and Johannes Schöning. 2024. Experiencing Dynamic Weight Changes in Virtual Reality Through Pseudo-Haptics and Vibrotactile Feedback. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–13.
[65]
Jonathan A Stevens and J Peter Kincaid. 2015. The relationship between presence and performance in virtual simulation training. Open Journal of Modelling and Simulation 3, 2 (2015), 41–48.
[66]
P Ström, Leif Hedman, L Särnå, A Kjellin, T Wredmark, and L Felländer-Tsai. 2006. Early exposure to haptic feedback enhances performance in surgical simulator training: a prospective randomized crossover study in surgical residents. Surgical endoscopy and other interventional techniques 20 (2006), 1383–1388.
[67]
Mihail Terenti, Cristian Pamparău, and Radu-Daniel Vatavu. 2024. The user experience of distal arm-level vibrotactile feedback for interactions with virtual versus physical displays. Virtual Reality 28, 2 (2024), 84.
[68]
Yusuke Ujitoko and Yuki Ban. 2021. Survey of pseudo-haptics: Haptic feedback design and application proposals. IEEE Transactions on Haptics 14, 4 (2021), 699–711.
[69]
Yusuke Ujitoko, Yuki Ban, and Koichi Hirota. 2019. Modulating fine roughness perception of vibrotactile textured surface using pseudo-haptic effect. IEEE Transactions on Visualization and Computer Graphics 25, 5 (2019), 1981–1990.
[70]
Keigo Ushiyama, Akifumi Takahashi, and Hiroyuki Kajimoto. 2022. Increasing Perceived Weight and Resistance by Applying Vibration to Tendons During Active Arm Movements. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 93–100.
[71]
Dangxiao Wang, Kouhei Ohnishi, and Weiliang Xu. 2019. Multimodal haptic display for virtual reality: A survey. IEEE Transactions on Industrial Electronics 67, 1 (2019), 610–623.
[72]
Yannick Weiss, Steeven Villa, Albrecht Schmidt, Sven Mayer, and Florian Müller. 2023. Using Pseudo-Stiffness to Enrich the Haptic Experience in Virtual Reality. (2023).
[73]
Frederik Winther, Linoj Ravindran, Kasper Paabøl Svendsen, and Tiare Feuchtner. 2020. Design and evaluation of a vr training simulation for pump maintenance based on a use case at grundfos. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 738–746.
[74]
Bob G Witmer and Michael J Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence 7, 3 (1998), 225–240.
[75]
Dennis Wolf, Michael Rietzler, Leo Hnatek, and Enrico Rukzio. 2019. Face/on: Multi-modal haptic feedback for head-mounted displays in virtual reality. IEEE transactions on visualization and computer graphics 25, 11 (2019), 3169–3177.
[76]
Peng Yang and Zhi Liu. 2022. The influence of immersive virtual reality (IVR) on skill transfer of learners: The moderating effects of learning engagement. International Journal of Emerging Technologies in Learning (iJET) 17, 10 (2022), 62–73.
[77]
Neng-Hao Yu, Shih-Yu Ma, Cong-Min Lin, Chi-Aan Fan, Luca E Taglialatela, Tsai-Yuan Huang, Carolyn Yu, Yun-Ting Cheng, Ya-Chi Liao, and Mike Y Chen. 2023. DrivingVibe: Enhancing VR Driving Experience using Inertia-based Vibrotactile Feedback around the Head. Proceedings of the ACM on Human-Computer Interaction 7, MHCI (2023), 1–22.
Index Terms
- Enriching Industrial Training Experience in Virtual Reality with Pseudo-Haptics and Vibrotactile Stimulation
Recommendations
Experiencing Dynamic Weight Changes in Virtual Reality Through Pseudo-Haptics and Vibrotactile Feedback
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing SystemsVirtual reality (VR) objects react dynamically to users’ touch interactions in real-time. However, experiencing changes in weight through the haptic sense remains challenging with consumer VR controllers due to their limited vibrotactile feedback. While ...
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Haptics-based virtual reality periodontal training simulator
This paper focuses upon the research and development of a prototype dental simulator for training of periodontal procedures. By the use of virtual reality and haptics technology, the periodontal simulator allows trainees to learn performing diagnosis ...
Comments
Information & Contributors
Information
Published In
October 2024
633 pages
ISBN:9798400705359
DOI:10.1145/3641825
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
VRST '24: 30th ACM Symposium on Virtual Reality Software and Technology
October 9 - 11, 2024
Trier, Germany
Acceptance Rates
Overall Acceptance Rate 66 of 254 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 186Total Downloads
- Downloads (Last 12 months)186
- Downloads (Last 6 weeks)89
Reflects downloads up to 23 Dec 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in