research-article

Design and Evaluation of Electrotactile Rendering Effects for Finger-Based Interactions in Virtual Reality

Authors:

Sebastian Vizcay,

Panagiotis Kourtesis,

Ferran Argelaguet,

Claudio Pacchierotti,

Maud MarchalAuthors Info & Claims

VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and Technology

Article No.: 35, Pages 1 - 11

https://doi.org/10.1145/3562939.3565634

Published: 29 November 2022 Publication History

Abstract

The use of electrotactile feedback in Virtual Reality (VR) has shown promising results for providing tactile information and sensations. While progress has been made to provide custom electrotactile feedback for specific interaction tasks, it remains unclear which modulations and rendering algorithms are preferred in rich interaction scenarios. In this paper, we propose a unified tactile rendering architecture and explore the most promising modulations to render finger interactions in VR. Based on a literature review, we designed six electrotactile stimulation patterns/effects (EFXs) striving to render different tactile sensations. In a user study (N=18), we assessed the six EFXs in three diverse finger interactions: 1) tapping on a virtual object; 2) pressing down a virtual button; 3) sliding along a virtual surface. Results showed that the preference for certain EFXs depends on the task at hand. No significant preference was detected for tapping (short and quick contact); EFXs that render dynamic intensities or dynamic spatio-temporal patterns were preferred for pressing (continuous dynamic force); EFXs that render moving sensations were preferred for sliding (surface exploration). The results showed the importance of the coherence between the modulation an the interaction being performed and the study proved the versatility of electrotactile feedback and its efficiency in rendering different haptic information and sensations.

Supplementary Material

M4V File (VRST_2022_EFXs_credits.m4v)

Presentation video

Download
30.63 MB

References

[1]

G. L. Aiello. 1998. Multidimensional electrocutaneous stimulation. IEEE Trans. Rehabilitation Eng. 6, 1 (1998), 95–101.

[2]

Mirjam Augstein and Thomas Neumayr. 2019. A Human-Centered Taxonomy of Interaction Modalities and Devices. Interacting with Computers 31, 1 (2019), 27–58.

[3]

Lucinda L Baker, Cynthia Wederich, Donald R McNeal, Craig J Newsam, and Robert L Waters. 2000. Neuro muscular electrical stimulation: a practical guide.

[4]

Adam O Bebko and Nikolaus F Troje. 2020. bmlTUX: Design and control of experiments in virtual reality and beyond. i-Perception 11, 4 (2020).

[5]

Guohong Chai, Josselin Briand, Shiyong Su, Xinjun Sheng, and Xiangyang Zhu. 2019. Electrotactile Feedback with Spatial and Mixed Coding for Object Identification and Closed-loop Control of Grasping Force in Myoelectric Prostheses. In Proc. Annual Intl. Conf. IEEE Engineering in Medicine and Biology Society, EMBS. 1805–1808.

[6]

Shaona Cheng, Andong Yi, U. Xuan Tan, and Dingguo Zhang. 2019. Closed-Loop System for Myoelectric Hand Control Based on Electrotactile Stimulation. In Proc. 3rd Intl. Conf. Advanced Robotics and Mechatronics. 486–490.

[7]

Kyunghwan Choi, Pyungkang Kim, Kyung Soo Kim, and Soohyun Kim. 2017. Mixed-modality stimulation to evoke two modalities simultaneously in one channel for electrocutaneous sensory feedback. IEEE Trans. Neural Systems and Rehab. Eng. 25, 12 (2017), 2258–2269.

[8]

Seungmoon Choi and Katherine J Kuchenbecker. 2012. Vibrotactile display: Perception, technology, and applications. Proc. IEEE 101, 9 (2012), 2093–2104.

[9]

D D Damian, A H Arita, H Martinez, and R Pfeifer. 2012. Slip speed feedback for grip force control. IEEE Trans. Biomedical Engineering 59, 8 (2012), 2200–2210.

[10]

Xavier De Tinguy, Claudio Pacchierotti, Maud Marchal, and Anatole Lécuyer. 2018. Enhancing the stiffness perception of tangible objects in mixed reality using wearable haptics. In Proc. IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 81–90.

[11]

Marta Franceschi, Lucia Seminara, Luigi Pinna, Maurizio Valle, Ali Ibrahim, and Strahinja Dosen. 2016. Towards the integration of e-skin into prosthetic devices. In 12th Conference on Ph.D. Research in Microelectronics and Electronics, PRIME. 1–4.

[12]

Bo Geng and Winnie Jensen. 2014. Human ability in identification of location and pulse number for electrocutaneous stimulation applied on the forearm. Journal of NeuroEngineering and Rehabilitation 11, 1(2014), 97.

[13]

Kai He, Peng Yu, Mi Li, Yang Yang, and Lianqing Liu. 2016. The quantitative evaluation of electrotactile stimulation mode. In Proc. IEEE Intl. Conf. Real-time Computing and Robotics (RCAR). 346–351.

[14]

Johannes Hummel, Janki Dodiya, Laura Eckardt, Robin Wolff, Andreas Gerndf, and Torsten W. Kuhlen. 2016. A lightweight electrotactile feedback device for grasp improvement in immersive virtual environments. In Proc. IEEE Virtual Reality. 39–48.

[15]

Milica Isaković, Minja Belić, Matija Štrbac, Igor Popović, Strahinja Došen, Dario Farina, and Thierry Keller. 2016. Electrotactile feedback improves performance and facilitates learning in the routine grasping task. European Journal of Translational Myology 26, 3 (2016), 197–202.

[16]

Milica Isaković, Jovana Malešević, Thierry Keller, Miloš Kostić, and Matija Štrbac. 2019. Optimization of semiautomated calibration algorithm of multichannel electrotactile feedback for myoelectric hand prosthesis. Applied bionics and biomechanics(2019).

[17]

Jan Jacobs, Michael Stengel, and Bernd Froehlich. 2012. A generalized god-object method for plausible finger-based interactions in virtual environments. In Proc. IEEE Symposium on 3D User Interfaces (3DUI). 43–51.

[18]

Camille Jeunet, Louis Albert, Ferran Argelaguet, and Anatole Lécuyer. 2018. “Do you feel in control?”: towards novel approaches to characterise, manipulate and measure the sense of agency in virtual environments. IEEE Trans. Vis. Comp. Graphics 24, 4 (2018), 1486–1495.

Digital Library

[19]

Kurt A Kaczmarek, Mitchell E Tyler, and Paul Bach-y Rita. 1994. Electrotactile haptic display on the fingertips: Preliminary results. In Proc. Intl. Conf. IEEE Engineering in Medicine and Biology Society, Vol. 2. 940–941.

[20]

Kurt A. Kaczmarek, Mitchell E. Tyler, Uchechukwu O. Okpara, and Steven J. Haase. 2017. Interaction of Perceived Frequency and Intensity in Fingertip Electrotactile Stimulation: Dissimilarity Ratings and Multidimensional Scaling. IEEE Trans. Neural Systems and Rehabilitation Eng. 25, 11 (2017), 2067–2074.

[21]

Hiroyuki Kajimoto. 2016. Electro-tactile Display: Principle and Hardware. 79–96.

[22]

Hiroyuki Kajimoto, Naoki Kawakami, T Maeda, and S Tachi. 2004. Electro-tactile display with tactile primary color approach. Graduate School of Information and Technology, The University of Tokyo (2004).

[23]

Norihide Kitamura and Norihisa Miki. 2015. Micro-needle-based electro tactile display to present various tactile sensation. In Proc. IEEE Intl. Conf. Micro Electro Mechanical Systems (MEMS), Vol. 2015. 649–650.

[24]

Panagiotis Kourtesis, Ferran Argelaguet, Sebastian Vizcay, Maud Marchal, and Claudio Pacchierotti. 2022. Electrotactile feedback applications for hand and arm interactions: A systematic review, meta-analysis, and future directions. IEEE Trans. Haptics (2022), 1–18.

[25]

Kairu Li, Peter Boyd, Yu Zhou, Zhaojie Ju, and Honghai Liu. 2018. Electrotactile Feedback in a Virtual Hand Rehabilitation Platform: Evaluation and Implementation.

[26]

Zhiming Liu, Yudong Luo, Jose Cordero, Na Zhao, and Yantao Shen. 2016. Finger-eye: A wearable text reading assistive system for the blind and visually impaired. In Proc. IEEE Intl. Conf. Real-Time Computing and Robotics, RCAR. 123–128.

[27]

M. Maisto, C. Pacchierotti, F. Chinello, G. Salvietti, A. De Luca, and D. Prattichizzo. 2017. Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications. IEEE Trans. Haptics 10, 4 (2017), 511–522.

Digital Library

[28]

Leonardo Meli, Claudio Pacchierotti, Gionata Salvietti, Francesco Chinello, Maurizio Maisto, Alessandro De Luca, and Domenico Prattichizzo. 2018. Combining wearable finger haptics and augmented reality: User evaluation using an external camera and the microsoft hololens. IEEE Robotics and Automation Letters 3, 4 (2018), 4297–4304.

[29]

Claudio Pacchierotti, Stephen Sinclair, Massimiliano Solazzi, Antonio Frisoli, Vincent Hayward, and Domenico Prattichizzo. 2017. Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives. IEEE Trans. Haptics 10, 4 (2017), 580–600.

Digital Library

[30]

Daniel S Pamungkas and Wahyu Caesarendra. 2018. Overview Electrotactile Feedback for Enhancing Human Computer Interface. Journal of Physics: Conference Series 1007 (2018), 012001.

[31]

Daniel S. Pamungkas and Koren Ward. 2013. Tele-operation of a robot arm with electro tactile feedback. In Proc. IEEE/ASME Intl. Conf. Advanced Intelligent Mechatronics: Mechatronics for Human Wellbeing, AIM 2013. 704–709.

[32]

Ismo Rakkolainen, Ahmed Farooq, Jari Kangas, Jaakko Hakulinen, Jussi Rantala, Markku Turunen, and Roope Raisamo. 2021. Technologies for Multimodal Interaction in Extended Reality: A Scoping Review. Multimodal Technologies and Interaction 5, 12 (2021).

[33]

M Sagardia, K Hertkorn, T Hulin, S Schätzle, R Wolff, J Hummel, J Dodiya, and A Gerndt. 2015. VR-OOS: The DLR’s virtual reality simulator for telerobotic on-orbit servicing with haptic feedback. In Proc. IEEE Aerospace Conference. 1–17.

[34]

Katsunari Sato, Hiroyuki Shinoda, and Susumu Tachi. 2011. Design and implementation of transmission system of initial haptic impression. In Proc. SICE Annual Conference. 616–621.

[35]

Katsunari Sato and Susumu Tachi. 2010. Design of electrotactile stimulation to represent distribution of force vectors. In Proc. IEEE Haptics Symposium. 121–128.

Digital Library

[36]

Stefano Scheggi, Leonardo Meli, Claudio Pacchierotti, and Domenico Prattichizzo. 2015. Touch the virtual reality: using the leap motion controller for hand tracking and wearable tactile devices for immersive haptic rendering. In Proc. ACM SIGGRAPH Posters.

Digital Library

[37]

Richard Skarbez, Frederick P Brooks, Jr, and Mary C Whitton. 2017. A survey of presence and related concepts. ACM Computing Surveys (CSUR) 50, 6 (2017), 1–39.

Digital Library

[38]

Mel Slater. 2009. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (2009), 3549–3557.

[39]

Giovanni Spagnoletti, Leonardo Meli, Tommaso Lisini Baldi, Guido Gioioso, Claudio Pacchierotti, and Domenico Prattichizzo. 2018. Rendering of pressure and textures using wearable haptics in immersive VR environments. In Proc. IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 691–692.

[40]

Matija Štrbac, Minja Belić, Milica Isaković, Vladimir Kojić, Goran Bijelić, Igor Popović, Milutin Radotić, Strahinja Došen, Marko Marković, Dario Farina, and Thierry Keller. 2016. Integrated and flexible multichannel interface for electrotactile stimulation. Journal of Neural Engineering 13, 4 (2016), 046014.

[41]

Matija Štrbac, Milica Isaković, Minja Belić, Igor Popović, Igor Simanić, Dario Farina, Thierry Keller, and Strahinja Došen. 2017. Short-and long-term learning of feedforward control of a myoelectric prosthesis with sensory feedback by amputees. IEEE Trans. Neural Systems and Rehabilitation Eng. 25, 11 (2017), 2133–2145.

[42]

Daniel Sutopo Pamungkas. 2016. Enhancing human computer interaction with electrotactile feedback. In University of Wollongong Thesis.

[43]

David Swapp, Vijay Pawar, and Céline Loscos. 2006. Interaction with co-located haptic feedback in virtual reality. Virtual Reality 10, 1 (2006), 24–30.

Digital Library

[44]

Sebastian Vizcay, Panagiotis Kourtesis, Ferran Argelaguet, Claudio Pacchierotti, and Maud Marchal. 2021. Electrotactile Feedback For Enhancing Contact Information in Virtual Reality. In Proc. Intl. Conf. Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE). 10.

[45]

Koren Ward and Daniel Pamungkas. 2018. Multi-channel electro-tactile feedback system for a prosthetic hand. Mechatronics and Machine Vision in Practice 3 (2018), 181–193.

[46]

Anusha Withana, Daniel Groeger, and Jürgen Steimle. 2018. Tacttoo: A thin and feel-through tattoo for on-skin tactile output. In UIST - Proc. 31st Annual ACM Symposium on User Interface Software and Technology. 365–378.

Digital Library

[47]

H J B Witteveen, E A Droog, J S Rietman, and P H Veltink. 2012. Vibro- and electrotactile user feedback on hand opening for myoelectric forearm prostheses. IEEE Trans. Biomedical Engineering 59, 8 (2012), 2219–2226.

[48]

Heng Xu, Linjun Bao, Dingguo Zhang, and Xiangyang Zhu. 2014. Identify key grasping-related properties based on cutaneous electrotactile stimulation. In Proc. IEEE 19th International Functional Electrical Stimulation Society Annual Conference.

[49]

Heng Xu, Digguo Zhang, Joel C. Huegel, Wendong Xu, and Xiangyang Zhu. 2016. Effects of Different Tactile Feedback on Myoelectric Closed-Loop Control for Grasping Based on Electrotactile Stimulation. IEEE Trans. Neural Systems and Rehabilitation Eng. 24, 8 (2016), 827–836.

[50]

Vibol Yem and Hiroyuki Kajimoto. 2017. Comparative Evaluation of Tactile Sensation by Electrical and Mechanical Stimulation. IEEE Trans. Haptics 10, 1 (2017), 130–134.

Digital Library

[51]

Vibol Yem, Kevin Vu, Yuki Kon, and Hiroyuki Kajimoto. 2018. Effect of Electrical Stimulation Haptic Feedback on Perceptions of Softness-Hardness and Stickiness while Touching a Virtual Object. In Proc. IEEE Conference on Virtual Reality and 3D User Interfaces. 89–96.

[52]

Shunsuke Yoshimoto, Yoshihiro Kuroda, Masataka Imura, and Osamu Oshiro. 2015. Material roughness modulation via electrotactile augmentation. IEEE Trans. Haptics 8, 2 (2015), 199–208.

Digital Library

[53]

Shunsuke Yoshimoto, Yoshihiro Kuroda, Masataka Imura, Osamu Oshiro, Kazunori Nozaki, Yoshiaki Taga, Hiroyuki Machi, and Hiroo Tamagawa. 2016. Electrotactile Augmentation for Carving Guidance. IEEE Trans. Haptics 9, 1 (2016), 43–53.

Digital Library

[54]

Shunsuke Yoshimoto, Yoshihiro Kuroda, Masataka Imura, Osamu Oshiro, and Kosuke Sato. 2013. Electrically multiplexed tactile interface: Fusion of smart tactile sensor and display. In Proc. World Haptics Conference. 151–156.

[55]

Ziliang Zhou, Yicheng Yang, Jinbiao Liu, Jia Zeng, Xiaoxin Wang, and Honghai Liu. 2022. Electrotactile Perception Properties and Its Applications: A Review. IEEE Trans. Haptics (2022).

Digital Library

Cited By

Teng SGupta ALopes P(2024)Haptic Permeability: Adding Holes to Tactile Devices Improves DexterityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642156(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642156
Teo YSaito TKameoka TMizoguchi IHiroyuki K(2024)Presentation of Slip Sensation Using Suction Pressure and Electrotactile StimulationHaptics: Understanding Touch; Technology and Systems; Applications and Interaction10.1007/978-3-031-70058-3_26(314-320)Online publication date: 30-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-70058-3_26
Ushiyama KLopes P(2023)FeetThrough: Electrotactile Foot Interface that Preserves Real-World SensationsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606808(1-11)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606808
Show More Cited By

Index Terms

Design and Evaluation of Electrotactile Rendering Effects for Finger-Based Interactions in Virtual Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
    2. Interaction paradigms
      1. Virtual reality

Recommendations

Design, evaluation and calibration of wearable electrotactile interfaces for enhancing contact information in virtual reality
Abstract
Electrotactile feedback is a convenient tactile rendering method thanks to reduced form factor and power consumption. Yet, its usage in immersive virtual reality has been scarcely addressed. This paper explores how electrotactile ...
Graphical abstract

Display Omitted
Highlights
- Electrotactile improves contact’s accuracy and precision when touching virtual objects
Virtual reality and the CAVE

One of the main goals of virtual reality is to provide immersive environments that take participants away from the real life into a virtual one. Many investigators have been interested in bringing new technologies, devices, and applications to ...
Experiencing 3D interactions in virtual reality and augmented reality
EUSAI '04: Proceedings of the 2nd European Union symposium on Ambient intelligence

We demonstrate basic 2D and 3D interactions in both a Virtual Reality (VR) system, called the Personal Space Station, and an Augmented Reality (AR) system, called the Visual Interaction Platform. Since both platforms use identical (optical) tracking ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and Technology

November 2022

466 pages

ISBN:9781450398893

DOI:10.1145/3562939

Editors:
Takafumi Koike
Hosei University, Japan
,
Naoya Koizumi
The University of Electro-Communications, Japan
,
Gerd Bruder
University of Central Florida, USA
,
Daniel Roth
Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
,
Kazuki Takashima
Tohoku University, Japan
,
Takefumi Hiraki
University of Tsukuba, Japan
,
Yuki Ban
The University of Tokyo, Japan
,
Michal Piovarci
Institute of Science and Technology Austria, Austria

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

H2020 Industrial Leadership

Conference

VRST '22

Sponsor:

VRST '22: 28th ACM Symposium on Virtual Reality Software and Technology

November 29 - December 1, 2022

Tsukuba, Japan

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
428
Total Downloads

Downloads (Last 12 months)166
Downloads (Last 6 weeks)17

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Teng SGupta ALopes P(2024)Haptic Permeability: Adding Holes to Tactile Devices Improves DexterityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642156(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642156
Teo YSaito TKameoka TMizoguchi IHiroyuki K(2024)Presentation of Slip Sensation Using Suction Pressure and Electrotactile StimulationHaptics: Understanding Touch; Technology and Systems; Applications and Interaction10.1007/978-3-031-70058-3_26(314-320)Online publication date: 30-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-70058-3_26
Ushiyama KLopes P(2023)FeetThrough: Electrotactile Foot Interface that Preserves Real-World SensationsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606808(1-11)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606808
Zhao SNi YDong GTian JChen Y(2023) Comparing three XR technologies in reviewing performance‐based building design: A pilot study of façade fenestrations Computer Animation and Virtual Worlds10.1002/cav.213934:6Online publication date: 3-Jan-2023
https://doi.org/10.1002/cav.2139

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten