research-article

Open access

Fiery Hands: Designing Thermal Glove through Thermal and Tactile Integration for Virtual Object Manipulation

Authors:

Yatharth Singhal,

Jin Ryong KimAuthors Info & Claims

UIST '24: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

Article No.: 101, Pages 1 - 15

https://doi.org/10.1145/3654777.3676457

Published: 11 October 2024 Publication History

All formats PDF

Abstract

We present a novel approach to render thermal and tactile feedback to the palm and fingertips through thermal and tactile integration. Our approach minimizes the obstruction of the palm and inner side of the fingers and enables virtual object manipulation while providing localized and global thermal feedback. By leveraging thermal actuators positioned strategically on the outer palm and back of the fingers in interplay with tactile actuators, our approach exploits thermal referral and tactile masking phenomena. Through a series of user studies, we validate the perception of localized thermal sensations across the palm and fingers, showcasing the ability to generate diverse thermal patterns. Furthermore, we demonstrate the efficacy of our approach in VR applications, replicating diverse thermal interactions with virtual objects. This work represents significant progress in thermal interactions within VR, offering enhanced sensory immersion at an optimal energy cost.

Supplemental Material

MP4 File

Video preview of "Fiery Hands: Designing Thermal Glove through Thermal and Tactile Integration for Virtual Object Manipulation".

Download
213.24 MB

MP4 File

Video previews of Fiery Hands: Designing Thermal Glove through Thermal and Tactile Integration for Virtual Object Manipulation

Download
223.32 MB

References

[1]

Rochelle Ackerley and Roger Holmes Watkins. 2018. Microneurography as a tool to study the function of individual C-fiber afferents in humans: responses from nociceptors, thermoreceptors, and mechanoreceptors. Journal of Neurophysiology 120, 6 (2018), 2834–2846. https://doi.org/10.1152/jn.00109.2018 30256737.

[2]

David S Alles. 1970. Information transmission by phantom sensations. IEEE transactions on man-machine systems 11, 1 (1970), 85–91.

[3]

Edward A. Arens and Hui Zhang. 2006. The skin’s role in human thermoregulation and comfort. Center for the Built Environment (2006), 560–602. https://escholarship.org/uc/item/3f4599hx

[4]

B Averbeck, F Rucker, Rüdiger P Laubender, and RW Carr. 2013. Thermal grill-evoked sensations of heat correlate with cold pain threshold and are enhanced by menthol and cinnamaldehyde. European journal of pain 17, 5 (2013), 724–734. https://doi.org/10.1002/j.1532-2149.2012.00239.x

[5]

Beate Averbeck, Lena Seitz, Florian P Kolb, and Dieter F Kutz. 2017. Sex differences in thermal detection and thermal pain threshold and the thermal grill illusion: a psychophysical study in young volunteers. Biology of sex differences 8 (2017), 1–13. https://doi.org/10.1186/s13293-017-0147-5

[6]

Mayienne Bakkers, Catharina G Faber, Martine JH Peters, Jos PH Reulen, Hessel Franssen, Tanya Z Fischer, and Ingemar SJ Merkies. 2013. Temperature threshold testing: a systematic review. Journal of the Peripheral Nervous System 18, 1 (2013), 7–18. https://doi.org/10.1111/jns5.12001

[7]

Jas Brooks, Steven Nagels, and Pedro Lopes. 2020. Trigeminal-based Temperature Illusions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (, Honolulu, HI, USA, ) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376806

Digital Library

[8]

Manuel Caeiro-Rodríguez, Iván Otero-González, Fernando A. Mikic-Fonte, and Martín Llamas-Nistal. 2021. A Systematic Review of Commercial Smart Gloves: Current Status and Applications. Sensors 21, 8 (2021). https://doi.org/10.3390/s21082667

[9]

Shaoyu Cai, Pingchuan Ke, Takuji Narumi, and Kening Zhu. 2020. ThermAirGlove: A Pneumatic Glove for Thermal Perception and Material Identification in Virtual Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 248–257. https://doi.org/10.1109/VR46266.2020.00044

[10]

Antonio Cataldo, Elisa Raffaella Ferrè, Giuseppe Di Pellegrino, and Patrick Haggard. 2016. Thermal referral: evidence for a thermoceptive uniformity illusion without touch. Scientific reports 6, 1 (2016), 1–10. https://doi.org/10.1038/srep35286

[11]

Zikun Chen, Wei Peng, Roshan Peiris, and Kouta Minamizawa. 2017. ThermoReality: thermally enriched head mounted displays for virtual reality. In ACM SIGGRAPH 2017 Posters (Los Angeles, California) (SIGGRAPH ’17). Association for Computing Machinery, New York, NY, USA, Article 32, 2 pages. https://doi.org/10.1145/3102163.3102222

Digital Library

[12]

Ian Darian-Smith and Kenneth O Johnson. 1977. Thermal sensibility and thermoreceptors. Journal of Investigative Dermatology 69, 1 (1977), 146–153. https://doi.org/10.1111/1523-1747.ep12497936

[13]

Peter James Dyck, I Zimmerman, DA Gillen, D Johnson, JL Karnes, and PC O’Brien. 1993. Cool, warm, and heat-pain detection thresholds: testing methods and inferences about anatomic distribution of receptors. Neurology 43, 8 (1993), 1500–1500. https://doi.org/10.1212/WNL.43.8.1500

[14]

Abdallah El Ali, Xingyu Yang, Swamy Ananthanarayan, Thomas Röggla, Jack Jansen, Jess Hartcher-O’Brien, Kaspar Jansen, and Pablo Cesar. 2020. ThermalWear: Exploring Wearable On-Chest Thermal Displays to Augment Voice Messages with Affect. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376682

Digital Library

[15]

Barry G Green. 1978. Referred thermal sensations: warmth versus cold.Sensory processes 2, 3 (1978), 220–230.

[16]

Sebastian Günther, Florian Müller, Dominik Schön, Omar Elmoghazy, Max Mühlhäuser, and Martin Schmitz. 2020. Therminator: Understanding the Interdependency of Visual and On-Body Thermal Feedback in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376195

Digital Library

[17]

Teng Han, Sirui Wang, Sijia Wang, Xiangmin Fan, Jie Liu, Feng Tian, and Mingming Fan. 2020. Mouillé: Exploring wetness illusion on fingertips to enhance immersive experience in vr. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–10.

Digital Library

[18]

Vincent Hayward. 2018. A brief overview of the human somatosensory system. Musical haptics (2018), 29–48. https://doi.org/10.1007/978-3-319-58316-7_3

[19]

Hsin-Ni Ho, Junji Watanabe, Hideyuki Ando, and Makio Kashino. 2010. Somatotopic or spatiotopic? Frame of reference for localizing thermal sensations under thermo-tactile interactions. Attention, Perception, & Psychophysics 72 (2010), 1666–1675. https://doi.org/10.3758/APP.72.6.1666

[20]

Hsin-Ni Ho, Junji Watanabe, Hideyuki Ando, and Makio Kashino. 2011. Mechanisms underlying referral of thermal sensations to sites of tactile stimulation. Journal of Neuroscience 31, 1 (2011), 208–213. https://doi.org/10.1523/JNEUROSCI.2640-10.2011

[21]

Lynette A Jones and Hsin-Ni Ho. 2008. Warm or cool, large or small? The challenge of thermal displays. IEEE Transactions on Haptics 1, 1 (2008), 53–70. https://doi.org/10.1109/TOH.2008.2

Digital Library

[22]

Hojoong Kim, Young-Tae Kwon, Hyo-Ryoung Lim, Jong-Hoon Kim, Yun-Soung Kim, and Woon-Hong Yeo. 2021. Recent Advances in Wearable Sensors and Integrated Functional Devices for Virtual and Augmented Reality Applications. Advanced Functional Materials 31, 39 (2021), 2005692. https://doi.org/10.1002/adfm.202005692 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/adfm.202005692

[23]

Seung-Won Kim, Sung Hee Kim, Choong Sun Kim, Kyoungsoo Yi, Jun-Sik Kim, Byung Jin Cho, and Youngsu Cha. 2020. Thermal display glove for interacting with virtual reality. Scientific reports 10, 1 (2020), 11403. https://doi.org/10.1038/s41598-020-68362-y

[24]

Susan J Lederman and Lynette A Jones. 2011. Tactile and haptic illusions. IEEE Transactions on Haptics 4, 4 (2011), 273–294. https://doi.org/10.1109/TOH.2011.2

Digital Library

[25]

Yuhu Liu, Satoshi Nishikawa, Young ah Seong, Ryuma Niiyama, and Yasuo Kuniyoshi. 2021. ThermoCaress: A Wearable Haptic Device with Illusory Moving Thermal Stimulation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (, Yokohama, Japan,) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 214, 12 pages. https://doi.org/10.1145/3411764.3445777

Digital Library

[26]

Jasmine Lu, Ziwei Liu, Jas Brooks, and Pedro Lopes. 2021. Chemical Haptics: Rendering Haptic Sensations via Topical Stimulants. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21). Association for Computing Machinery, New York, NY, USA, 239–257. https://doi.org/10.1145/3472749.3474747

Digital Library

[27]

David D. McKemy. 2012. The molecular and cellular basis of cold sensation. ACS Chem Neurosci 4 (2012), 238–247. https://doi.org/10.1021/cn300193h

[28]

Chantal Morin and MC Bushnell. 1998. Temporal and qualitative properties of cold pain and heat pain: a psychophysical study. Pain 74, 1 (1998), 67–73. https://doi.org/10.1016/S0304-3959(97)00152-8

[29]

Roshan Lalintha Peiris, Liwei Chan, and Kouta Minamizawa. 2018. LiquidReality: Wetness Sensations on the Face for Virtual Reality. In Haptics: Science, Technology, and Applications, Domenico Prattichizzo, Hiroyuki Shinoda, Hong Z. Tan, Emanuele Ruffaldi, and Antonio Frisoli (Eds.). Springer International Publishing, Cham, 366–378.

[30]

Roshan Lalitha Peiris, Yuan-Ling Feng, Liwei Chan, and Kouta Minamizawa. 2019. ThermalBracelet: Exploring Thermal Haptic Feedback Around the Wrist. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300400

Digital Library

[31]

Roshan Lalintha Peiris, Wei Peng, Zikun Chen, Liwei Chan, and Kouta Minamizawa. 2017. ThermoVR: Exploring Integrated Thermal Haptic Feedback with Head Mounted Displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5452–5456. https://doi.org/10.1145/3025453.3025824

Digital Library

[32]

Nimesha Ranasinghe, Pravar Jain, Shienny Karwita, David Tolley, and Ellen Yi-Luen Do. 2017. Ambiotherm: Enhancing Sense of Presence in Virtual Reality by Simulating Real-World Environmental Conditions. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 1731–1742. https://doi.org/10.1145/3025453.3025723

Digital Library

[33]

Nimesha Ranasinghe, Pravar Jain, Nguyen Thi Ngoc Tram, Koon Chuan Raymond Koh, David Tolley, Shienny Karwita, Lin Lien-Ya, Yan Liangkun, Kala Shamaiah, Chow Eason Wai Tung, Ching Chiuan Yen, and Ellen Yi-Luen Do. 2018. Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (, Montreal QC, Canada, ) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174151

Digital Library

[34]

Hannes P Saal, Benoit P Delhaye, Brandon C Rayhaun, and Sliman J Bensmaia. 2017. Simulating tactile signals from the whole hand with millisecond precision. Proceedings of the National Academy of Sciences 114, 28 (2017), E5693–E5702. https://doi.org/10.1073/pnas.1704856114

[35]

Kenichiro Shirota, Makoto Uju, Roshan Peiris, and Kouta Minamizawa. 2019. Liquid-VR - Wetness Sensations for Immersive Virtual Reality Experiences. In Haptic Interaction, Hiroyuki Kajimoto, Dongjun Lee, Sang-Youn Kim, Masashi Konyo, and Ki-Uk Kyung (Eds.). Springer Singapore, Singapore, 252–255.

[36]

Yatharth Singhal, Haokun Wang, Hyunjae Gil, and Jin Ryong Kim. 2021. Mid-Air Thermo-Tactile Feedback using Ultrasound Haptic Display. In Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology (Osaka, Japan) (VRST ’21). Association for Computing Machinery, New York, NY, USA, Article 28, 11 pages. https://doi.org/10.1145/3489849.3489889

Digital Library

[37]

Yatharth Singhal, Haokun Wang, and Jin Ryong Kim. 2023. FIRE: Mid-Air Thermo-Tactile Display. In SIGGRAPH Asia 2023 Emerging Technologies (Sydney, NSW, Australia) (SA ’23). Association for Computing Machinery, New York, NY, USA, Article 8, 2 pages. https://doi.org/10.1145/3610541.3614584

Digital Library

[38]

Hyungki Son, Haokun Wang, Yatharth Singhal, and Jin Ryong Kim. 2023. Upper Body Thermal Referral and Tactile Masking for Localized Feedback. IEEE Transactions on Visualization and Computer Graphics 29, 5 (2023), 2211–2219. https://doi.org/10.1109/TVCG.2023.3247068

Digital Library

[39]

Weart S.r.l. 2022. TouchDIVER. https://weart.it/haptic-vr-products/touchdiver-haptic-glove/

[40]

Chan Lek Tan and Zachary A Knight. 2018. Regulation of body temperature by the nervous system. Neuron 98, 1 (2018), 31–48. https://doi.org/10.1016/j.neuron.2018.02.022

[41]

G v. Békésy. 1957. Sensations on the skin similar to directional hearing, beats, and harmonics of the ear. The Journal of the Acoustical Society of America 29, 4 (1957), 489–501.

[42]

Myla van Wegen, Just L. Herder, Rolf Adelsberger, Manuela Pastore-Wapp, Erwin E. H. van Wegen, Stephan Bohlhalter, Tobias Nef, Paul Krack, and Tim Vanbellingen. 2023. An Overview of Wearable Haptic Technologies and Their Performance in Virtual Object Exploration. Sensors 23, 3 (2023). https://doi.org/10.3390/s23031563

[43]

Marika Vellei, Richard De Dear, Christian Inard, and Ollie Jay. 2021. Dynamic thermal perception: A review and agenda for future experimental research. Building and Environment 205 (2021), 108269. https://doi.org/10.1016/j.buildenv.2021.108269

[44]

Haokun Wang, Yatharth Singhal, Hyunjae Gil, and Jin Ryong Kim. 2024. Thermal Masking: When the Illusion Takes Over the Real. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 419, 16 pages. https://doi.org/10.1145/3613904.3641941

Digital Library

[45]

Haokun Wang, Yatharth Singhal, and Jin Ryong Kim. 2023. Fabric Thermal Display using Ultrasonic Waves. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 336–345.

[46]

Haokun Wang, Yatharth Singhal, and Jin Ryong Kim. 2024. Let It Snow: Designing Snowfall Experience in VR. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 8, 2, Article 54 (may 2024), 24 pages. https://doi.org/10.1145/3659587

Digital Library

[47]

Ryo Watanabe, Ryuta Okazaki, and Hiroyuki Kajimoto. 2014. Mutual referral of thermal sensation between two thermal-tactile stimuli. In 2014 IEEE haptics symposium (haptics). IEEE, 299–302.

[48]

Rui Xiao and XZ Shawn Xu. 2021. Temperature sensation: from molecular thermosensors to neural circuits and coding principles. Annual review of physiology 83 (2021), 205–230. https://doi.org/10.1146/annurev-physiol-031220-095215

[49]

Kening Zhu, Simon Perrault, Taizhou Chen, Shaoyu Cai, and Roshan Lalintha Peiris. 2019. A sense of ice and fire: Exploring thermal feedback with multiple thermoelectric-cooling elements on a smart ring. International Journal of Human-Computer Studies 130 (2019), 234–247. https://doi.org/10.1016/j.ijhcs.2019.07.003

Digital Library

Cited By

Wang HSinghal YGil HKim J(2024)Demonstration of Fiery Hands: Thermal Gloves through Thermal and Tactile Integration2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00190(642-643)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00190

Index Terms

Fiery Hands: Designing Thermal Glove through Thermal and Tactile Integration for Virtual Object Manipulation
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
  2. Interaction design
    1. Interaction design theory, concepts and paradigms

Recommendations

Thermal In Motion: Designing Thermal Flow Illusions with Tactile and Thermal Interaction
UIST '24: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

This study presents a novel method for creating moving thermal sensations by integrating the thermal referral illusion with tactile motion. Conducted through three experiments on human forearms, the first experiment examined the impact of temperature ...
Fingertip Tactile Devices for Virtual Object Manipulation and Exploration
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

One of the main barriers to immersivity during object manipulation in virtual reality is the lack of realistic haptic feedback. Our goal is to convey compelling interactions with virtual objects, such as grasping, squeezing, pressing, lifting, and ...
Thermoelectric Tactile Display Based on the Thermal Grill Illusion
EuroHaptics '08: Proceedings of the 6th international conference on Haptics: Perception, Devices and Scenarios

Among the challenges of haptic displays is the objective of creating a tactile language. Tactile language is used as a substitute to visual and auditory languages when either of the modalities is overloaded or impaired. Haptic displays have been ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

UIST '24: Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology

October 2024

2334 pages

ISBN:9798400706288

DOI:10.1145/3654777

Editors:
Lining Yao
University of California, Berkeley
,
Mayank Goel
Carnegie Mellon University
,
Alexandra Ion
Carnegie Mellon University
,
Pedro Lopes
University of Chicago

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2024

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Research Council of Science and Technology

Conference

UIST '24

UIST '24: The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

PA, Pittsburgh, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
426
Total Downloads

Downloads (Last 12 months)426
Downloads (Last 6 weeks)124

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang HSinghal YGil HKim J(2024)Demonstration of Fiery Hands: Thermal Gloves through Thermal and Tactile Integration2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00190(642-643)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00190

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten