Stick-To-XR: Understanding Stick-Based User Interface Design for Extended Reality
Pages 168 - 179
Abstract
This work explores the design of stick-shaped Tangible User Interfaces (TUI) for Extended Reality (XR). While sticks are widely used in everyday objects, their applications as a TUI in XR have not been systematically studied. We conducted a participatory design session with twelve experts in XR and human-computer interaction to investigate the affordances of stick-based objects and how to utilize them in XR. The results led us to develop a taxonomy of stick-based objects’ affordances in terms of their functions and holding gestures. Following that, we proposed four types of stick-based XR controller forms and discussed their advantages and limitations. In the end, we juxtaposed twenty-six existing XR controllers against our proposed forms and identified Landed (Cane) Stick, Thin Stick’s flexible usages, and Modular Design as the major opportunities that remain unexamined yet for stick-based XR TUI design.
References
[1]
Jacopo Aleotti, Giorgio Micconi, and Stefano Caselli. 2014. Programming Manipulation Tasks by Demonstration in Visuo-Haptic Augmented Reality. In 2014 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE) Proceedings. 13–18. https://doi.org/10.1109/HAVE.2014.6954324
[2]
Amazon. 2022. Product description for Amavasion VR Golf Club Handle Accessories Compatible with Meta or Oculus Quest 2 Enhance Immersive VR Game Experience (White or Red) - Amazon.com. https://www.amazon.com/dp/B0B4W4HJDC?ref_=cm_sw_r_cp_ud_dp_MV5DZFR769FC3Q4R3GSY&skipTwisterOG=1&th=1 Accessed: 6-May-2024.
[3]
Amazon. 2022. Product description for HUIUKE VR Game Handle Accessories for Quest 2 Controllers, Extension Grips for Playing Beat Saber Gorilla Tag Long Arms, VR Handle Attachments Compatible with Playing VR Games - Amazon.com. https://www.amazon.com/Accessories-Quest-Controllers-Attachment-Compatible-2/dp/B0B4H8S1NB. Accessed: 6-May-2024.
[4]
Amazon. 2022. Product description for Magni Stock + - VR Carbon Fiber Controller Stock Rifle Adapter compatible with Meta Quest 2 or Quest Pro or Quest - Amazon.com. https://www.amazon.com/Magni-Stock-Controller-Adapter-compatible-Oculus/dp/B0BLBTTGGH. Accessed: 6-May-2024.
[5]
ArborXR. 2022. Enterprise Review of the Meta Quest Pro Headset - ArborXR. https://arborxr.com/blog/enterprise-review-of-the-meta-quest-pro-headset/ Accessed: 6-May-2024.
[6]
Armon. [n. d.]. Edero from Armon - Armon Products BV. https://www.armonproducts.com/products/edero/. Accessed: 6-May-2024.
[7]
Hrvoje Benko, Christian Holz, Mike Sinclair, and Eyal Ofek. 2016. NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 717–728. https://doi.org/10.1145/2984511.2984526
[8]
Daniel K.Y. Chen, Jean-Baptiste Chossat, and Peter B. Shull. 2019. HaptiVec: Presenting Haptic Feedback Vectors in Handheld Controllers using Embedded Tactile Pin Arrays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300401
[9]
Inrak Choi, Eyal Ofek, Hrvoje Benko, Mike Sinclair, and Christian Holz. 2018. CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174228
[10]
Jan Ciger, Mario Gutierrez, Frederic Vexo, and Daniel Thalmann. 2003. The Magic Wand. In Proceedings of the 19th Spring Conference on Computer Graphics (Budmerice, Slovakia) (SCCG ’03). Association for Computing Machinery, New York, NY, USA, 119–124. https://doi.org/10.1145/984952.984972
[11]
M.R. Cutkosky. 1989. On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Transactions on Robotics and Automation 5, 3 (1989), 269–279. https://doi.org/10.1109/70.34763
[12]
Thomas Feix, Javier Romero, Heinz-Bodo Schmiedmayer, Aaron M. Dollar, and Danica Kragic. 2016. The GRASP Taxonomy of Human Grasp Types. IEEE Transactions on Human-Machine Systems 46, 1 (2016), 66–77. https://doi.org/10.1109/THMS.2015.2470657
[13]
Twisted Barrel Games. 2021. Product Promotion for FOX ONE VR Flight Stick for Oculus Quest 1 and 2 - Facebook.com. https://www.facebook.com/TwistedBarrelShop/videos/587630915888387/. Accessed: 6-May-2024.
[14]
Matthias Harders, Gerald Bianchi, Benjamin Knoerlein, and Gabor Szekely. 2009. Calibration, Registration, and Synchronization for High Precision Augmented Reality Haptics. IEEE Transactions on Visualization and Computer Graphics 15, 1 (2009), 138–149. https://doi.org/10.1109/TVCG.2008.63
[15]
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’97). Association for Computing Machinery, New York, NY, USA, 234–241. https://doi.org/10.1145/258549.258715
[16]
Bret Jackson, Logan B Caraco, and Zahara M Spilka. 2020. Arc-Type and Tilt-Type: Pen-based Immersive Text Input for Room-Scale VR. In Proceedings of the 2020 ACM Symposium on Spatial User Interaction (Virtual Event, Canada) (SUI ’20). Association for Computing Machinery, New York, NY, USA, Article 18, 10 pages. https://doi.org/10.1145/3385959.3418454
[17]
Benjamin Knoerlein, Gábor Székely, and Matthias Harders. 2007. Visuo-haptic collaborative augmented reality ping-pong. In Proceedings of the International Conference on Advances in Computer Entertainment Technology (Salzburg, Austria) (ACE ’07). Association for Computing Machinery, New York, NY, USA, 91–94. https://doi.org/10.1145/1255047.1255065
[18]
Jaeyeon Lee, Mike Sinclair, Mar Gonzalez-Franco, Eyal Ofek, and Christian Holz. 2019. TORC: A Virtual Reality Controller for In-Hand High-Dexterity Finger Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300301
[19]
Thomas H Massie, J Kenneth Salisbury, 1994. The PHANTOM Haptic Interface: A Device for Probing Virtual Objects. In Proceedings of the ASME winter annual meeting, symposium on haptic interfaces for virtual environment and teleoperator systems. Chicago, IL, 295–300.
[20]
Meta. 2024. Meta Quest 2 Controller. https://www.meta.com/quest/accessories/quest-2-controllers/. Accessed: 6-May-2024.
[21]
Diego Monteiro, Hai-Ning Liang, Xian Wang, Wenge Xu, and Huawei Tu. 2021. Design and Development of a Low-cost Device for Weight and Center of Gravity Simulation in Virtual Reality. In Proceedings of the 2021 International Conference on Multimodal Interaction (Montréal, QC, Canada) (ICMI ’21). Association for Computing Machinery, New York, NY, USA, 453–460. https://doi.org/10.1145/3462244.3479907
[22]
T. Nojima, D. Sekiguchi, M. Inami, and S. Tachi. 2002. The SmartTool: a System for Augmented Reality of Haptics. In Proceedings IEEE Virtual Reality 2002. 67–72. https://doi.org/10.1109/VR.2002.996506
[23]
Duc-Minh Pham and Wolfgang Stuerzlinger. 2019. Is the Pen Mightier than the Controller? A Comparison of Input Devices for Selection in Virtual and Augmented Reality. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology (Parramatta, NSW, Australia) (VRST ’19). Association for Computing Machinery, New York, NY, USA, Article 35, 11 pages. https://doi.org/10.1145/3359996.3364264
[24]
Majken K. Rasmussen, Esben W. Pedersen, Marianne G. Petersen, and Kasper Hornbæk. 2012. Shape-Changing Interfaces: A Review of the Design Space and Open Research Questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 735–744. https://doi.org/10.1145/2207676.2207781
[25]
Majken K. Rasmussen, Giovanni M. Troiano, Marianne G. Petersen, Jakob G. Simonsen, and Kasper Hornbæk. 2016. Sketching Shape-changing Interfaces: Exploring Vocabulary, Metaphors Use, and Affordances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 2740–2751. https://doi.org/10.1145/2858036.2858183
[26]
Anne Roudaut, Abhijit Karnik, Markus Löchtefeld, and Sriram Subramanian. 2013. Morphees: Toward High "Shape Resolution" in Self-Actuated Flexible Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 593–602. https://doi.org/10.1145/2470654.2470738
[27]
Marcos Serrano, Jolee Finch, Pourang Irani, Andrés Lucero, and Anne Roudaut. 2022. Mold-It: Understanding How Physical Shapes Affect Interaction with Handheld Freeform Devices. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 402, 14 pages. https://doi.org/10.1145/3491102.3502022
[28]
Orit Shaer and Eva Hornecker. 2010. Tangible User Interfaces: Past, Present, and Future Directions. Foundations and Trends® in Human–Computer Interaction 3, 1–2 (2010), 4–137. https://doi.org/10.1561/1100000026
[29]
Adwait Sharma, Michael A. Hedderich, Divyanshu Bhardwaj, Bruno Fruchard, Jess McIntosh, Aditya Shekhar Nittala, Dietrich Klakow, Daniel Ashbrook, and Jürgen Steimle. 2021. SoloFinger: Robust Microgestures while Grasping Everyday Objects. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 744, 15 pages. https://doi.org/10.1145/3411764.3445197
[30]
Jotaro Shigeyama, Takeru Hashimoto, Shigeo Yoshida, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2019. Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering based on Computational Perception Model. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300241
[31]
Alejandro Jarillo Silva, Omar A. Domínguez Ramirez, Vicente Parra Vega, and Jesus P. Ordaz Oliver. 2009. PHANToM OMNI Haptic Device: Kinematic and Manipulability. In 2009 Electronics, Robotics and Automotive Mechanics Conference (CERMA). 193–198. https://doi.org/10.1109/CERMA.2009.55
[32]
Alexa F. Siu, Mike Sinclair, Robert Kovacs, Eyal Ofek, Christian Holz, and Edward Cutrell. 2020. Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376353
[33]
Evan Strasnick, Christian Holz, Eyal Ofek, Mike Sinclair, and Hrvoje Benko. 2018. Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174218
[34]
Hsin-Ruey Tsai, Ching-Wen Hung, Tzu-Chun Wu, and Bing-Yu Chen. 2020. ElastOscillation: 3D Multilevel Force Feedback for Damped Oscillation on VR Controllers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376408
[35]
Vive. 2024. VIVE Controller. https://www.vive.com/ca/accessory/controller/. Accessed: 6-May-2024.
[36]
Xian Wang, Diego Monteiro, Lik-Hang Lee, Pan Hui, and Hai-Ning Liang. 2022. VibroWeight: Simulating Weight and Center of Gravity Changes of Objects in Virtual Reality for Enhanced Realism. In 2022 IEEE Haptics Symposium (HAPTICS). 1–7. https://doi.org/10.1109/HAPTICS52432.2022.9765609
[37]
Eric Whitmire, Hrvoje Benko, Christian Holz, Eyal Ofek, and Mike Sinclair. 2018. Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173660
[38]
HyeonBeom Yi, Jiwoo Hong, Hwan Kim, and Woohun Lee. 2019. DexController: Designing a VR Controller with Grasp-Recognition for Enriching Natural Game Experience. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology (Parramatta, NSW, Australia) (VRST ’19). Association for Computing Machinery, New York, NY, USA, Article 22, 11 pages. https://doi.org/10.1145/3359996.3364263
[39]
André Zenner and Antonio Krüger. 2019. Drag:on: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300441
[40]
Yuhang Zhao, Cynthia L. Bennett, Hrvoje Benko, Edward Cutrell, Christian Holz, Meredith Ringel Morris, and Mike Sinclair. 2018. Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3173574.3173690
[41]
Joshua Z. Zheng, Sara De La Rosa, and Aaron M. Dollar. 2011. An Investigation of Grasp Type and Frequency in Daily Household and Machine Shop Tasks. In 2011 IEEE International Conference on Robotics and Automation. 4169–4175. https://doi.org/10.1109/ICRA.2011.5980366
[42]
Qian Zhou, Sarah Sykes, Sidney Fels, and Kenrick Kin. 2020. Gripmarks: Using Hand Grips to Transform In-Hand Objects into Mixed Reality Input. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376313
Index Terms
- Stick-To-XR: Understanding Stick-Based User Interface Design for Extended Reality
Recommendations
Designing Stick-Based Extended Reality Controllers: A Participatory Approach
CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing SystemsThis work explores the design of stick-shaped tangible user interfaces (TUI) for Extended Reality (XR). While sticks are widely used in everyday objects, their applications as a TUI in XR have not been systematically studied. We conducted a ...
A Framework for Virtual Reality with Tangible Augmented Reality-Based User Interface
In this paper, we propose a framework for virtual reality, I2-NEXT, which enables users to interact with virtual objects by tangible objects in immersive networked virtual environment. The primary goal of this framework is to support rapid development ...
Transtiff: Haptic Interaction with a Stick Interface with Various Stiffness
SIGGRAPH '23: ACM SIGGRAPH 2023 Emerging TechnologiesWe propose Transtiff, a stick-shaped device that can display various stiffness for stick-based haptic interaction. The device has a stiffness-changing joint replicating an artificial muscle mechanism in the relay portion of the stick to change its ...
Comments
Information & Contributors
Information
Published In
July 2024
3616 pages
ISBN:9798400705830
DOI:10.1145/3643834
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
DIS '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,158 of 4,684 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 298Total Downloads
- Downloads (Last 12 months)298
- Downloads (Last 6 weeks)81
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in