Designing Stick-Based Extended Reality Controllers: A Participatory Approach
Article No.: 103, Pages 1 - 6
Abstract
This work explores the design of stick-shaped tangible user interfaces (TUI) for Extended Reality (XR). While sticks are widely used in everyday objects, their applications as a TUI in XR have not been systematically studied. We conducted a participatory design session with twelve experts in XR and HCI to investigate the affordances of stick-based objects and how to utilize them in XR. As a result, we present a taxonomy of stick-based objects’ affordances and propose three types of stick-based XR controllers and their dynamic variations. The paper discusses design considerations for selecting the appropriate stick-based form in XR TUI design.
Supplemental Material
MP4 File
Talk Video
- Download
- 9.31 MB
- Transcript
References
[1]
Jacopo Aleotti, Giorgio Micconi, and Stefano Caselli. [n. d.]. Programming manipulation tasks by demonstration in visuo-haptic augmented reality. In 2014 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE) Proceedings (2014-10). 13–18. https://doi.org/10.1109/HAVE.2014.6954324
[2]
Ian M Bullock, Joshua Z Zheng, Sara De La Rosa, Charlotte Guertler, and Aaron M Dollar. 2013. Grasp frequency and usage in daily household and machine shop tasks. IEEE transactions on haptics 6, 3 (2013), 296–308.
[3]
M.R. Cutkosky. [n. d.]. On grasp choice, grasp models, and the design of hands for manufacturing tasks. 5, 3 ([n. d.]), 269–279. https://doi.org/10.1109/70.34763
[4]
M.R. Cutkosky. 1989. On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Transactions on Robotics and Automation 5, 3 (1989), 269–279. https://doi.org/10.1109/70.34763
[5]
Thomas Feix, Javier Romero, Heinz-Bodo Schmiedmayer, Aaron M Dollar, and Danica Kragic. 2015. The grasp taxonomy of human grasp types. IEEE Transactions on human-machine systems 46, 1 (2015), 66–77.
[6]
Colin Gallacher, Arash Mohtat, Steve Ding, and József Kövecses. [n. d.]. Toward open-source portable haptic displays with visual-force-tactile feedback colocation. In 2016 IEEE Haptics Symposium (HAPTICS) (2016-04). 65–71. https://doi.org/10.1109/HAPTICS.2016.7463157 ISSN: 2324-7355.
[7]
Matthias Harders, Gerald Bianchi, Benjamin Knoerlein, and Gabor Szekely. 2009. Calibration, Registration, and Synchronization for High Precision Augmented Reality Haptics. IEEE Transactions on Visualization and Computer Graphics 15, 1 (2009), 138–149. https://doi.org/10.1109/TVCG.2008.63
[8]
Shuqi He, Haonan Yao, Luyan Jiang, Kaiwen Li, Nan Xiang, Yue Li, Hai-Ning Liang, and Lingyun Yu. 2024. Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed Reality. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA, 33 pages. https://doi.org/10.1145/3613904.3642740
[9]
Thea Iberall. 1987. The nature of human prehension: Three dextrous hands in one. In Proceedings. 1987 ieee international conference on robotics and automation, Vol. 4. IEEE, 396–401.
[10]
Thomas H Massie and J K Salisbury. [n. d.]. The PHANTOM Haptic Interface: A Device for Probing Virtual Objects. ([n. d.]).
[11]
Meta. 2024. Meta Quest 2 Controller. https://www.meta.com/quest/accessories/quest-2-controllers/. Accessed: 1-Jan-2024.
[12]
Diego Monteiro, Hai-Ning Liang, Xian Wang, Wenge Xu, and Huawei Tu. 2021. Design and Development of a Low-Cost Device for Weight and Center of Gravity Simulation in Virtual Reality. In Proceedings of the 2021 International Conference on Multimodal Interaction (Montréal, QC, Canada) (ICMI ’21). Association for Computing Machinery, New York, NY, USA, 453–460. https://doi.org/10.1145/3462244.3479907
[13]
T. Nojima, D. Sekiguchi, M. Inami, and S. Tachi. [n. d.]. The SmartTool: a system for augmented reality of haptics. In Proceedings IEEE Virtual Reality 2002 (2002-03). 67–72. https://doi.org/10.1109/VR.2002.996506 ISSN: 1087-8270.
[14]
Majken K. Rasmussen, Esben W. Pedersen, Marianne G. Petersen, and Kasper Hornbæk. 2012. Shape-Changing Interfaces: A Review of the Design Space and Open Research Questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 735–744. https://doi.org/10.1145/2207676.2207781
[15]
Majken K. Rasmussen, Giovanni M. Troiano, Marianne G. Petersen, Jakob G. Simonsen, and Kasper Hornbæk. 2016. Sketching Shape-Changing Interfaces: Exploring Vocabulary, Metaphors Use, and Affordances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 2740–2751. https://doi.org/10.1145/2858036.2858183
[16]
Anne Roudaut, Abhijit Karnik, Markus Löchtefeld, and Sriram Subramanian. 2013. Morphees: Toward High "Shape Resolution" in Self-Actuated Flexible Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 593–602. https://doi.org/10.1145/2470654.2470738
[17]
Marcos Serrano, Jolee Finch, Pourang Irani, Andrés Lucero, and Anne Roudaut. 2022. Mold-It: Understanding How Physical Shapes Affect Interaction with Handheld Freeform Devices. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 402, 14 pages. https://doi.org/10.1145/3491102.3502022
[18]
Adwait Sharma, Michael A. Hedderich, Divyanshu Bhardwaj, Bruno Fruchard, Jess McIntosh, Aditya Shekhar Nittala, Dietrich Klakow, Daniel Ashbrook, and Jürgen Steimle. [n. d.]. SoloFinger: Robust Microgestures while Grasping Everyday Objects. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2021-05-07) (CHI ’21). Association for Computing Machinery, 1–15. https://doi.org/10.1145/3411764.3445197
[19]
Jotaro Shigeyama, Takeru Hashimoto, Shigeo Yoshida, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2019. Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering Based on Computational Perception Model. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300241
[20]
Alexa F. Siu, Mike Sinclair, Robert Kovacs, Eyal Ofek, Christian Holz, and Edward Cutrell. 2020. Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376353
[21]
Evan Strasnick, Christian Holz, Eyal Ofek, Mike Sinclair, and Hrvoje Benko. 2018. Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174218
[22]
Hsin-Ruey Tsai, Ching-Wen Hung, Tzu-Chun Wu, and Bing-Yu Chen. 2020. ElastOscillation: 3D Multilevel Force Feedback for Damped Oscillation on VR Controllers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376408
[23]
Vive. 2024. VIVE Controller. https://www.vive.com/ca/accessory/controller/. Accessed: 1-Jan-2024.
[24]
Xian Wang, Diego Monteiro, Lik-Hang Lee, Pan Hui, and Hai-Ning Liang. 2022. VibroWeight: Simulating Weight and Center of Gravity Changes of Objects in Virtual Reality for Enhanced Realism. In 2022 IEEE Haptics Symposium (HAPTICS). 1–7. https://doi.org/10.1109/HAPTICS52432.2022.9765609
[25]
André Zenner and Antonio Krüger. 2019. Drag:On: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300441
[26]
Yuhang Zhao, Cynthia L. Bennett, Hrvoje Benko, Edward Cutrell, Christian Holz, Meredith Ringel Morris, and Mike Sinclair. 2018. Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3173574.3173690
[27]
Joshua Z. Zheng, Sara De La Rosa, and Aaron M. Dollar. [n. d.]. An investigation of grasp type and frequency in daily household and machine shop tasks. In 2011 IEEE International Conference on Robotics and Automation (2011-05). 4169–4175. https://doi.org/10.1109/ICRA.2011.5980366 ISSN: 1050-4729.
[28]
Qian Zhou, Sarah Sykes, Sidney Fels, and Kenrick Kin. 2020. Gripmarks: Using Hand Grips to Transform In-Hand Objects into Mixed Reality Input. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376313
Index Terms
- Designing Stick-Based Extended Reality Controllers: A Participatory Approach
Recommendations
Stick-To-XR: Understanding Stick-Based User Interface Design for Extended Reality
DIS '24: Proceedings of the 2024 ACM Designing Interactive Systems ConferenceThis work explores the design of stick-shaped Tangible User Interfaces (TUI) for Extended Reality (XR). While sticks are widely used in everyday objects, their applications as a TUI in XR have not been systematically studied. We conducted a ...
User Experience Guidelines for Designing HMD Extended Reality Applications
Human-Computer Interaction – INTERACT 2019AbstractWith the rise of Extended Reality (XR) technologies, such as head mounted displays (HMD) for Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR), designers are presented with many unique challenges and opportunities when creating ...
A Framework for Virtual Reality with Tangible Augmented Reality-Based User Interface
In this paper, we propose a framework for virtual reality, I2-NEXT, which enables users to interact with virtual objects by tangible objects in immersive networked virtual environment. The primary goal of this framework is to support rapid development ...
Comments
Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Author Tags
Qualifiers
- Work in progress
- Research
- Refereed limited
Data Availability
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 154Total Downloads
- Downloads (Last 12 months)154
- Downloads (Last 6 weeks)37
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format