research-article

DualPanto: A Haptic Device that Enables Blind Users to Continuously Interact with Virtual Worlds

Authors:

Oliver Schneider,

Jotaro Shigeyama,

Thijs Jan Roumen,

Sebastian Marwecki,

Nico Boeckhoff,

Daniel Amadeus Gloeckner,

Patrick BaudischAuthors Info & Claims

UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

Pages 877 - 887

https://doi.org/10.1145/3242587.3242604

Published: 11 October 2018 Publication History

Abstract

We present a new haptic device that enables blind users to continuously interact with spatial virtual environments that contain moving objects, as is the case in sports or shooter games. Users interact with DualPanto by operating the me handle with one hand and by holding on to the it handle with the other hand. Each handle is connected to a pantograph haptic input/output device. The key feature is that the two handles are spatially registered with respect to each other. When guiding their avatar through a virtual world using the me handle, spatial registration enables users to track moving objects by having the device guide the output hand. This allows blind players of a 1-on-1 soccer game to race for the ball or evade an opponent; it allows blind players of a shooter game to aim at an opponent and dodge shots. In our user study, blind participants reported very high enjoyment when using the device to play (6.5/7).

Supplementary Material

suppl.mov (ufp1110.mp4)

Supplemental video

Download
179.48 MB

suppl.mov (ufp1110p.mp4)

Supplemental video

Download
12.70 MB

MP4 File (p877-schneider.mp4)

Download
244.94 MB

References

[1]

Audio Game Hub. http://www.audiogamehub.com

[2]

Audio Games. https://www.audiogames.net

[3]

Olivier Bau, Ivan Poupyrev, Ali Israr, and Chris Harrison. 2010. TeslaTouch: electrovibration for touch surfaces. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST '10). ACM, New York, NY, USA, 283--292.

Digital Library

[4]

Ravin Balakrishnan, George Fitzmaurice, Gordon Kurtenbach, and William Buxton. 1999. Digital tape drawing. In Proceedings of the 12th annual ACM symposium on User interface software and technology (UIST '99). ACM, New York, NY, USA, 161--169.

Digital Library

[5]

Johann Borenstein, and Iwan Ulrich. 1997. The GuideCane -a computerized travel aid for the active guidance of blind pedestrians. In IEEE International Conference on Robotics and Automation (ICRA '97), Albuquerque, NM, USA, vol. 2, 1283--1288.

[6]

Christophe Ramstein and Vincent Hayward. 1994. The pantograph: a large workspace haptic device for multimodal human computer interaction. In Conference Companion on Human Factors in Computing Systems (CHI '94). ACM, New York, NY, USA, 57--58.

Digital Library

[7]

Gianni Campion, Qi Wang, and Vincent Hayward. 2005. The Pantograph Mk-II: A Haptic Instrument. In Proc. IROS 2005, IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp. 723--728.

[8]

DiMaio, S. P., Salcudean, S. E., & Sirouspour, M. R. 2000. Haptic interaction within a planar environment. In Proceedings of the 2000 ASME International Mechanical Engineering Congress & Exposition, Dynamic Systems and Control Division. pp. 69--2.

[9]

J. Edward Colgate, Michael C. Stanley, and J. Michael Brown. 1995. Issues in the haptic display of tool use. In International Conference on Intelligent Robots and Systems 95.'Human Robot Interaction and Cooperative Robots'(IROS '95). IEEE, vol. 3,140--145.

Digital Library

[10]

Polly Edman. Tactile graphics. American Foundation for the Blind, 1992.

[11]

Colin Gallacher, Arash Mohtat, Steve Ding, and József Kövecses. 2016. Toward open-source portable haptic displays with visual-force-tactile feedback colocation. In Haptics Symposium (HAPTICS). IEEE, 65--71.

[12]

Eric Gunther and Sile O'Modhrain. 2003. Cutaneous grooves: composing for the sense of touch. Journal of New Music Research 32, no. 4, 369--381.

[13]

Mikro Horstmann, Cornelius Hagen, Alisdair King, Sijo Dijkstra, David Crombie, David Gareth Evans, George T. Ioannidis, Paul Blenkhorn, Otthein Herzog, and Christoph Schlieder. 2004. TeDUB: Automatic interpretation and presentation of technical diagrams for blind people. In Proceedings of Conference and Workshop on Assistive Technologies for Vision and Hearing Impairment (CVHI '04).

[14]

HyperBraille, http://www.hyperbraille.de/

[15]

Alexandra Ion, Edward Jay Wang, and Patrick Baudisch. 2015. Skin Drag Displays: Dragging a Physical Tactor across the User's Skin Produces a Stronger Tactile Stimulus than Vibrotactile. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2501--2504.

Digital Library

[16]

Ali Israr and Ivan Poupyrev. 2011. Tactile brush: drawing on skin with a tactile grid display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), New York, NY, USA, 2019--2028.

Digital Library

[17]

Susan J Lederman, Roberta L Klatzky. 1987. Hand movements: A window into haptic object recognition. In Cognitive Psychology, Science Direct, Volume 19, Issue 3, 342--368.

[18]

Pedro Lopes, Alexandra Ion, Willi Mueller, Daniel Hoffmann, Patrik Jonell, and Patrick Baudisch. 2015. Proprioceptive Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 939--948.

Digital Library

[19]

Pedro Lopes, Doǎa Yüksel, François Guimbretière, and Patrick Baudisch. 2016. Muscle-plotter: An Interactive System based on Electrical Muscle Stimulation that Produces Spatial Output. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 207--217.

Digital Library

[20]

Karon MacLean and Mario Enriquez. 2003. Perceptual design of haptic icons. In Proceedings of EuroHaptics. IEEE, 351--363.

[21]

Daniel Miller, Aaron Parecki, and Sarah A. Douglas. 2007. Finger dance: a sound game for blind people. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility (Assets '07). ACM, New York, NY, USA, 253--254.

Digital Library

[22]

Tony Morelli, John Foley, Luis Columna, Lauren Lieberman, and Eelke Folmer. VI-Tennis: a vibrotactile/audio exergame for players who are visually impaired. 2010. In Proceedings of the Fifth International Conference on the Foundations of Digital Games (FDG '10). ACM, New York, NY, USA, 147--154.

Digital Library

[23]

Tony Morelli, John Foley, and Eelke Folmer. 2010. Vi-bowling: a tactile spatial exergame for individuals with visual impairments. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '10). ACM, New York, NY, USA, 179--186.

Digital Library

[24]

Hermann, T., Höner, O., & Ritter, H. (2005, May). AcouMotion--an interactive sonification system for acoustic motion control. In International Gesture Workshop. Springer, Berlin, Heidelberg. 312--323.

Digital Library

[25]

Shades of Doom, http://www.gmagames.com/sod.html

[26]

Papa Sangre, http://www.papasangre.com/

[27]

Konstantinos Moustakas, Georgios Nikolakis, Konstantinos Kostopoulos, Dimitrios Tzovaras, and Michael G. Strintzis. 2007. Haptic rendering of visual data for the visually impaired. IEEE MultiMedia 14, no. 1.

Digital Library

[28]

Sabrina Paneels and J. C. Roberts. 2010. Review of Designs for Haptic Data Visualization. in IEEE Transactions on Haptics, vol. 3, no. 2, 119--137.

Digital Library

[29]

Peter Parente, and Gary Bishop. 2003. BATS: the blind audio tactile mapping system. In Proceedings of the ACM Southeast regional conference (ACMSE '03). ACM, New York, NY, USA, 132--137.

[30]

Dianne TV Pawluk, Richard J. Adams, and Ryo Kitada. 2015. Designing haptic assistive technology for individuals who are blind or visually impaired. IEEE transactions on haptics 8. IEEE, no. 3: 258--278.

[31]

Grégory Petit, Aude Dufresne, Vincent Levesque, Vincent Hayward, and Nicole Trudeau. 2008. Refreshable tactile graphics applied to schoolbook illustrations for students with visual impairment. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '08). ACM, New York, NY, USA, 89--96.

Digital Library

[32]

Beryl Plimmer, Andrew Crossan, Stephen A. Brewster, and Rachel Blagojevic. 2008. Multimodal collaborative handwriting training for visually-impaired people. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 393--402.

Digital Library

[33]

de Pascale M., Mulatto S., Prattichizzo D. (2008) Bringing Haptics to Second Life for Visually Impaired People. In Ferre M. (eds) Haptics: Perception, Devices and Scenarios. EuroHaptics 2008. Lecture Notes in Computer Science, vol 5024. Springer, Berlin, Heidelberg.

Digital Library

[34]

Gery W. Ryan and H. Russell Bernard. 2003. Techniques to Identify Themes. Field Methods, SAGE Journals, Vol 15, Issue 1, 85--109.

[35]

Steven Strachan, Wiertlewski, M., Zophoniasson, H., and Anastassova, M. 2013. ViPong: Probabilistic haptic feedback for eyes-free interaction. In IEEE World Haptics Conference (WHC). IEEE, 193--198.

[36]

Raised Line Drawing Kits: http://www.maxiaids.com/raised-line-drawing-kit

[37]

David A. Ross, and Bruce B. Blasch. 2000. Wearable interfaces for orientation and wayfinding. In Proceedings of the fourth international ACM conference on Assistive technologies (ASSETS '00). ACM, New York, NY, USA, 93--200.

Digital Library

[38]

Kyle Rector, Cynthia L. Bennett, and Julie A. Kientz. 2013. Eyes-free yoga: an exergame using depth cameras for blind & low vision exercise. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '13). New York, NY, USA.

Digital Library

[39]

Anne Roudaut, Andreas Rau, Christoph Sterz, Max Plauth, Pedro Lopes, and Patrick Baudisch. 2013. Gesture output: eyes-free output using a force feedback touch surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2547--2556.

Digital Library

[40]

Saiganesh Swaminathan, Thijs Roumen, Robert Kovacs, David Stangl, Stefanie Mueller, and Patrick Baudisch. 2016. Linespace: A Sensemaking Platform for the Blind. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2175--2185.

Digital Library

[41]

Oliver S. Schneider, Ali Israr, and Karon E. MacLean. 2015. Tactile Animation by Direct Manipulation of Grid Displays. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). ACM, New York, NY, USA, 21--30.

Digital Library

[42]

Survival Shooter Tutorial "Zombunny", Unity Asset store. https://www.assetstore.unity3d.com/en/#!/content/40756

[43]

Swell-form-graphics: http://www.americanthermoform.com/product/swell- form-graphics-ii-machine/

[44]

Anthony Talvas, Maud Marchal, and Anatole Lécuyer. 2014. A survey on bimanual haptic interaction. IEEE transactions on haptics 7, IEEE, no. 3: 285--300.

[45]

Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell, and Chia Shen. 2007. Lucid touch: a see-through mobile device. In Proceedings of the 20th annual ACM symposium on User interface software and technology (UIST '07). ACM, New York, NY, USA, 269--278.

Digital Library

[46]

Christopher Williamson and Ben Shneiderman. 1992. The dynamic HomeFinder: evaluating dynamic queries in a real-estate information exploration system. In Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '92). ACM, New York, NY, USA, 338--346.

Digital Library

[47]

L. Winfield, J. Glassmire, J. E. Colgate and M. Peshkin. 2007. T-PaD: Tactile Pattern Display through Variable Friction Reduction, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC '07). 421--426.

Digital Library

[48]

Yasuyoshi Yokokohji, Ralph L. Hollis, Takeo Kanade, Kazuyuki Henmi, and Tsuneo Yoshikawa. 1996. Toward machine mediated training of motor skills. Skill transfer from human to human via virtual environment. In 5th IEEE International Workshop on Robot and Human Communication. IEEE, 32--37.

[49]

C. B. Zilles, and J. K. Salisbury. 1995. A constraint-based god-object method for haptic display. In Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots. IEEE, Vol. 3, 146--151.

Digital Library

Cited By

Hassan L(2023)Accessibility of games and game-based applications: A systematic literature review and mapping of future directionsNew Media & Society10.1177/1461444823120402026:4(2336-2384)Online publication date: 9-Nov-2023
https://doi.org/10.1177/14614448231204020
Dudley JYin LGaraj VKristensson P(2023)Inclusive Immersion: a review of efforts to improve accessibility in virtual reality, augmented reality and the metaverseVirtual Reality10.1007/s10055-023-00850-827:4(2989-3020)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s10055-023-00850-8
Fatima SBaalsrud Hauge J(2023)Accessibility Issues Within Serious Games in Engineering Education for Visual ImpairmentEntertainment Computing – ICEC 202310.1007/978-981-99-8248-6_8(96-114)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1007/978-981-99-8248-6_8
Show More Cited By

Index Terms

DualPanto: A Haptic Device that Enables Blind Users to Continuously Interact with Virtual Worlds
1. Human-centered computing
  1. Accessibility
    1. Accessibility technologies
  2. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices

Recommendations

Virtual Showdown: An Accessible Virtual Reality Game with Scaffolds for Youth with Visual Impairments
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Virtual Reality (VR) is a growing source of entertainment, but people who are visually impaired have not been effectively included. Audio cues are motivated as a complement to visuals, making experiences more immersive, but are not a primary cue. To ...
Real-time sensory substitution to enable players who are blind to play video games using whole body gestures
FDG '11: Proceedings of the 6th International Conference on Foundations of Digital Games

Gesture-based interaction adds a new level of immersion to video games, but players who are blind are unable to play them as games use visual cues to indicate what gesture to provide and when. Though visual cues can be substituted with audio or haptic ...
Blind hero: enabling guitar hero for the visually impaired
Assets '08: Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility

Very few video games have been designed or adapted to allow people with vision impairment to play. Music/rhythm games however are particularly suitable for such people as they are perfectly capable of perceiving audio signals. Guitar Hero is a popular ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

October 2018

1016 pages

ISBN:9781450359481

DOI:10.1145/3242587

General Chairs:
Patrick Baudisch
Hasso-Plattner Institute, Germany
,
Albrecht Schmidt
LMU, Germany
,
Program Chair:
Andy Wilson
Microsoft Research, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Sciences and Engineering Research Council of Canada

Conference

UIST '18

Sponsor:

UIST '18: The 31st Annual ACM Symposium on User Interface Software and Technology

October 14, 2018

Berlin, Germany

Acceptance Rates

UIST '18 Paper Acceptance Rate 80 of 375 submissions, 21%;

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
863
Total Downloads

Downloads (Last 12 months)69
Downloads (Last 6 weeks)7

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hassan L(2023)Accessibility of games and game-based applications: A systematic literature review and mapping of future directionsNew Media & Society10.1177/1461444823120402026:4(2336-2384)Online publication date: 9-Nov-2023
https://doi.org/10.1177/14614448231204020
Dudley JYin LGaraj VKristensson P(2023)Inclusive Immersion: a review of efforts to improve accessibility in virtual reality, augmented reality and the metaverseVirtual Reality10.1007/s10055-023-00850-827:4(2989-3020)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s10055-023-00850-8
Fatima SBaalsrud Hauge J(2023)Accessibility Issues Within Serious Games in Engineering Education for Visual ImpairmentEntertainment Computing – ICEC 202310.1007/978-981-99-8248-6_8(96-114)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1007/978-981-99-8248-6_8
Valakou AMargetis GNtoa SStephanidis C(2023)A Framework for Accessibility in XR EnvironmentsHCI International 2023 – Late Breaking Posters10.1007/978-3-031-49215-0_31(252-263)Online publication date: 12-Dec-2023
https://doi.org/10.1007/978-3-031-49215-0_31
Tan SHuang WShang J(2022)Research Status and Trends of the Gamification Design for Visually Impaired People in Virtual RealityHCI in Games10.1007/978-3-031-05637-6_41(637-651)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-05637-6_41
Mazursky AKoo JMason TWoo SYang T(2021)Design and Experimental Evaluation of an Electrorheological Haptic Module with Embedded SensingApplied Sciences10.3390/app1116772311:16(7723)Online publication date: 22-Aug-2021
https://doi.org/10.3390/app11167723
Antoine AMalacria SMarquardt NCasiez GKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Interaction Illustration Taxonomy: Classification of Styles and Techniques for Visually Representing Interaction ScenariosProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445586(1-22)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445586
Jung JSon SLee SKim YLee GKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)ThroughHand: 2D Tactile Interaction to Simultaneously Recognize and Touch Multiple ObjectsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445530(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445530
Singhal TSchneider OKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Juicy Haptic Design: Vibrotactile Embellishments Can Improve Player Experience in GamesProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445463(1-11)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445463
Gay SPissaloux ERomeo KTruong N(2021)F2T: A Novel Force-Feedback Haptic Architecture Delivering 2D Data to Visually Impaired PeopleIEEE Access10.1109/ACCESS.2021.30914419(94901-94911)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3091441
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents