research-article

Open access

Usability Studies of an Egocentric Vision-Based Robotic Wheelchair

Authors:

Mohammed Kutbi,

Nikolaos Agadakos,

Philippos MordohaiAuthors Info & Claims

ACM Transactions on Human-Robot Interaction (THRI), Volume 10, Issue 1

Article No.: 4, Pages 1 - 23

https://doi.org/10.1145/3399434

Published: 20 July 2020 Publication History

All formats PDF

Abstract

Motivated by the need to improve the quality of life for the elderly and disabled individuals who rely on wheelchairs for mobility, and who may have limited or no hand functionality at all, we propose an egocentric computer vision based co-robot wheelchair to enhance their mobility without hand usage. The robot is built using a commercially available powered wheelchair modified to be controlled by head motion. Head motion is measured by tracking an egocentric camera mounted on the user’s head and faces outward. Compared with previous approaches to hands-free mobility, our system provides a more natural human robot interface because it enables the user to control the speed and direction of motion in a continuous fashion, as opposed to providing a small number of discrete commands. This article presents three usability studies, which were conducted on 37 subjects. The first two usability studies focus on comparing the proposed control method with existing solutions while the third study was conducted to assess the effectiveness of training subjects to operate the wheelchair over several sessions. A limitation of our studies is that they have been conducted with healthy participants. Our findings, however, pave the way for further studies with subjects with disabilities.

References

[1]

Rafael Barea, Luciano Boquete, Manuel Mazo, and Elena López. 2002. System for assisted mobility using eye movements based on electrooculography. IEEE Transactions on Neural Systems and Rehabilitation Engineering 10, 4 (2002), 209--218.

[2]

Patrice Boucher, Amin Atrash, Sousso Kelouwani, Wormser Honoré, Hai Nguyen, Julien Villemure, François Routhier, Paul Cohen, Louise Demers, Robert Forget, and Joelle Pineau. 2013. Design and validation of an intelligent wheelchair towards a clinically-functional outcome. Journal of Neuroengineering and Rehabilitation 10, 1 (2013), 1--16.

[3]

Jorge L. Candiotti, Deepan C. Kamaraj, Brandon Daveler, Cheng-Shiu Chung, Garrett G. Grindle, Rosemarie Cooper, and Rory A. Cooper. 2019. Usability evaluation of a novel robotic power wheelchair for indoor and outdoor navigation. Archives of Physical Medicine and Rehabilitation 100, 4 (2019), 627--637.

[4]

Tom Carlson, Robert Leeb, Ricardo Chavarriaga, and José del R. Millán. 2012. The birth of the brain-controlled wheelchair. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. 5444--5445.

[5]

Tom Carlson and Jose del R. Millan. 2013. Brain-controlled wheelchairs: A robotic architecture. IEEE Robotics and Automation Magazine 20, 1 (2013), 65--73.

[6]

Ramon Ceres, Jose L. Pons, Leopoldo Calderon, Antonio R. Jimenez, and Luis Azevedo. 2005. A robotic vehicle for disabled children. IEEE Engineering in Medicine and Biology Magazine 24, 6 (2005), 55--63.

[7]

Rodrigo Chacón-Quesada and Yiannis Demiris. 2018. Augmented reality control of smart wheelchair using eye-gaze--enabled selection of affordances. In Robots for Assisted Living workshop.

[8]

Yizhe Chang, Mohammed Kutbi, Niklaos Agadakos, Bo Sun, and Philippos Mordohai. 2017. A shared autonomy approach for wheelchair navigation based on learned user preferences. In Proceedings of the 5th International Workshop on Assistive Computer Vision and Robotics. 1490--1499.

[9]

Show-Hong Chen, Yu-Luen Chen, Ying-Han Chiou, Jen-Cheng Tsai, and Te-Son Kuo. 2003. Head-controlled device with M3S-based for people with disabilities. In International Conference of the IEEE Engineering in Medicine and Biology Society, Vol. 2. 1587--1589.

[10]

Albert M. Cook and Janice Miller Polgar. 2014. Assistive Technologies: Principles and Practice. Elsevier Health Sciences.

[11]

Linda Fehr, W. Edwin Langbein, and Steven B. Skaar. 2000. Adequacy of power wheelchair control interfaces for persons with severe disabilities: A clinical survey. Journal of Rehabilitation Research and Development 37, 3 (2000), 353--360.

[12]

Torsten Felzer and Bernd Freisleben. 2002. HaWCoS: The hands-free wheelchair control system. In Proceedings of the 5th International ACM Conference on Assistive Technologies. 127--134.

Digital Library

[13]

Andre Ferreira, Wanderley C. Celeste, Fernando A. Cheein, Teodiano F. Bastos-Filho, Mario Sarcinelli-Filho, and Ricardo Carelli. 2008. Human-machine interfaces based on EMG and EEG applied to robotic systems. Journal of Neuroengineering and Rehabilitation 5, 1 (2008), 1--15.

[14]

Mohamed Fezari and Mounir Bousbia-Salah. 2007. Speech and sensor in guiding an electric wheelchair. Automatic Control and Computer Sciences 41, 1 (2007), 39--43.

[15]

J. O. Gray, Pei Jia, Huosheng H. Hu, Tao Lu, and Kui Yuan. 2007. Head gesture recognition for hands-free control of an intelligent wheelchair. Industrial Robot: An International Journal 34, 1 (2007), 60--68.

[16]

Alaa Halawani, Shafiq Ur Réhman, Haibo Li, and Adi Anani. 2012. Active vision for controlling an electric wheelchair. Intelligent Service Robotics 5, 2 (2012), 89--98.

Digital Library

[17]

Jeong-Su Han, Z. Zenn Bien, Dae-Jin Kim, Hyong-Euk Lee, and Jong-Sung Kim. 2003. Human-machine interface for wheelchair control with EMG and its evaluation. In International Conference of the IEEE Engineering in Medicine and Biology Society, Vol. 2. 1602--1605.

[18]

Tuck-Voon How, Rosalie H. Wang, and Alex Mihailidis. 2013. Evaluation of an intelligent wheelchair system for older adults with cognitive impairments. Journal of Neuroengineering and Rehabilitation 10, 1 (2013), 90--105.

[19]

Xueliang Huo, Jia Wang, and Maysam Ghovanloo. 2008. A magneto-inductive sensor based wireless tongue-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering 16, 5 (2008), 497--504.

[20]

Pei Jia, Huosheng H. Hu, Tao Lu, and Kui Yuan. 2007. Head gesture recognition for hands-free control of an intelligent wheelchair. Industrial Robot: An International Journal 34, 1 (2007), 60--68.

[21]

Jin Sun Ju, Yunhee Shin, and Eun Yi Kim. 2009. Intelligent wheelchair (IW) interface using face and mouth recognition. In Proceedings of the 14th International Conference on Intelligent User Interfaces. ACM, 307--314.

Digital Library

[22]

Bong Keun Kim, Hideyuki Tanaka, and Yasushi Sumi. 2015. Robotic wheelchair using a high accuracy visual marker LentiBar and its application to door crossing navigation. In Proceedings of the IEEE International Conference on Robotics and Automation. 4478--4483.

[23]

Jeonghee Kim, Hangue Park, Joy Bruce, Erica Sutton, Diane Rowles, Deborah Pucci, Jaimee Holbrook, Julia Minocha, Beatrice Nardone, Dennis West, Anne Laumann, Eliot Roth, Mike Jones, Emir Veledar, and Maysam Ghovanloo. 2013. The tongue enables computer and wheelchair control for people with spinal cord injury. Science Translational Medicine 5, 213 (2013), 166--187.

[24]

R. L. Kirby, C. Smith, K. Parker, M. McAllister, J. Boyce, P. W. Rushton, F. Routhier, K. L. Best, B. Mortenson, and A. Brandt. 2015. Wheelchair Skills Training Program (WSTP) Manual, version 4.3. Retrieved from http://www.wheelchairskillsprogram.ca/eng/documents/WSTP_Manual_4.1.50.pdf.

[25]

Mohammed Kutbi, Yizhe Chang, and Philippos Mordohai. 2017. Hands-free wheelchair navigation based on egocentric computer vision: A usability study. In IROS Workshop on Assistance and Service Robotics in a Human Environment. IEEE.

[26]

James R. Lewis. 1995. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7, 1 (1995), 57--78.

Digital Library

[27]

Haoxiang Li, Mohammed Kutbi, Xin Li, Changjiang Cai, Philippos Mordohai, and Gang Hua. 2016. An egocentric computer vision based co-robot wheelchair. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 1829--1836.

Digital Library

[28]

Haoxiang Li, Philippos Mordohai, and Gang Hua. 2016. Attention-driven egocentric computer vision for robotic wheelchair navigation. In Proceedings of the 4th Workshop on Egocentric (First-Person) Vision. IEEE.

[29]

Ronald Lipskin. 1970. An evaluation program for powered wheelchair control systems. Bulletin of Prosthetics Research 6 (1970), 121--219.

[30]

Ana C. Lopes, Gabriel Pires, and Urbano Nunes. 2012. RobChair: Experiments evaluating brain-computer interface to steer a semi-autonomous wheelchair. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 5135--5136.

[31]

Ana C. Lopes, Gabriel Pires, and Urbano Nunes. 2013. Assisted navigation for a brain-actuated intelligent wheelchair. Robotics and Autonomous Systems 61, 3 (2013), 245--258.

Digital Library

[32]

Morten Enemark Lund, Henrik Vie Christiensen, Héctor Caltenco, Eugen Romulus Lontis, Bo Bentsen, and Lotte N.S. Andreasen Struijk. 2010. Inductive tongue control of powered wheelchairs. In Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society. 3361--3364.

[33]

Michael Mace, Khondaker Abdullah-Al-Mamun, Ravi Vaidyanathan, Shouyan Wang, and Lalit Gupta. 2010. Real-time implementation of a non-invasive tongue-based human-robot interface. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 5486--5491.

[34]

Eric Marchand, Fabien Spindler, and François Chaumette. 2005. ViSP for visual servoing: A generic software platform with a wide class of robot control skills. IEEE Robotics 8 Automation Magazine 12, 4 (2005), 40--52.

[35]

Georg Nebehay and Roman Pflugfelder. 2014. Consensus-based matching and tracking of keypoints for object tracking. In Proceedings of theIEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 862--869.

[36]

H. T. Nguyen, L. M. King, and Glenn Knight. 2004. Real-time head movement system and embedded linux implementation for the control of power wheelchairs. In Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society. 4892--4895.

[37]

Masato Nishimori, Takeshi Saitoh, and Ryosuke Konishi. 2007. Voice controlled intelligent wheelchair. In Proceedings of the Society of Instrument and Control Engineers (SICE) Annual Conference. 336--340.

[38]

Yuusuke Oonishi, Sehoon Oh, and Yoichi Hori. 2010. A new control method for power-assisted wheelchair based on the surface myoelectric signal. IEEE Transactions on Industrial Electronics 57, 9 (2010), 3191--3196.

[39]

Sarangi P. Parikh, Valdir Grassi, Vijay Kumar, and Jr. Jun Okamoto. 2005. Usability study of a control framework for an intelligent wheelchair. In Proceedings of the IEEE International Conference on Robotics and Automation. 4745--4750.

[40]

Elisa Perez, Carlos Soria, Natalia M. López, Oscar Nasisi, and Vicente Mut. 2013. Vision based interface applied to assistive robots. International Journal of Advanced Robotic Systems 10 (2013), 116--125.

[41]

Djoko Purwanto, Ronny Mardiyanto, and Kohei Arai. 2009. Electric wheelchair control with gaze direction and eye blinking. Artificial Life and Robotics 14, 3 (2009), 397--400.

[42]

Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y. Ng. 2009. ROS: An open-source robot operating system. In ICRA Workshop on Open Source Software. IEEE, 1--6.

[43]

Ericka Janet Rechy-Ramirez and Huosheng Hu. 2014. Head movement and facial expression-based human-machine interface for controlling an intelligent wheelchair. International Journal of Biomechatronics and Biomedical Robotics 3, 2 (2014), 80--91.

[44]

Margot Shields. 2004. Use of wheelchairs and other mobility support devices. Health Reports 15, 3 (May 2004), 37--41.

[45]

Richard C. Simpson. 2005. Smart wheelchairs: A literature review. Journal of Rehabilitation Research and Development 42, 4 (2005), 423--436.

[46]

Richard C. Simpson, Edmund F. LoPresti, and Rory A. Cooper. 2008. How many people would benefit from a smart wheelchair? Journal of Rehabilitation Research and Development 45, 1 (2008), 53--72.

[47]

Chun Sing Louis Tsui, Pei Jia, John Q. Gan, Huosheng Hu, and Kui Yuan. 2007. EMG-based hands-free wheelchair control with EOG attention shift detection. In Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 1266--1271.

[48]

Lai Wei, Huosheng Hu, and Kui Yuan. 2008. Use of forehead bio-signals for controlling an intelligent wheelchair. In Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 108--113.

[49]

Lai Wei, Huosheng Hu, and Yi Zhang. 2011. Fusing EMG and visual data for hands-free control of an intelligent wheelchair. International Journal of Humanoid Robotics 8, 04 (2011), 707--724.

[50]

Ruofei Xu, Robin Hartshorn, Ryan Huard, James Irwin, Kaitlyn Johnson, Gregory Nelson, Jon Campbell, Sakire Arslan Ay, and Matthew E. Taylor. 2016. Towards a semi-autonomous wheelchair for users with ALS. In IJCAI Workshop on Autonomous Mobile Service Robots.

[51]

Rui Zhang, Yuanqing Li, Yongyong Yan, Hao Zhang, Shaoyu Wu, Tianyou Yu, and Zhenghui Gu. 2016. Control of a wheelchair in an indoor environment based on a brain--computer interface and automated navigation. IEEE Transactions on Neural Systems and Rehabilitation Engineering 24, 1 (2016), 128--139.

[52]

Mark Zolotas, Joshua Elsdon, and Yiannis Demiris. 2018. Head-mounted augmented reality for explainable robotic wheelchair assistance. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. 1823--1829.

Digital Library

Cited By

Fang IChen YWang YZhang JZhang QXu JHe XGao WSu HLi YFeng C(2024)EgoPAT3Dv2: Predicting 3D Action Target from 2D Egocentric Vision for Human-Robot Interaction2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610283(3036-3043)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610283
Kim MKim KLee S(2024)Effects of track-based stair climbing robot on muscle activity, usability, and psychological anxiety: a preliminary studyDisability and Rehabilitation: Assistive Technology10.1080/17483107.2024.2393701(1-6)Online publication date: 20-Aug-2024
https://doi.org/10.1080/17483107.2024.2393701
Wang ZXu JZhang JSlavin RZhu D(2024)An intelligent assistive driving solution based on smartphone for power wheelchair mobilityJournal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2024.103105149:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.sysarc.2024.103105
Show More Cited By

Index Terms

Usability Studies of an Egocentric Vision-Based Robotic Wheelchair

Recommendations

Egocentric Computer Vision for Hands-Free Robotic Wheelchair Navigation
Abstract
In this paper, we present an approach for navigating a robotic wheelchair that provides users with multiple levels of autonomy and navigation capabilities to fit their individual needs and preferences. We focus on three main aspects: (i) ...
An egocentric computer vision based co-robot wheelchair
2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Motivated by the emerging needs to improve the quality of life for the elderly and disabled individuals who rely on wheelchairs for mobility, and who might have limited or no hand functionality at all, we propose an egocentric computer vision based co-...
A multi-agent control architecture for a robotic wheelchair

Assistant robots like robotic wheelchairs can perform an effective and valuable work in our daily lives. However, they eventually may need external help from humans in the robot environment particularly, the driver in the case of a wheelchair to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Human-Robot Interaction

ACM Transactions on Human-Robot Interaction Volume 10, Issue 1

Research Notes

March 2021

202 pages

EISSN:2573-9522

DOI:10.1145/3407734

Editors:
Odest Chadwicke Jenkins
University of Michigan, USA
,
Selma Sabanovic
Indiana University, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 July 2020

Accepted: 01 May 2020

Revised: 01 March 2020

Received: 01 January 2019

Published in THRI Volume 10, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Institute of Nursing Research of the National Institutes of Health
National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
1,526
Total Downloads

Downloads (Last 12 months)327
Downloads (Last 6 weeks)35

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fang IChen YWang YZhang JZhang QXu JHe XGao WSu HLi YFeng C(2024)EgoPAT3Dv2: Predicting 3D Action Target from 2D Egocentric Vision for Human-Robot Interaction2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610283(3036-3043)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610283
Kim MKim KLee S(2024)Effects of track-based stair climbing robot on muscle activity, usability, and psychological anxiety: a preliminary studyDisability and Rehabilitation: Assistive Technology10.1080/17483107.2024.2393701(1-6)Online publication date: 20-Aug-2024
https://doi.org/10.1080/17483107.2024.2393701
Wang ZXu JZhang JSlavin RZhu D(2024)An intelligent assistive driving solution based on smartphone for power wheelchair mobilityJournal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2024.103105149:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.sysarc.2024.103105
Borges LAbreu Rosa de SÁ ANaves E(2024)A heuristic-based approach for assessing usability in electric powered wheelchairs: A preliminary investigationRehabilitación10.1016/j.rh.2023.10083158:2(100831)Online publication date: Apr-2024
https://doi.org/10.1016/j.rh.2023.100831
Takagi LMiyafuji SPardomuan JKoike HWard JMcGill MMarky K(2023)LUNAChair: Remote Wheelchair System Linking Users to Nearby People and AssistantsProceedings of the Augmented Humans International Conference 202310.1145/3582700.3582714(122-134)Online publication date: 12-Mar-2023
https://dl.acm.org/doi/10.1145/3582700.3582714
Bilius LUngurean OVatavu R(2023)An Expressivity-Complexity Tradeoff?: User-Defined Gestures from the Wheelchair Space are Mostly DeicticExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585695(1-8)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585695
Bilius LUngurean OVatavu R(2023)Understanding Wheelchair Users’ Preferences for On-Body, In-Air, and On-Wheelchair GesturesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580929(1-16)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580929
Morbidi FDevigne LTeodorescu CFraudet BLeblong ÉCarlson TBabel MCaron GDelmas SPasteau FVailland GGouranton VGuégan SLe Breton RRagot N(2023)Assistive Robotic Technologies for Next-Generation Smart Wheelchairs: Codesign and Modularity to Improve Users’ Quality of LifeIEEE Robotics & Automation Magazine10.1109/MRA.2022.317896530:1(24-35)Online publication date: Mar-2023
https://doi.org/10.1109/MRA.2022.3178965
Maio RMarques BAlves JSantos BDias PLau N(2022)An Augmented Reality Serious Game for Learning Intelligent Wheelchair Control: Comparing Configuration and Tracking MethodsSensors10.3390/s2220778822:20(7788)Online publication date: 13-Oct-2022
https://doi.org/10.3390/s22207788
Sarsenbayeva Zvan Berkel NVelloso EGoncalves JKostakos V(2022)Methodological Standards in Accessibility Research on Motor Impairments: A SurveyACM Computing Surveys10.1145/354350955:7(1-35)Online publication date: 15-Dec-2022
https://dl.acm.org/doi/10.1145/3543509
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents