research-article

Gesture-based Interaction for Individuals with Developmental Disabilities in India

Authors:

Saurabh Srivastava,

Krishnaveni Achary,

Blessin Varkey,

Jaakko Hakulinen,

Markku Turunen,

Nitendra RajputAuthors Info & Claims

ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility

Pages 61 - 70

https://doi.org/10.1145/2982142.2982166

Published: 23 October 2016 Publication History

Abstract

Gesture-based interaction provides a multitude of benefits to individuals with disabilities, for example, enhancing social, motor and cognitive skills. However, applications that encourage self-efficacy by promoting a life-skill through simulations of real world scenarios are largely missing. We explore the benefits of using a gesture-based application for individuals with developmental disabilities. The context is a special school in New Delhi, Nai Disha, where we designed and developed an application, Kirana, that integrates arithmetic and social interaction to teach purchasing of items from a local grocery store. In our study, 18 participants with developmental disabilities, previously unable to visit a grocery store, used Kirana for three weeks. Our results indicate that gesture-based applications can teach a life skill and enable self-efficacy for individuals with developmental disabilities by breaking down complex tasks that require social, mathematical and decision-making skills.

References

[1]

Bartoli, L., Corradi, C., Garzotto, F., & Valoriani, M. (2013, June). Exploring motion-based touchless games for autistic children's learning. Proc. Interaction Design and Children (IDC) (pp. 102--111). ACM.

Digital Library

[2]

Bartoli, L., Garzotto, F., Gelsomini, M., Oliveto, L., & Valoriani, M. (2014). Designing and Evaluating Touchless Playful Interaction for ASD Children. Proc. Interaction Design and Children (IDC)

Digital Library

[3]

Boujarwah, F. A., Hong, H., Abowd, G. D., & Arriaga, R. I. (2011). Towards a framework to situate assistive technology design in the context of culture. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility (pp. 19--26). ACM.

Digital Library

[4]

Bauminger-Zviely, N., Eden, S., Zancanaro, M., Weiss, P. L., & Gal, E. (2013). Increasing social engagement in children with high-functioning autism spectrum disorder using collaborative technologies in the school environment. Autism, 17(3), 317--339.

[5]

Chang, Y. J., Chen, S. F., & Huang, J. D. (2011). A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in developmental disabilities, 32(6), 2566--2570.

[6]

Coles, C. D., Strickland, D. C., Padgett, L., & Bellmoff, L. (2007). Games that "work": Using computer games to teach alcohol-affected children about fire and street safety. Research in developmental disabilities, 28(5), 518--530.

[7]

Christinaki, E., Triantafyllidis, G., & Vidakis, N. (2010). A gesture-controlled Serious Game for teaching emotion recognition skills to preschoolers with autism. In Proceedings of fdg. 2013, 417--418.

[8]

Cromby, J. J., Standen, P. J., Newman, J., & Tasker, H. (1996). Successful transfer to the real world of skills practised in a virtual environment by students with severe learning difficulties. In Proc. 1st European Conference on Disability, Virtual Reality, and Associated Technologies (pp. 8--10).

[9]

Dias, M. B., & Brewer, E. (2009). How computer science serves the developing world. Communications of the ACM, 52(6), 74--80.

Digital Library

[10]

Damasio, A. R. (1999). The feeling of what happens: Body and emotion in the making of consciousness. Houghton Mifflin Harcourt.

[11]

Dourish, P. (2004). Where the action is: the foundations of embodied interaction. MIT press.

[12]

Filmer, D. (2008). Disability, poverty, and schooling in developing countries: results from 14 household surveys. The World Bank Economic Review, 22(1), 141--163.

[13]

Firth, N. I. A. L. L. (2012). Kinect cameras watch for autism. New scientist, 214(2863), 17

[14]

González-Ortega, D., Díaz-Pernas, F. J., Martínez-Zarzuela, M., & Antón-Rodríguez, M. (2014). A Kinect-based system for cognitive rehabilitation exercises monitoring. Computer methods and programs in biomedicine, 113(2), 620--631.

Digital Library

[15]

Gover, M. R. (1996). The Embodied Mind: Cognitive Science and Human Experience (Book). Mind, Culture, and Activity, 3(4), 295--299.

[16]

Grinker, R. R. (2008). Unstrange minds: Remapping the world of autism. Da Capo Press.

[17]

Kandalaft, M. R., Didehbani, N., Krawczyk, D. C., Allen, T. T., & Chapman, S. B. (2013). Virtual reality social cognition training for young adults with high-functioning autism. Journal of autism and developmental disorders, 43(1), 34--44.

[18]

Ke, F., & Im, T. (2013). Virtual-reality-based social interaction training for children with high-functioning autism. The Journal of Educational Research, 106(6), 441--461.

[19]

Khakhar, J., & Madhvanath, S. (2010, November). Jollymate: Assistive technology for young children with dyslexia. In Frontiers in Handwriting Recognition (ICFHR), 2010 International Conference on (pp. 576--580). IEEE.

Digital Library

[20]

Lahiri, U., Bekele, E., Dohrmann, E., Warren, Z., & Sarkar, N. (2013). Design of a virtual reality based adaptive response technology for children with autism. Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 21(1), 55--64.

[21]

Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. Basic books.

[22]

Lindgren, R., & Johnson-Glenberg, M. (2013). Emboldened by embodiment six precepts for research on embodied learning and mixed reality. Educational Researcher, 42(8), 445--452.

[23]

Lotan, M., Yalon-Chamovitz, S., & Weiss, P. L. T. (2009). Improving physical fitness of individuals with intellectual and developmental disability through a Virtual Reality Intervention Program. Research in developmental disabilities, 30(2), 229--239.

[24]

Malinverni, L., MoraGuiard, J., Padillo, V., Mairena, M., Hervás, A., & Pares, N. (2014, June). Participatory design strategies to enhance the creative contribution of children with special needs. Proc. Interaction Design and Children (IDC) (pp. 85--94). ACM.

Digital Library

[25]

Mäkelä, V., Heimonen, T., & Turunen, M. (2014). Magnetic Cursor: Improving Target Selection in Freehand Pointing Interfaces. In Proceedings of The International Symposium on Pervasive Displays (PerDis '14), 112, ACM,

Digital Library

[26]

Mennecke, B. E., Triplett, J. L., Hassall, L. M., Conde, Z. J., & Heer, R. (2011). An examination of a theory of embodied social presence in virtual worlds*. Decision Sciences, 42(2), 413--450.

[27]

Moore, D., Cheng, Y., McGrath, P., & Powell, N. J. (2005). Collaborative virtual environment technology for people with autism. Focus on Autism and Other Developmental Disabilities, 20(4), 231--243.

[28]

Padgett, L. S., Strickland, D., and Coles, C. D. (2006). Case study: Using a virtual reality computer game to teach fire safety skills to children diagnosed with fetal alcohol syndrome. Journal of Pediatric Psychology, 31(1), 65--70.

[29]

Parasuram, K. (2006). Variables that affect teachers' attitudes towards disability and inclusive education in Mumbai, India. Disability & Society, 21(3), 231--242.

[30]

Parés, N., Carreras, A., Durany, J., Ferrer, J., Freixa, P., Gómez, D., & Sanjurjo, À. (2005, June). Promotion of creative activity in children with severe autism through visuals in an interactive multisensory environment. In Proceedings of the 2005 conference on Interaction design and children (pp. 110--116). ACM.

Digital Library

[31]

Parsons, S., Mitchell, P., & Leonard, A. (2004). The use and understanding of virtual environments by adolescents with autistic spectrum disorders. Journal of Autism and Developmental disorders, 34(4), 449--466.

[32]

Parsons, S., & Cobb, S. (2011). State-of-the-art of virtual reality technologies for children on the autism spectrum. European Journal of Special Needs Education, 26(3), 355--366.

[33]

Parsons, T. D., Rizzo, A. A., Rogers, S., & York, P. (2009). Virtual reality in paediatric rehabilitation: A review. Developmental Neurorehabilitation, 12(4), 224--238.

[34]

Reid, D. (2002). Virtual reality and the person-environment experience. CyberPsychology & Behavior, 5(6), 559--564.

[35]

Ringland, K. E., Zalapa, R., Neal, M., Escobedo, L., Tentori, M., & Hayes, G. R. (2014, September). SensoryPaint: a multimodal sensory intervention for children with neurode-velopmental disorders. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 873--884). ACM.

Digital Library

[36]

Michael Rutter, Anthony Bailey, and Cathrine Lord. 2003. The social communication questionnaire: Manual. Western Psychological Services.

[37]

Sampath, H., Sivaswamy, J., & Indurkhya, B. (2010). Assistive systems for children with dyslexia and autism. ACM Sigaccess Accessibility and Computing, (96), 32--36.

Digital Library

[38]

Sharma, S., Srivastava, S., Achary, K., Varkey, B., Heimonen, T., Hakulinen, J. S., ... & Rajput, N. (2016, February). Promoting Joint Attention with Computer Supported Collaboration in Children with Autism. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (pp. 1560--1571). ACM.

Digital Library

[39]

Standen, P. J., & Brown, D. J. (2005). Virtual reality in the rehabilitation of people with intellectual disabilities: review. Cyberpsychology & behavior, 8(3), 272--282.

[40]

Thompson, E. (2007). Mind in life: Biology, phenomenology, and the sciences of mind. Harvard University Press.

[41]

Wallace, S., Parsons, S., Westbury, A., White, K., White, K., & Bailey, A. (2010). Sense of presence and atypical social judgments in immersive virtual environments Responses of adolescents with Autism Spectrum Disorders. Autism, 14(3), 199--213.

[42]

Wilson, M. (2002). Six views of embodied cognition. Psychonomic bulletin & review, 9(4)

[43]

Venkatesh, S., Phung, D., Duong, T., Greenhill, S., & Adams, B. (2013, April). TOBY: early intervention in autism through technology. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 3187--3196). ACM

Digital Library

Cited By

Cosentino GGiannakos M(2023)Multisensory Interaction and Analytics to Enhance Smart Learning Environments: A Systematic Literature ReviewIEEE Transactions on Learning Technologies10.1109/TLT.2023.324321016:3(414-430)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TLT.2023.3243210
Hernández Lara MCaro KMartínez-García A(2023)Technology for supporting emotion regulation of individuals with developmental disabilities: A scoping reviewResearch in Developmental Disabilities10.1016/j.ridd.2023.104467136(104467)Online publication date: May-2023
https://doi.org/10.1016/j.ridd.2023.104467
Saren SMukhopadhyay AGhose DBiswas P(2023)Comparing alternative modalities in the context of multimodal human–robot interactionJournal on Multimodal User Interfaces10.1007/s12193-023-00421-w18:1(69-85)Online publication date: 19-Oct-2023
https://doi.org/10.1007/s12193-023-00421-w
Show More Cited By

Index Terms

Gesture-based Interaction for Individuals with Developmental Disabilities in India
1. Human-centered computing
  1. Accessibility
    1. Empirical studies in accessibility
  2. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input

Recommendations

Designing Gesture-Based Applications for Individuals with Developmental Disabilities: Guidelines from User Studies in India
Special Issue of Papers from ASSETS 2016

Gesture interaction provides a multitude of benefits to individuals with developmental disabilities, from enhancing social, motor and cognitive skills to providing a safe and controlled environment for simulating real-world scenarios. As gesture-based ...
Kirana: A Gesture-based Market App for Life Skills Learning for Individuals with Developmental Disabilities
ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility

Kirana is a gesture-based application that simulates purchasing experience at a local Indian grocery store. It provides individuals with developmental difficulties a safe and controlled environment to explore the processes of purchasing an item: namely, ...
A Review of Augmented Reality-Based Human-Computer Interaction Applications of Gesture-Based Interaction
HCI International 2019 – Late Breaking Papers
Abstract
In recent years, augmented reality (AR) is an extremely growing field in information technology, computer science, and computer engineering. Although there are many recent works that use augmented reality for different purposes, most of the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility

October 2016

362 pages

ISBN:9781450341240

DOI:10.1145/2982142

General Chair:
Jinjuan Heidi Feng
Towson University, USA
,
Program Chair:
Matt Huenerfauth
Rochester Institute of Technology, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASSETS '16

Sponsor:

SIGACCESS

ASSETS '16: The 18th International ACM SIGACCESS Conference on Computers and Accessibility

October 23 - 26, 2016

Nevada, Reno, USA

Acceptance Rates

ASSETS '16 Paper Acceptance Rate 24 of 95 submissions, 25%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
378
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)1

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cosentino GGiannakos M(2023)Multisensory Interaction and Analytics to Enhance Smart Learning Environments: A Systematic Literature ReviewIEEE Transactions on Learning Technologies10.1109/TLT.2023.324321016:3(414-430)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TLT.2023.3243210
Hernández Lara MCaro KMartínez-García A(2023)Technology for supporting emotion regulation of individuals with developmental disabilities: A scoping reviewResearch in Developmental Disabilities10.1016/j.ridd.2023.104467136(104467)Online publication date: May-2023
https://doi.org/10.1016/j.ridd.2023.104467
Saren SMukhopadhyay AGhose DBiswas P(2023)Comparing alternative modalities in the context of multimodal human–robot interactionJournal on Multimodal User Interfaces10.1007/s12193-023-00421-w18:1(69-85)Online publication date: 19-Oct-2023
https://doi.org/10.1007/s12193-023-00421-w
Sridhar APoddar RJain MKumar P(2023)Challenges Faced by the Employed Indian DHH CommunityHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42280-5_13(201-223)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-42280-5_13
Choi YKim JPark CKim JYi JHong J(2022)We Play and Learn Rhythmically: Gesture-based Rhythm Game for Children with Intellectual Developmental Disabilities to Learn Manual SignProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517456(1-13)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517456
Colley MKränzle TRukzio E(2022)Accessibility-Related Publication Distribution in HCI Based on a Meta-AnalysisExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519701(1-28)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519701
Sharma SAvellan TLinna JAchary KTurunen MHakulinen JVarkey B(2020)Socio-Technical Aspirations for Children with Special NeedsACM Transactions on Accessible Computing10.1145/339607613:3(1-27)Online publication date: 3-Aug-2020
https://dl.acm.org/doi/10.1145/3396076
Sharma SAchary KKinnula MIivari NVarkey BRubegni EVasalou AParés NSawhney N(2020)Gathering garbage or going green?Proceedings of the Interaction Design and Children Conference10.1145/3392063.3394394(311-322)Online publication date: 21-Jun-2020
https://dl.acm.org/doi/10.1145/3392063.3394394
Patil SDennis DPabbaraju CShaheer NSimhadri HSeshadri VVarma MJain PGuimbretière FBernstein MReinecke K(2019)GesturePodProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology10.1145/3332165.3347881(403-415)Online publication date: 17-Oct-2019
https://dl.acm.org/doi/10.1145/3332165.3347881
Bayor ASitbon LPloderer BBircanin FKoplick SBrereton MBigham JAzenkot SKane S(2019)Leveraging ParticipationProceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3308561.3353793(143-155)Online publication date: 24-Oct-2019
https://dl.acm.org/doi/10.1145/3308561.3353793
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents