research-article

Interaction and recognition challenges in interpreting children's touch and gesture input on mobile devices

Authors:

Shreya MohanAuthors Info & Claims

ITS '12: Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces

Pages 225 - 234

https://doi.org/10.1145/2396636.2396671

Published: 11 November 2012 Publication History

Abstract

As mobile devices like the iPad and iPhone become increasingly commonplace, touchscreen interactions are quickly overtaking other interaction methods in terms of frequency and experience for many users. However, most of these devices have been designed for the general, typical user. Trends indicate that children are using these devices (either their parents' or their own) for entertainment or learning activities. Previous work has found key differences in how children use touch and surface gesture interaction modalities vs. adults. In this paper, we specifically examine the impact of these differences in terms of automatically and reliably understanding what kids meant to do. We present a study of children and adults performing touch and surface gesture interaction tasks on mobile devices. We identify challenges related to (a) intentional and unintentional touches outside of onscreen targets and (b) recognition of drawn gestures, that both indicate a need to design tailored interaction for children to accommodate and overcome these challenges.

References

[1]

Anthony, L. and Wobbrock, J.O. A lightweight multistroke recognizer for user interface prototypes. Proc. GI 2010, Canadian Information Processing Society (2010), 245--252.

Digital Library

[2]

Anthony, L. and Wobbrock, J.O. $N-protractor: a fast and accurate multistroke recognizer. Proc. GI 2012, (2012), 117--120.

Digital Library

[3]

Beery, K., Buktenica, N., and Beery, N.A. The Beery-Buktenica Developmental Test of Visual-Motor Integration, 5th Edition. Modern Curriculum Press, New Jersey, 2004.

[4]

Brown, Q. and Anthony, L. Toward comparing the touchscreen interaction patterns of kids and adults. ACM SIGCHI EIST Workshop, 4pp.

[5]

Brown, Q., Bonsignore, E., Hatley, L., Druin, A., Walsh, G., Foss, E., Brewer, R., Hammer, J., and Golub, E. Clear Panels: a technique to design mobile application interactivity. Proc. DIS 2010, ACM (2010), 360--363.

Digital Library

[6]

Chiong, C. and Shuler, C. Learning: Is there an app for that? Investigations of young children's usage and learning with mobile devices and apps. The Joan Ganz Cooney Center at Sesame Workshop, New York, NY, 2010.

[7]

Donker, A. and Reitsma, P. Aiming and clicking in young children's use of the computer mouse. Computers in Human Behavior 23, 6 (2007), 2863--2874.

Digital Library

[8]

Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., and Janet, S. UNIPEN project of on-line data exchange and recognizer benchmarks. Proc. ICPR 1994, IEEE (1994), 29--33.

[9]

Hammond, T.A., Logsdon, D., Paulson, B., Johnston, J., Peschel, J., Wolin, A., and Taele, P. A sketch recognition system for recognizing free-hand course of action diagrams. Proc. IAAI 2010, AAAI Press (2010), 1781--1786.

[10]

Harris, A., Rick, J., Bonnett, V., Yuill, N., Fleck, R., Marshall, P., and Rogers, Y. Around the table: Are multiple-touch surfaces better than single-touch for children's collaborative interactions? Proc. CSCL 2009, International Society of the Learning Sciences (2009), 335--344.

Digital Library

[11]

Hinrichs, U. and Carpendale, S. Gestures in the wild. Proc. CHI 2011, ACM (2011), 3023--3032.

Digital Library

[12]

Hourcade, J.P., Bederson, B.B., Druin, A., and Guimbretière, F. Differences in pointing task performance between preschool children and adults using mice. ACM Transactions on Computer-Human Interaction 11, 4 (2004), 357--386.

Digital Library

[13]

Hse, H.H. and Newton, A.R. Recognition and beautification of multi-stroke symbols in digital ink. Computers & Graphics 29, 4 (2005), 533--546.

Digital Library

[14]

Inkpen, K.M. Drag-and-drop versus point-and-click mouse interaction styles for children. ACM Transactions on Computer-Human Interaction 8, 1 (2001), 1--33.

Digital Library

[15]

Joiner, R., Messer, D., Light, P., and Littleton, K. It is best to point for young children: a comparison of children's pointing and dragging. Computers in Human Behavior 14, 3 (1998), 513--529.

[16]

Jones, T. An empirical study of children's use of computer pointing devices. Journal of Educational Computing Research 7, 1 (1991), 61--76.

[17]

Keates, S. and Trewin, S. Effect of age and Parkinson's disease on cursor positioning using a mouse. Proc. ASSETS 2005, ACM (2005), 68--75.

Digital Library

[18]

Li, Y. Gesture search: a tool for fast mobile data access. Proc. UIST 2010, ACM (2010), 87--96.

Digital Library

[19]

MacKenzie, I.S., Kauppinen, T., and Silfverberg, M. Accuracy measures for evaluating computer pointing devices. Proc. CHI 2001, ACM Press (2001), 9--16.

Digital Library

[20]

Norris, C. and Soloway, E. Envisioning the handheld-centric classroom. Journal of Educational Computing Research 30, 4 (2004), 281--294.

[21]

Pitrelli, J.F. and Perrone, M.P. Confidence-scoring post-processing for off-line handwritten-character recognition verification. Proc. ICDAR 2003, IEEE (2003), 278--282.

Digital Library

[22]

Read, J.C., MacFarlane, S., and Casey, C. Pens behaving badly-usability of pens and graphics tablets for text entry with children. Adj. Proc. UIST 2002, ACM Press (2002), 21--

[23]

Rick, J., Harris, A., Marshall, P., Fleck, R., Yuill, N., and Rogers, Y. Children designing together on a multi-touch tabletop: An analysis of spatial orientation and user interactions. Proc. IDC 2009, ACM (2009), 106--114.

Digital Library

[24]

Rubine, D. Specifying gestures by example. SIGGRAPH Computer Graphics 25, 4 (1991), 329--337.

Digital Library

[25]

Ryall, K., Morris, M.R., Everitt, K., Forlines, C., and Shen, C. Experiences with and observations of direct-touch tabletops. Proc. Tabletop 2006, IEEE (2006), 8pp.

Digital Library

[26]

Shin, N., Norris, C., and Soloway, E. Effects of handheld games on students learning in mathematics. Proc. ICLS 2006, International Society of the Learning Sciences (2006), 702--708.

Digital Library

[27]

Shuler, C. Pockets of Potential: Using Mobile Technologies to Promote Children's Learning. Joan Ganz Cooney Center at Sesame Workshop, New York, NY, 2009.

[28]

Vatavu, R.-D., Vogel, D., Casiez, G., and Grisoni, L. Estimating the perceived difficulty of pen gestures. Proc. INTERACT 2011, Springer (2011), 89--106.

Digital Library

[29]

Vogel, D. and Balakrishnan, R. Direct pen interaction with a conventional graphical user interface. Human-Computer Interaction 25, 4 (2010), 324--388.

[30]

Willems, D., Niels, R., Van Gerven, M., and Vuurpijl, L. Iconic and multi-stroke gesture recognition. Pattern Recognition 42, 12 (2009), 3303--3312.

Digital Library

[31]

Wobbrock, J.O., Aung, H.H., Rothrock, B., and Myers, B.A. Maximizing the guessability of symbolic input. Ext. Abstracts CHI 2005, ACM (2005), 1869--1872.

Digital Library

[32]

Wobbrock, J.O., Wilson, A.D., and Li, Y. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. Proc. UIST 2007, ACM (2007), 159--168.

Digital Library

Cited By

Bachert MHesenius M(2024)Towards a Framework for Evaluating Synthetic Surface GesturesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661327(22-30)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661327
Quander KMilky TAponte NCaceres Carrascal NWoodward J(2024)“Are you smart?”: Children's Understanding of “Smart” TechnologiesProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655787(625-638)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655787
Hosseini MHamidi HKermanshahani S(2024)The impacts of situational visual impairment on usability of touch screensMultimedia Tools and Applications10.1007/s11042-024-18689-983:34(81685-81709)Online publication date: 9-Mar-2024
https://doi.org/10.1007/s11042-024-18689-9
Show More Cited By

Index Terms

Interaction and recognition challenges in interpreting children's touch and gesture input on mobile devices
1. Human-centered computing

Recommendations

Examining the need for visual feedback during gesture interaction on mobile touchscreen devices for kids
IDC '13: Proceedings of the 12th International Conference on Interaction Design and Children

Surface gesture interaction styles used on modern mobile touchscreen devices are often dependent on the platform and application. Some applications show a visual trace of gesture input as it is made by the user, whereas others do not. Little work has ...
Using gamification to motivate children to complete empirical studies in lab environments
IDC '13: Proceedings of the 12th International Conference on Interaction Design and Children

In this paper, we describe the challenges we encountered and solutions we developed while collecting mobile touch and gesture interaction data in laboratory conditions from children ages 5 to 7 years old. We identify several challenges of conducting ...
Designing smarter touch-based interfaces for educational contexts

In next-generation classrooms and educational environments, interactive technologies such as surface computing, natural gesture interfaces, and mobile devices will enable new means of motivating and engaging students in active learning. Our foundational ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ITS '12: Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces

November 2012

430 pages

ISBN:9781450312097

DOI:10.1145/2396636

General Chairs:
Orit Shaer
Wellesley College, USA
,
Chia Shen
Harvard University, USA
,
Program Chairs:
Meredith Ringel Morris
Microsoft Research, USA
,
Michael Horn
Northwestern University, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITS'12

Sponsor:

SIGCHI

ITS'12: Interactive Tabletops and Surfaces

November 11 - 14, 2012

Massachusetts, Cambridge, USA

Acceptance Rates

ITS '12 Paper Acceptance Rate 24 of 63 submissions, 38%;

Overall Acceptance Rate 119 of 418 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

83
Total Citations
View Citations
1,395
Total Downloads

Downloads (Last 12 months)52
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bachert MHesenius M(2024)Towards a Framework for Evaluating Synthetic Surface GesturesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661327(22-30)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661327
Quander KMilky TAponte NCaceres Carrascal NWoodward J(2024)“Are you smart?”: Children's Understanding of “Smart” TechnologiesProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3655787(625-638)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3655787
Hosseini MHamidi HKermanshahani S(2024)The impacts of situational visual impairment on usability of touch screensMultimedia Tools and Applications10.1007/s11042-024-18689-983:34(81685-81709)Online publication date: 9-Mar-2024
https://doi.org/10.1007/s11042-024-18689-9
Dorouka PKalogiannakis MBlonder R(2024)Tablets and Apps for Promoting Nanoliteracy in Early Childhood Education: Results from an Experimental StudyJournal of Science Education and Technology10.1007/s10956-024-10132-w33:6(910-927)Online publication date: 9-Jul-2024
https://doi.org/10.1007/s10956-024-10132-w
Miao YTian QLin CSong TZhou YZhao JGao SYang MShen CElkind E(2023)Learning heuristically-selected and neurally-guided feature for age group recognition using unconstrained smartphone interactionProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/338(3029-3037)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/338
Hermann JDogangün AHesenius M(2023)Age-Based Differences in Drone Control Gestures: An Exploratory StudyProceedings of the 35th Australian Computer-Human Interaction Conference10.1145/3638380.3638401(49-58)Online publication date: 2-Dec-2023
https://dl.acm.org/doi/10.1145/3638380.3638401
Cosentino GGelsomini MGiannakos M(2023)MOVES: Going beyond hardwired multisensory environments for childrenProceedings of the 22nd Annual ACM Interaction Design and Children Conference10.1145/3585088.3594493(716-720)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3585088.3594493
Cosentino GGelsomini MSharma KGiannakos M(2023)Interaction Modalities and Children’s Learning in Multisensory Environments: Challenges and Trade-offsProceedings of the 22nd Annual ACM Interaction Design and Children Conference10.1145/3585088.3589385(397-410)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3585088.3589385
Woodward JRuiz J(2023)Designing Textual Information in AR Headsets to Aid in Adults’ and Children's Task PerformanceProceedings of the 22nd Annual ACM Interaction Design and Children Conference10.1145/3585088.3589373(27-39)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3585088.3589373
Severin BWerger OHesenius M(2023)Not What I was Trained for – Out-of-Distribution-Tests for Interactive AIsEngineering Interactive Computer Systems. EICS 2023 International Workshops and Doctoral Consortium10.1007/978-3-031-59235-5_12(127-147)Online publication date: 27-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-59235-5_12
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents