research-article

When deictic gestures in a robot can harm child-robot collaboration

Authors:

Elmira Yadollahi,

Pierre DillenbourgAuthors Info & Claims

IDC '18: Proceedings of the 17th ACM Conference on Interaction Design and Children

Pages 195 - 206

https://doi.org/10.1145/3202185.3202743

Published: 19 June 2018 Publication History

Abstract

This paper describes research aimed at supporting children's reading practices using a robot designed to interact with children as their reading companion. We use a learning by teaching scenario in which the robot has a similar or lower reading level compared to children, and needs help and extra practice to develop its reading skills. The interaction is structured with robot reading to the child and sometimes making mistakes as the robot is considered to be in the learning phase. Child corrects the robot by giving it instant feedbacks. To understand what kind of behavior can be more constructive to the interaction especially in helping the child, we evaluated the effect of a deictic gesture, namely pointing on the child's ability to find reading mistakes made by the robot. We designed three types of mistakes corresponding to different levels of reading mastery. We tested our system in a within-subject experiment with 16 children. We split children into a high and low reading proficiency even-though they were all beginners. For the high reading proficiency group, we observed that pointing gestures were beneficial for recognizing some types of mistakes that the robot made. For the earlier stage group of readers pointing were helping to find mistakes that were raised upon a mismatch between text and illustrations. However, surprisingly, for this same group of children, the deictic gestures were disturbing in recognizing mismatches between text and meaning.

References

[1]

Maurice A Feldman. Early intervention : The essential readings, volume 6. John Wiley & Sons, 2008.

[2]

Carlos Valiente, Jodi Swanson, and Nancy Eisenberg. Linking students' emotions and academic achievement : When and why emotions matter. Child development perspectives, 6(2) :129--135, 2012.

[3]

Franciscus A van Vught. Isomorphism in higher education? towards a theory of differentiation and diversity in higher education systems. In In : V. Lynn Meek, L. Goedegebuure, O. Kivinen and R. Rinne (eds.), The Mockers and Mocked: Comparative Perspectives on Differentiation, Convergence and Diversity in Higher Education. Pergamon, 1996.

[4]

Ronald R Schmeck. Learning strategies and learning styles. Springer Science & Business Media, 2013.

[5]

Gavin Reid. Dyslexia : A practitioner's handbook. John Wiley & Sons, 2016.

[6]

David Wood, Jerome S Bruner, and Gail Ross. The role of tutoring in problem solving. Journal of child psychology and psychiatry, 17(2) :89--100, 1976.

[7]

Sinclair Goodlad and Beverley Hirst. Peer Tutoring. A Guide to Learning by Teaching. ERIC, 1989.

[8]

Etienne Wenger. Artificial intelligence and tutoring systems : computational and cognitive approaches to the communication of knowledge. Morgan Kaufmann, 2014.

[9]

Tony Belpaeme, James Kennedy, Paul Baxter, Paul Vogt, Emiel EJ Krahmer, Stefan Kopp, Kirsten Bergmann, Paul Leseman, Aylin C Küntay, Tilbe Göksun, et al. L2tor-second language tutoring using social robots. In Proceedings of the ICSR 2015 WONDER Workshop, 2015.

[10]

Daniel Leyzberg, Samuel Spaulding, and Brian Scassellati. Personalizing robot tutors to individuals' learning differences. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pages 423--430. ACM, 2014.

Digital Library

[11]

LaVonda Brown and Ayanna M Howard. Engaging children in math education using a socially interactive humanoid robot. In Humanoid Robots (Humanoids), 2013 13th IEEE-RAS International Conference on, pages 183--188. IEEE, 2013.

[12]

Takayuki Kanda, Takayuki Hirano, Daniel Eaton, and Hiroshi Ishiguro. Interactive robots as social partners and peer tutors for children : A field trial. Human-computer interaction, 19(1) :61--84, 2004.

Digital Library

[13]

Ginevra Castellano, André Pereira, Iolanda Leite, Ana Paiva, and Peter W McOwan. Detecting user engagement with a robot companion using task and social interaction-based features. In Proceedings of the 2009 international conference on Multimodal interfaces, pages 119--126. ACM, 2009.

Digital Library

[14]

Tiago Ribeiro, André Pereira, Amol Deshmukh, Ruth Aylett, and Ana Paiva. I'm the mayor : a robot tutor in enercities-2. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, pages 1675--1676. International Foundation for Autonomous Agents and Multiagent Systems, 2014.

Digital Library

[15]

James Kennedy, Paul Baxter, and Tony Belpaeme. The robot who tried too hard : Social behaviour of a robot tutor can negatively affect child learning. In Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction, pages 67--74. ACM, 2015.

Digital Library

[16]

Deanna Hood, Séverin Lemaignan, and Pierre Dillenbourg. When children teach a robot to write : An autonomous teachable humanoid which uses simulated handwriting. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, pages 83--90. ACM, 2015.

Digital Library

[17]

Catherine C Chase, Doris B Chin, Marily A Oppezzo, and Daniel L Schwartz. Teachable agents and the protégé effect : Increasing the effort towards learning. Journal of Science Education and Technology, 18(4) :334--352, 2009.

[18]

John R Kirby. Style, strategy, and skill in reading. In Learning strategies and learning styles, pages 229--274. Springer, 1988.

[19]

Omar Mubin, Catherine J Stevens, Suleman Shahid, Abdullah Al Mahmud, and Jian-Jie Dong. A review of the applicability of robots in education. Journal of Technology in Education and Learning, 1 :209--0015, 2013.

[20]

Iolanda Leite, Carlos Martinho, and Ana Paiva. Social robots for long-term interaction : a survey. International Journal of Social Robotics, 5(2) :291--308, 2013.

[21]

Fumihide Tanaka, Aaron Cicourel, and Javier R Movellan. Socialization between toddlers and robots at an early childhood education center. Proceedings of the National Academy of Sciences, 104(46) :17954--17958, 2007.

[22]

Eunja Hyun, Hyunmin Yoon, and Sooryun Son. Relationships between user experiences and children's perceptions of the education robot. In Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction, pages 199--200. IEEE Press, 2010.

Digital Library

[23]

Séverin Lemaignan, Alexis Jacq, Deanna Hood, Fernando Garcia, Ana Paiva, and Pierre Dillenbourg. Learning by teaching a robot : The case of handwriting. IEEE Robotics & Automation Magazine, 23(2) :56--66, 2016.

[24]

John A Bargh and Yaacov Schul. On the cognitive benefits of teaching. Journal of Educational Psychology, 72(5) :593, 1980.

[25]

Gautam Biswas, Krittaya Leelawong, Daniel Schwartz, Nancy Vye, and The Teachable Agents Group at Vanderbilt. Learning by teaching : A new agent paradigm for educational software. Applied Artificial Intelligence, 19(3--4) :363--392, 2005.

[26]

Elisa Rubegni and Monica Landoni. Evaluating engagement in reading : Comparing children and adult assessors. In Proceedings of the The 15th International Conference on Interaction Design and Children, pages 113--124. ACM, 2016.

Digital Library

[27]

Marie Carbo. Teaching reading with talking books. The Reading Teacher, 32(3) :267--273, 1978.

[28]

Carol Chomsky. After decoding : what? Language Arts, 53(3) :288--314, 1976.

[29]

Emilie Gerbier, Gérard Bailly, and Marie Line Bosse. Audio-visual synchronization in reading while listening to texts : Effects on visual behavior and verbal learning. Computer Speech & Language, 47 :74--92, 2018.

Digital Library

[30]

Min Fan, Alissa N Antle, and Emily S Cramer. Design rationale : Opportunities and recommendations for tangible reading systems for children. In Proceedings of the The 15th International Conference on Interaction Design and Children, pages 101--112. ACM, 2016.

Digital Library

[31]

Joseph E Michaelis and Bilge Mutlu. Someone to read with : Design of and experiences with an in-home learning companion robot for reading. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 301--312. ACM, 2017.

Digital Library

[32]

Hayes Raffle, Rafael Ballagas, Glenda Revelle, Hiroshi Horii, Sean Follmer, Janet Go, Emily Reardon, Koichi Mori, Joseph Kaye, and Mirjana Spasojevic. Family story play : reading with young children (and elmo) over a distance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1583--1592. ACM, 2010.

Digital Library

[33]

Bilge Mutlu, Jodi Forlizzi, and Jessica Hodgins. A storytelling robot : Modeling and evaluation of human-like gaze behavior. In Humanoid robots, 2006 6th IEEE-RAS international conference on, pages 518--523. IEEE, 2006.

[34]

Fotios Papadopoulos, Kerstin Dautenhahn, and Wan Ching Ho. Aibostory-autonomous robots supporting interactive, collaborative story-telling. Paladyn, Journal of Behavioral Robotics, 4(1) :10--22, 2013.

[35]

Jacqueline Kory and Cynthia Breazeal. Storytelling with robots: Learning companions for preschool children's language development. In Robot and Human Interactive Communication, 2014 RO-MAN: The 23rd IEEE International Symposium on, pages 643--648. IEEE, 2014.

[36]

Anthony D Pellegrini, Gene H Brody, and Irving E Sigel. Parents' book-reading habits with their children. Journal of Educational Psychology, 77(3) :332, 1985.

[37]

Adriana G Bus, Marinus H Van Ijzendoorn, and Anthony D Pellegrini. Joint book reading makes for success in learning to read : A meta-analysis on intergenerational transmission of literacy. Review of educational research, 65(1) :1--21, 1995.

[38]

David McNeill. Hand and mind: What gestures reveal about thought. University of Chicago press, 1992.

[39]

Jan Peter de Ruirer. 14 the production of gesture and speech. Language and gesture, 2 :284, 2000.

[40]

Laura M Justice, Paige C Pullen, and Khara Pence. Influence of verbal and nonverbal references to print on preschoolers' visual attention to print during storybook reading. Developmental Psychology, 44(3) :855, 2008.

[41]

Kshitij Sharma, Sarah D'Angelo, Darren Gergle, and Pierre Dillenbourg. Visual augmentation of deictic gestures in mooc videos. Singapore : International Society of the Learning Sciences, 2016.

[42]

Adam Kendon. Conducting interaction : Patterns of behavior in focused encounters, volume 7. CUP Archive, 1990.

[43]

David O Johnson, Raymond H Cuijpers, and David van der Pol. Imitating human emotions with artificial facial expressions. International Journal of Social Robotics, 5(4) :503--513, 2013.

[44]

Kenneth S Goodman. Analysis of oral reading miscues : Applied psycholinguistics. Reading Research Quarterly, pages 9--30, 1969.

[45]

Kenneth S Goodman. Miscue analysis : Applications to reading instruction. 1973.

[46]

James W Cunningham. A simplified miscue analysis for classroom and clinic. Reading Horizons, 24(2) :2, 1984.

[47]

Michael Tomasello et al. Joint attention as social cognition. Joint attention : Its origins and role in development, 103130.

[48]

Peter Mundy, Jessica Block, Christine Delgado, Yuly Pomares, Amy Vaughan Van Hecke, and Meaghan Venezia Parlade. Individual differences and the development of joint attention in infancy. Child development, 78(3) :938--954, 2007.

[49]

Chien-Ming Huang and Andrea Lockerd Thomaz. Joint attention in human-robot interaction. In AAAI Fall Symposium : Dialog with Robots, 2010.

[50]

Frederic Kaplan and Verena V Hafner. The challenges of joint attention. Interaction Studies, 7(2) :135--169, 2006.

[51]

Cynthia Breazeal, Cory D Kidd, Andrea Lockerd Thomaz, Guy Hoffman, and Matt Berlin. Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. In Intelligent Robots and Systems, 2005.(IROS 2005). 2005 IEEE/RSJ International Conference on, pages 708--713. IEEE, 2005.

Cited By

Chen PLiu Y(2024)IMPACT OF AI ROBOT IMAGE RECOGNITION TECHNOLOGY ON IMPROVING STUDENTS’ CONCEPTUAL UNDERSTANDING OF CELL DIVISION AND SCIENCE LEARNING MOTIVATIONJournal of Baltic Science Education10.33225/jbse/24.23.20823:2(208-220)Online publication date: 21-Apr-2024
https://doi.org/10.33225/jbse/24.23.208
Read JChisik YYadollahi EHorton M(2024)Children and Emerging Technologies: Ethical and Practical Research and DesignExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636275(1-4)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636275
Rudenko IRudenko ALilienthal AArras KBruno B(2024)The Child Factor in Child–Robot Interaction: Discovering the Impact of Developmental Stage and Individual CharacteristicsInternational Journal of Social Robotics10.1007/s12369-024-01121-516:8(1879-1900)Online publication date: 14-Aug-2024
https://doi.org/10.1007/s12369-024-01121-5
Show More Cited By

Index Terms

When deictic gestures in a robot can harm child-robot collaboration

Recommendations

When Children Teach a Robot to Write: An Autonomous Teachable Humanoid Which Uses Simulated Handwriting
HRI '15: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction

This article presents a novel robotic partner which children can teach handwriting. The system relies on the learning by teaching paradigm to build an interaction, so as to stimulate meta-cognition, empathy and increased self-esteem in the child user. ...
Children teach a care-receiving robot to promote their learning: field experiments in a classroom for vocabulary learning
Inaugural Special Issue: Intersection of Systems Sciences and Human Sciences

In contrast to conventional teaching agents (including robots) that were designed to play the role of human teachers or caregivers, we propose the opposite scenario in which robots receive instruction or care from children. We hypothesize that by using ...
Do adults and children learn differently from video lectures with an instructor’s deictic gestures?
Abstract
Evidence regarding the benefits of an instructor’s deictic gestures in video lectures on adults’ learning may not necessarily be reflective of their effect on children’s learning. Furthermore, there is a lack evidence regarding how deictic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IDC '18: Proceedings of the 17th ACM Conference on Interaction Design and Children

June 2018

789 pages

ISBN:9781450351522

DOI:10.1145/3202185

Conference Chairs:
Michail N. Giannakos
Norwegian University of Science and Technology, Norway
,
Letizia Jaccheri
Norwegian University of Science and Technology, Norway
,
Monica Divitini
Norwegian University of Science and Technology, Norway

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Swiss National Centre of Cometence in Research (NCCR)
Fundação para a Ciência e a Tecnologia (FCT)

Conference

IDC '18

Sponsor:

SIGCHI

IDC '18: Interaction Design and Children

June 19 - 22, 2018

Trondheim, Norway

Acceptance Rates

IDC '18 Paper Acceptance Rate 28 of 96 submissions, 29%;

Overall Acceptance Rate 172 of 578 submissions, 30%

Upcoming Conference

IDC '25

Sponsor:
sigchi

Interaction Design and Children

June 23 - 26, 2025

Reykjavik , Iceland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
529
Total Downloads

Downloads (Last 12 months)86
Downloads (Last 6 weeks)7

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen PLiu Y(2024)IMPACT OF AI ROBOT IMAGE RECOGNITION TECHNOLOGY ON IMPROVING STUDENTS’ CONCEPTUAL UNDERSTANDING OF CELL DIVISION AND SCIENCE LEARNING MOTIVATIONJournal of Baltic Science Education10.33225/jbse/24.23.20823:2(208-220)Online publication date: 21-Apr-2024
https://doi.org/10.33225/jbse/24.23.208
Read JChisik YYadollahi EHorton M(2024)Children and Emerging Technologies: Ethical and Practical Research and DesignExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636275(1-4)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636275
Rudenko IRudenko ALilienthal AArras KBruno B(2024)The Child Factor in Child–Robot Interaction: Discovering the Impact of Developmental Stage and Individual CharacteristicsInternational Journal of Social Robotics10.1007/s12369-024-01121-516:8(1879-1900)Online publication date: 14-Aug-2024
https://doi.org/10.1007/s12369-024-01121-5
Joshi SKamino WŠabanović S(2024)Social robot accessories for tailoring and appropriation of social robotsInternational Journal of Social Robotics10.1007/s12369-023-01077-yOnline publication date: 1-Mar-2024
https://doi.org/10.1007/s12369-023-01077-y
Geçkin V(2023)Assessing the Learning Outcomes of Robot-Assisted Second Language LearningHandbook of Research on Perspectives in Foreign Language Assessment10.4018/978-1-6684-5660-6.ch005(71-88)Online publication date: 13-Jan-2023
https://doi.org/10.4018/978-1-6684-5660-6.ch005
Guneysu AYadollahi EIndurkhya B(2023)Lessons Learned from in the Wild Child-Robot Interaction in Multiple Ecosystems of Care and EducationProceedings of the 11th International Conference on Human-Agent Interaction10.1145/3623809.3623855(142-151)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3623809.3623855
Wróbel AŹróbek KIndurkhya BSchaper MGunia AZguda P(2023)Are robots vegan? Unexpected behaviours in child-robot interactions and their design implicationsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585666(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585666
Love RLaw ECohen PKulić D(2023)Adapting a Teachable Robot’s Dialog Responses using Reinforcement Learning in Teaching Conversation2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309622(2541-2548)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309622
Wróbel AŹróbek KSchaper MZguda PIndurkhya B(2023)Age-Appropriate Robot Design: In-The-Wild Child-Robot Interaction Studies of Perseverance Styles and Robot’s Unexpected Behavior2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309332(1451-1458)Online publication date: 28-Aug-2023
https://doi.org/10.1109/RO-MAN57019.2023.10309332
Stower RAbdelghani RTschopp MEvangelista KChetouani MKappas A(2023)Exploring space for robot mistakes in child robot interactionsInteraction Studies. Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies / Social Behaviour and Communication in Biological and Artificial SystemsInteraction Studies10.1075/is.21034.sto23:2(243-288)Online publication date: 24-Mar-2023
https://doi.org/10.1075/is.21034.sto
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten