research-article

Chatbots to Support Children in Coping with Online Threats: Socio-technical Requirements

Authors:

Lara Schibelsky Godoy Piccolo,

Pinelopi Troullinou,

Harith AlaniAuthors Info & Claims

DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems Conference

Pages 1504 - 1517

https://doi.org/10.1145/3461778.3462114

Published: 28 June 2021 Publication History

Abstract

Online threats to children, in the form of cyberbullying, grooming, and sexting, have reached unprecedented and alarming levels. Support is still lacking despite endless efforts by child safety organisations and online safety educational programmes. This is mainly due to children feeling apprehensive in such situations, ashamed of revealing their distressing encounter to an adult or even for not having anyone to approach with their concerns. This paper investigates how children envision the potential support of a chatbot in such contexts. We captured design requirements for such a chatbot through a participatory design approach involving 110 schoolchildren in the UK. Using LEGO figures, they elaborated and performed stories featuring the interaction of a child under threat with a chatbot. The analysis of the dialogues in their performances and their reflections resulted in a set of expected tasks for the chatbot, a conversation flow, and novel socio-technical requirements addressing potential users’ main concerns and expectations.

References

[1]

WePROTECT Global Alliance. 2019. Global Threat Assessment 2019 Working together to end the sexual exploitation of children online. Retrieved July, 2020 from https://www.paconsulting.com/insights/2019/weprotect-global-threat-assessment/

[2]

Debbie Allnock and Pam Miller. 2013. No one noticed, no one heard: a study of disclosures of childhood abuse - NSPCC. Retrieved July, 2020 from https://learning.nspcc.org.uk/research-resources/2013/no-one-noticed-no-one-heard

[3]

Zahra Ashktorab and Jessica Vitak. 2016. Designing Cyberbullying Mitigation and Prevention Solutions through Participatory Design With Teenagers. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3895–3905. https://doi.org/10.1145/2858036.2858548

Digital Library

[4]

M Cecília C Baranauskas. 2014. Social awareness in HCI. Interactions 21, 4 (2014), 66–69.

Digital Library

[5]

Holly et al Bentley. 2019. How Safe are Our Children? 2019 – An Overview of Data on Child Abuse Online.Technical Report. NSPCC, UK.

[6]

Timothy W. Bickmore and Rosalind W. Picard. 2005. Establishing and Maintaining Long-Term Human-Computer Relationships. ACM Trans. Comput.-Hum. Interact. 12, 2 (June 2005), 293–327. https://doi.org/10.1145/1067860.1067867

Digital Library

[7]

Cezary Biele, Anna Jaskulska, Wieslaw Kopec, Jaroslaw Kowalski, Kinga Skorupska, and Aldona Zdrodowska. 2019. How might voice assistants raise our children?. In International Conference on Intelligent Human Systems Integration. Springer, New York, 162–167.

[8]

Michael Bonfert, Maximilian Spliethöver, Roman Arzaroli, Marvin Lange, Martin Hanci, and Robert Porzel. 2018. If You Ask Nicely: A Digital Assistant Rebuking Impolite Voice Commands. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI ’18). Association for Computing Machinery, New York, NY, USA, 95–102. https://doi.org/10.1145/3242969.3242995

Digital Library

[9]

L. Bowler, E. Mattern, and Cory P. Knobel. 2014. Developing Design Interventions for Cyberbullying: A Narrative-Based Participatory Approach. In iConference 2014 Proceedings. iSchools, Illinois, 153–162.

[10]

Virginia Braun and Victoria Clarke. 2012. Thematic analysis. APA handbook of research methods in psychology 2 (2012), 57–71.

[11]

Zhifa Chen, Yichen Lu, Mika P Nieminen, and Andrés Lucero. 2020. Creating a Chatbot for and with Migrants: Chatbot Personality Drives Co-Design Activities. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. Association for Computing Machinery, New York, NY, USA, 219–230.

Digital Library

[12]

Hyojin Chin, Lebogang Wame Molefi, and Mun Yong Yi. 2020. Empathy Is All You Need: How a Conversational Agent Should Respond to Verbal Abuse. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376461

Digital Library

[13]

Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications, Newbury Park, California.

[14]

Allison Druin. 2002. The role of children in the design of new technology. Behaviour and information technology 21, 1 (2002), 1–25.

[15]

Bentley H. et al. 2019. How safe are our children? 2019. An overview of data on child abuse online. Retrieved July, 2020 from https://learning.nspcc.org.uk/media/1747/how-safe-are-our-children-2019.pdf

[16]

Kathleen Kara Fitzpatrick, Alison Darcy, and Molly Vierhile. 2017. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR mental health 4, 2 (2017), e19.

[17]

Asbjørn Følstad and Petter Bae Brandtzæg. 2017. Chatbots and the New World of HCI. Interactions 24, 4 (June 2017), 38–42. https://doi.org/10.1145/3085558

Digital Library

[18]

Asbjørn Følstad, Cecilie Bertinussen Nordheim, and Cato Alexander Bjørkli. 2018. What Makes Users Trust a Chatbot for Customer Service? An Exploratory Interview Study. In Internet Science, Svetlana S. Bodrunova (Ed.). Springer International Publishing, Cham, 194–208.

[19]

Radhika Garg and Subhasree Sengupta. 2020. Conversational Technologies for In-Home Learning: Using Co-Design to Understand Children’s and Parents’ Perspectives. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376631

Digital Library

[20]

David Gauntlett. 2007. Creative explorations: New approaches to identities and audiences. Routledge, London.

[21]

Siddharth Gulati, Sonia Sousa, and David Lamas. 2019. Design, development and evaluation of a human-computer trust scale. Behaviour & Information Technology 38, 10 (2019), 1004–1015. https://doi.org/10.1080/0144929X.2019.1656779 arXiv:https://doi.org/10.1080/0144929X.2019.1656779

[22]

Ole Sejer Iversen, Rachel Charlotte Smith, and Christian Dindler. 2017. Child as Protagonist: Expanding the Role of Children in Participatory Design. In Proceedings of the 2017 Conference on Interaction Design and Children (Stanford, California, USA) (IDC ’17). Association for Computing Machinery, New York, NY, USA, 27–37. https://doi.org/10.1145/3078072.3079725

Digital Library

[23]

Junhan Kim, Yoojung Kim, Byungjoon Kim, Sukyung Yun, Minjoon Kim, and Joongseek Lee. 2018. Can a Machine Tend to Teenagers’ Emotional Needs? A Study with Conversational Agents. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems(Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3188548

Digital Library

[24]

Taewan Kim, Mintra Ruensuk, and Hwajung Hong. 2020. In Helping a Vulnerable Bot, You Help Yourself: Designing a Social Bot as a Care-Receiver to Promote Mental Health and Reduce Stigma. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376743

Digital Library

[25]

Ahmet Baki Kocaballi, Juan C. Quiroz, Liliana Laranjo, Dana Rezazadegan, Rafal Kocielnik, Leigh Clark, Q. Vera Liao, Sun Young Park, Robert J. Moore, and Adam Miner. 2020. Conversational Agents for Health and Wellbeing. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375154

Digital Library

[26]

Yi-Chieh Lee, Naomi Yamashita, Yun Huang, and Wai Fu. 2020. “I Hear You, I Feel You”: Encouraging Deep Self-Disclosure through a Chatbot. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376175

Digital Library

[27]

Kecheng Liu. 2000. Semiotics in information systems engineering. Cambridge University Press, Cambridge.

[28]

Ewa Luger and Abigail Sellen. 2016. “Like Having a Really Bad PA” The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI conference on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, 5286–5297.

Digital Library

[29]

Brenna McNally, Priya Kumar, Chelsea Hordatt, Matthew Louis Mauriello, Shalmali Naik, Leyla Norooz, Alazandra Shorter, Evan Golub, and Allison Druin. 2018. Co-Designing Mobile Online Safety Applications with Children. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3173574.3174097

Digital Library

[30]

Charles Morris. 1946. Signs, Language and Behaviour.

[31]

Andreea Muresan and Henning Pohl. 2019. Chats with Bots: Balancing Imitation and Engagement. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–6.

Digital Library

[32]

Jaya Narain, Tina Quach, Monique Davey, Hae Won Park, Cynthia Breazeal, and Rosalind Picard. 2020. Promoting Wellbeing with Sunny, a Chatbot That Facilitates Positive Messages within Social Groups. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3383062

Digital Library

[33]

Ofcom. 2020. Children and parents: Media use and attitudes report 2019. Ofcom, London.

[34]

Lara Piccolo. 2021. Recruitment and data collection forms for participatory design activity with children. https://doi.org/10.21954/ou.rd.14460063.v1

[35]

Lara S. G. Piccolo, Martino Mensio, and Harith Alani. 2019. Chasing the Chatbots. In Internet Science, Svetlana S. Bodrunova, Olessia Koltsova, Asbjørn Følstad, Harry Halpin, Polina Kolozaridi, Leonid Yuldashev, Anna Smoliarova, and Heiko Niedermayer (Eds.). Springer International Publishing, Cham, 157–169.

[36]

Janet C. Read, Matthew Horton, Gavin Sim, Peggy Gregory, Daniel Fitton, and Brendan Cassidy. 2013. CHECk: A Tool to Inform and Encourage Ethical Practice in Participatory Design with Children. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems (Paris, France) (CHI EA ’13). Association for Computing Machinery, New York, NY, USA, 187–192. https://doi.org/10.1145/2468356.2468391

Digital Library

[37]

JigSaw Research. 2020. Internet users’ experience of potential online harms: summary of survey research. Ofcom, London. https://www.ofcom.org.uk/__data/assets/pdf_file/0024/196413/concerns-and-experiences-online-harms-2020-chart-pack.pdf

[38]

Carolina Beniamina Rutta, Gianluca Schiavo, Massimo Zancanaro, and Elisa Rubegni. 2019. Comic-Based Digital Storytelling with Primary School Children. In Proceedings of the 18th ACM International Conference on Interaction Design and Children (Boise, ID, USA) (IDC ’19). Association for Computing Machinery, New York, NY, USA, 508–513. https://doi.org/10.1145/3311927.3325331

Digital Library

[39]

Alane Sanders. 2018. Generative tools enhance creative and ethical research with children. Qualitative Research Journal 18, 1 (2018), 33–44.

[40]

Elizabeth Sanders and Colin William. 2002. Harnessing people’s creativity: Ideation and expression through visual communication. In Focus Groups. CRC Press, US, 147–158.

[41]

Kyle-Althea Santos, Ethel Ong, and Ron Resurreccion. 2020. Therapist vibe: children’s expressions of their emotions through storytelling with a chatbot. In Proceedings of the Interaction Design and Children Conference. Association for Computing Machinery, New York, NY, USA, 483–494.

[42]

Joseph Seering, Michal Luria, Connie Ye, Geoff Kaufman, and Jessica Hammer. 2020. It Takes a Village: Integrating an Adaptive Chatbot into an Online Gaming Community. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376708

Digital Library

[43]

Amir Shevat. 2017. Designing bots: Creating conversational experiences. ” O’Reilly Media, Inc.”, Boston.

[44]

Ronald Stamper. 1973. Information in business and administrative systems. John Wiley & Sons, Inc., US.

Digital Library

[45]

Ronald K Stamper. 2001. Organisational semiotics: Informatics without the computer?In Information, organisation and technology. Springer, NY,US, 115–171.

[46]

UNICEF. 2020. COVID-19 and its implications for protecting children online. Retrieved July, 2020 from https://www.unicef.org/sites/default/files/2020-04/COVID-19-and-Its-Implications-for-Protecting-Children-Online.pdf

[47]

UNICEF. 2020. Cyberbullying: What is it and how to stop it 10 things teens want to know about cyberbullying. Retrieved August, 2020 from https://www.unicef.org/end-violence/how-to-stop-cyberbullying

[48]

Pamela Wisniewski, Arup Kumar Ghosh, Heng Xu, Mary Beth Rosson, and John M. Carroll. 2017. Parental Control vs. Teen Self-Regulation: Is There a Middle Ground for Mobile Online Safety?. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW ’17). Association for Computing Machinery, New York, NY, USA, 51–69. https://doi.org/10.1145/2998181.2998352

Digital Library

[49]

Pamela Wisniewski, Haiyan Jia, Na Wang, Saijing Zheng, Heng Xu, Mary Beth Rosson, and John M. Carroll. 2015. Resilience Mitigates the Negative Effects of Adolescent Internet Addiction and Online Risk Exposure. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 4029–4038. https://doi.org/10.1145/2702123.2702240

Digital Library

[50]

Ziang Xiao, Michelle X. Zhou, Wenxi Chen, Huahai Yang, and Changyan Chi. 2020. If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376131

Digital Library

Cited By

Ma RZhang ZGui XKou Y(2024)Labeling in the Dark: Exploring Content Creators’ and Consumers’ Experiences with Content Classification for Child Safety on YouTubeProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661565(1518-1532)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661565
Hedderich MBazarova NZou WShim RMa XYang Q(2024)A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying EducationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642379(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642379
Bellini RTseng EWarford NDaffalla AMatthews TConsolvo SWoelfer JGage Kelley PMazurek MCuomo DDell NRistenpart T(2024)SoK: Safer Digital-Safety Research Involving At-Risk Users2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00071(635-654)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00071
Show More Cited By

Recommendations

Are autistic children more vulnerable online? Relating autism to online safety, child wellbeing and parental risk management
NordiCHI '20: Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society

Many autistic children are active online users. Research suggests that they are subject to distress and poor wellbeing following online safety threats. However, it is unclear if autistic children are more likely to experience online safety risks ...
Finding common ground: comparing children's and parents' views on children's online safety
HCI '17: Proceedings of the 31st British Computer Society Human Computer Interaction Conference

This study examines children's online safety with a particular interest in children's online safety skills and conduct as well as parents' mediation strategies. A survey of 141 children and 163 parents was conducted. Children have encountered a variety ...
Children with ADHD and their Care Ecosystem: Designing Beyond Symptoms
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Designing for children with ADHD has been of increasing interest to the HCI community. However, current approaches do not adequately involve all relevant stakeholders, and primarily focus on addressing symptoms, following a medical model of disability ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems Conference

June 2021

2082 pages

ISBN:9781450384766

DOI:10.1145/3461778

Editors:
Wendy Ju
Jacobs Technion-Cornell Institute at Cornell Tech
,
Lora Oehlberg
University of Calgary
,
Sean Follmer
Stanford University
,
Sarah Fox
Carnegie Mellon University
,
Stacey Kuznetsov
Arizona State University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Centre for Policing Research and Learning

Conference

DIS '21

Sponsor:

SIGCHI

DIS '21: Designing Interactive Systems Conference 2021

June 28 - July 2, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
806
Total Downloads

Downloads (Last 12 months)203
Downloads (Last 6 weeks)15

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ma RZhang ZGui XKou Y(2024)Labeling in the Dark: Exploring Content Creators’ and Consumers’ Experiences with Content Classification for Child Safety on YouTubeProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661565(1518-1532)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661565
Hedderich MBazarova NZou WShim RMa XYang Q(2024)A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying EducationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642379(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642379
Bellini RTseng EWarford NDaffalla AMatthews TConsolvo SWoelfer JGage Kelley PMazurek MCuomo DDell NRistenpart T(2024)SoK: Safer Digital-Safety Research Involving At-Risk Users2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00071(635-654)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00071
Raveenthiran GSivarajah KMohamed Riyal AKugathasan VChandrasiri SRajendran K(2024)Revolutionalize Your Learning Experience with EQU ACCESS2024 International Conference on Electrical, Computer and Energy Technologies (ICECET10.1109/ICECET61485.2024.10698521(1-6)Online publication date: 25-Jul-2024
https://doi.org/10.1109/ICECET61485.2024.10698521
Lafrance St-Martin LVilleneuve S(2024)The uses of chatbots in the context of children and teenagers bullying: a systematic literature reviewCogent Education10.1080/2331186X.2024.231203211:1Online publication date: 9-Feb-2024
https://doi.org/10.1080/2331186X.2024.2312032
Bowman RCooney ONewbold JThieme AClark LDoherty GCowan B(2024)Exploring how politeness impacts the user experience of chatbots for mental health supportInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103181184:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103181
Zou WYang QDiFranzo DChen MHui WBazarova N(2024)Social Media Co-Pilot: Designing a Chatbot with Teens and Educators to Combat CyberbullyingInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2024.100680(100680)Online publication date: Aug-2024
https://doi.org/10.1016/j.ijcci.2024.100680
Karaturhan POrhan İKuşcu KYantaç A(2024)Informing the design of question-asking conversational agents for reflectionPersonal and Ubiquitous Computing10.1007/s00779-024-01831-728:6(1001-1019)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s00779-024-01831-7
Ong EPena FSotelo MCabredo IPascual J(2023)Chatbots as Peer Counselors in Helping Students Manage Stress: Socio-Technical Requirements2023 9th International HCI and UX Conference in Indonesia (CHIuXiD)10.1109/CHIuXiD59550.2023.10452735(59-64)Online publication date: 18-Nov-2023
https://doi.org/10.1109/CHIuXiD59550.2023.10452735
Ong E(2022)How HCI can Inform the Design of Human Language TechnologiesProceedings of the Asian HCI Symposium 202210.1145/3516492.3558811(44-47)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3516492.3558811
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents