It's Still Complicated: From Privacy-Invasive Parental Control To Teen-Centric Solutions For Digital Resilience
It's Still Complicated: From Privacy-Invasive Parental Control To Teen-Centric Solutions For Digital Resilience
It's Still Complicated: From Privacy-Invasive Parental Control To Teen-Centric Solutions For Digital Resilience
Jinkyung Katie Park , Mamtaj Akter , and Pamela Wisniewski | Vanderbilt University
Karla Badillo-Urquiola | University of Notre Dame
We discuss the paradigm shift from restrictive approaches toward resilience-based solutions to promote
adolescents’ online safety and well-being. We describe how restrictive strategies induce a tradeoff
between teens’ privacy and online safety and present empirical studies that examine resilience-based
approaches.
www.computer.org/security 53
INCLUSIVE PRIVACY AND SECURITY
www.computer.org/security 55
INCLUSIVE PRIVACY AND SECURITY
to reflect on their social media usage throughout the the foundation of the design of ML-based sociotechnical
day and establish goals for the following day. Locus is systems to support them. In an effort to build teen-centric
a wrapper application that allows users to open social ML systems, researchers explored ways to work with teens
media apps directly through the Locus app. After open- to collect ecologically valid online risk data. For instance,
ing the Locus app, users can view the list of all social Razi et al.9 built an online system to collect youth-donated
media apps installed on their devices. When users select Instagram data. Researchers created a secure website
a specific social media app, they are shown a text-based where youth participants could fill out a survey about
reflective prompt before being taken to the desired their social media usage and unsafe experiences, upload
app asking, “What would make you feel good about their Instagram data, and annotate their own data (e.g.,
your time on Twitter today?” Responding to this entry conversation) as “safe” or “unsafe.” If unsafe, they anno-
prompt is optional and be done via text or speech input. tated for risk type and risk level, the context for each con-
Locus also sends a general notification once per day at versation around why it made them feel unsafe, and the
9 p.m. asking, “How do you think you’ll use social media relationships between conversation partners.
tomorrow?” Through a two-week experimental study The dataset resulted in building automated systems
involving pre- and postsurveys and exit interviews with to identify risky media, sexual conversation, and suicide
adolescents aged 14–18, Davis et al. found that adoles- ideation with high accuracy. It also allowed researchers
cents exhibited enhanced self-control and autonomy in to provide valuable insights into understanding the con-
managing their social media consumption, coupled with text and multidimensionality of online risk teens experi-
reduced instances of unintentional usage. Teens shared ence in private settings, as these are pivotal for designing
that they felt a heightened sense of purpose and empow- youth-centric and customized risk prevention strate-
erment in their interactions on social media platforms. gies to promote youth resilience from online risk. For
As such, intentional and planned media usage has instance, using the youth-provided labels for the level of
long been recognized as an effective means of fostering risk (i.e., high risk versus low risk), researchers were able
self-regulation, particularly from early childhood. With to build ML algorithms to prioritize identifying high-risk
initial guidance from parents, adolescents are capable of cases for prompt risk mitigation, while the algorithms
acquiring intentionality and making goal-oriented deci- take more time to take into account the conversation con-
sions as part of planned usage, which serves as a catalyst text for a more accurate understanding of the risk context
for self-regulatory development. By allowing adolescents for low-risk cases. The idea of building customized algo-
to establish their own healthy boundaries, intentional rithms for differing levels of online risk is that there are
and preplanned media use can alleviate privacy tensions cases where rapid responses are critical to prevent immi-
between parents and teens, especially those who strug- nent risk (e.g., suicide), while there are other cases where
gle with negotiating conflicting boundaries. At the same an accurate understanding of conversation context is
time, Davis et al. observed considerable individual varia- needed to avoid false alarms (e.g., content moderation).
tion among adolescents in the “just right” level of support As such, by taking human-in-the-loop approaches (i.e.,
for their self-regulation behaviors on social media.8 This working with youth to share their risk data and annotate
indicates the need for designing interventions considering for risk experience themselves) to design automated risk
factors such as gender, age, socioeconomic status, and race detection algorithms, researchers have been able to move
as well as the role of individual characteristics and motiva- toward youth-centered and context-aware “real-time” risk
tion to change one’s social media use. In addition, adoles- detection models as “just-in-time” interventions to miti-
cents’ engagement with social media could be varied for gate their online risk experience.
social media platforms with different affordances. There- In the meantime, building risk detection with data
fore, how to tailor interventions to cater to adolescents donation approaches could face challenges, rang-
with different needs and expectations as well as differing ing from the technical issues of dealing with gather-
social media environments is a crucial design consider- ing a sensitive dataset to ethical considerations. One
ation that warrants further investigation. major technical challenge, as noted by Razi et al., was
the compatibility between Instagram’s data and the
Youth-Centered Risk Detection data collection systems. Instagram frequently changes
From a human-centered perspective, collecting ground- how it organizes and formats user data, necessitat-
truth annotations from those who experience the risk ing continuous technical updates to the system. More
ensures that the training risk detection models reflect importantly, collecting online risk data from minors
real-world experiences and accurately represent the risks requires increased precautions, considering the com-
users face online. Risk perceptions are highly subjective; plexity and sensitive nature of private data. Therefore,
therefore, understanding the risk perceptions of people a series of additional measures beyond the institutional
who experienced the risk (i.e., adolescents in our case) is review (e.g., National Institutes of Health Certificate of
www.computer.org/security 57
INCLUSIVE PRIVACY AND SECURITY
the sender. The private scam and explicit content nudge Moreover, as nudges influence one’s behaviors, the
used filters to censor the risk, with choices to view, most prominent ethical concern related to nudging is
delete, and inform others. that it could compromise adolescent autonomy. In the
With these real-time nudges designed with teens, context of adolescent online safety, risk prevention re-
Agha et al. moved forward with co-designing social me- quires content moderation and censorship, which are
dia situation platforms to evaluate online safety nudges considered a breach of freedom of speech in many con-
with teens. The rationale behind employing simulated texts. However, teen participants in the study by Aga et
environments lies in al. perceived that con-
the endeavor to assess trolled and personalized
the impact of nudges ways of filtering unsafe
on promoting safe on- content respected their
line decision-making Nudges, defined as subtle cues intended to decision-making auton-
among adolescents, influence behavior, are being investigated omy. In addition, teens
all within settings that believed that moder-
closely mirror real-life
as effective means of guiding adolescents’ ating harmful content
scenarios while safe- actions without removing their autonomy. was necessary and that
guarding teens from compromising an in-
real risks. Therefore, dividual’s freedom of
Agha et al. risk scenar- speech was reasonable
ios and nudge designs if it protected minors
to a cohort of 20 teens to refine the designs. Teens were from online harm.11 Therefore, nudges that prompt
asked to redesign at least one or more of the risk scenarios the perpetrator to reconsider their actions, along with
and nudges with high-level feedback to make them more the freedom to continue—but not without conse-
realistic and effective. Feedback was provided through de- quences—should be carefully designed.
sign annotations on Fig Jam, along with verbal discussions.
They found that teens co-designed risk scenarios that Socioecological Approaches to
were subtle and higher in risk to be believable, perpetu- Adolescent Online Safety
ated by risky personas that tricked the teen by establish- The landscape of adolescent online safety has shifted
ing trust or a shared context. Teens recommended nudges toward collaborative family-based approaches, fostering
for risk prevention through personalized sensitivity fil- communication, privacy, and autonomy within digital fam-
ters, with the autonomy to view the risk. Moreover, teens ily contexts. Recent studies have investigated joint family
expressed a desire for proactive coping mechanisms, oversight mechanisms, enabling parents and teens to make
measures to hold perpetrators accountable, and educa- decisions together and providing teens the autonomy to
tional community guidelines. In terms of evaluating these support themselves and their families in online risk man-
nudges, the majority of participants expressed a prefer- agement. These innovations signal a move toward col-
ence for simulated social media environments wherein laborative models, recognizing the varied needs within
they could observe tangible behavioral changes without families. However, while family-centric strategies remain
exposure to actual risks. Concurrently, teens emphasized prevalent in industry and policy discourse, there is a grow-
the importance of transparency regarding the collection ing recognition of the limitations of solely relying on
and recording of data to address any privacy concerns parental mediation, particularly for vulnerable youth with-
stemming from their involvement in the study. out robust familial support systems. This section lays the
Meanwhile, we recognize that the implementation of groundwork for examining the socioecological framework,
real-time online safety nudges is not simple, as they rely which integrates digital parenting with broader social sys-
on the accurate detection of risk at the right moment. tems to promote resilience and community-based support
Although the just-in-time behavioral interventions dis- in adolescent online safety.
cussed can be effective in adolescents’ decisions toward
their online safety, they cannot cater to all adolescents Family-Based Collaborative Approaches
with different digital experiences and needs. For the In recent years, researchers have explored the imple-
interventions to be effective, they need to be context mentation of collaborative family-centric approaches
aware, which, in many cases, relies on techniques such as to address adolescent online safety concerns. These
ML-based risk detection. Therefore, collaborative efforts approaches aim to facilitate open discussions within
bridging the design space and technical implementation families while respecting teens’ privacy and autonomy
of the designed systems are crucial to moving toward pro- in making not only their own online safety decisions but
viding personalized nudge-based intervention. also decisions related to their families’ mobile privacy
www.computer.org/security 59
INCLUSIVE PRIVACY AND SECURITY
solutions can be adaptable to diverse needs and con- strategies could be different among cultures. Therefore,
texts. For instance, teens in foster care are even more future work should explore resilience-based adoles-
susceptible to higher levels of online risk, such as sex cent online safety solutions from a global perspective
trafficking. However, foster parents often lack the tech- to extend the discussion on adolescent online safety,
nology expertise to effectively manage teens’ user of considering different cultural, legal, and social contexts.
technology. As a result, they resorted to restrictive prac- This way, teen-centered online safety solutions would be
tices.13 Online safety technologies in which the power is applicable to a wider audience and contribute to a more
balanced between the stakeholders of the foster youth’s comprehensive understanding of the issues at hand.
support system and the foster youth are needed to One way to account for these important contextual
empower them to make their own decisions and learn. differences is through teen-centered design that puts
However, few evidence-based interventions to empower teens as the primary stakeholders and authority of their
foster youth self-regulation and online safety have been lived digital experiences. By shifting the power dynamic
developed. Therefore, the new systems should be devel- from focusing on the needs and perspectives of parents
oped for and with foster youth with input from foster and adults, amplifying adolescent voices can empower
parents and caseworkers. them to learn how to self-regulate their online behaviors
Moreover, much of the discourse around online in ways that promote resilience, autonomy, and safety.
safety currently emerges from the Global North (e.g., Further, engaging teens as co-designers and researchers
North America, Europe, and East Asia); however, can lead to novel design patterns and solutions that will
research evidence shows that cultural factors may con- transform the current technology landscape into one that
tribute to different parental mediation strategies. For promotes the digital inclusion of teens in the shaping of
example, Western European parents take more protec- the platforms in which they engage and the policies put
tive approaches, even if it might cost the children online forth to protect them online. Finally, we recognize that
opportunities, while parents of Nordic and northern resilience-based online safety solutions cannot be con-
European countries favor children’s rights and freedoms sidered effective until they have been built, implemented,
in online environments, even if this may put children at and evaluated in real-world settings. Therefore, future
risk.14 As such, the effectiveness of parental mediation work to implement the suggested solutions and evaluate
them with adolescents in realistic settings is essential. As
Social Ecologies of Adolescent Resilience Framework we do future research with and for adolescents, we must
Nested Social Systems That Contribute to Adolescents’ consider ethical responsibilities.
Well-Being Under Risky Circumstances
Community
Conducting Ethical and Privacy-
Preserving Research With and
School for Adolescents
Several challenges and ethical considerations surface
Peers
when conducting online safety and privacy-related
Family
research with adolescents. We must ensure that con-
ducting privacy-related research with adolescents does
Adolescents not violate their privacy. For example, power imbal-
ances related to informed consent, data collection,
and other aspects of the research process can surface
Ages <8–12
Ages 13–15
Ages 16–18
www.computer.org/security 61
INCLUSIVE PRIVACY AND SECURITY
4. Z. Iftikhar et al., “Designing parental monitoring and Co-designing for the ethical treatment and protection of
control technology: A systematic review,” in Proc. adolescents,” in Proc. ACM Hum.-Comput. Interaction, vol.
Human-Comput. Interaction–INTERACT 2021: 18th IFIP 4, 2021, pp. 1–46, doi: 10.1145/3432930.
TC 13 Int. Conf., Bari, Italy. Cham, Switzerland: Springer
International Publishing, 2021, pp. 676–700. Jinkyung Katie Park is a postdoctoral scholar in com-
5. P. J. Wisniewski, J. Vitak, and H. Hartikainen, “Privacy in puter science at Vanderbilt University, Nashville, TN
adolescence,” in Modern Socio Technical Perspectives on Pri- 37235 USA; she will join the School of Computing
vacy, B. P. Knijnenburg, X. Page, P. J. Wisniewski, H. R. at Clemson University as an assistant professor in
Lipford, N. Proferes, J. Romano, eds. Cham, Switzerland: fall 2024. Her research interests include human–
Springer International Publishing, 2022, pp. 315–336. computer interaction, adolescent online safety,
6. S. Smirnova, S. Livingstone, and M. Stoilova, Understanding of and human-centered artificial intelligence. Park
User Needs and Problems: A Rapid Evidence Review of Age Assur- received a Ph.D. in information science from Rut-
ance and Parental Controls. euConsent. Accessed: May 15, gers University. She is an active member of the
2024. [Online]. Available: https://eprints.lse.ac.uk/112559/ Association for Computing Machinery Special
7. A. K. Ghosh, K. Badillo-Urquiola, S. Guha, J. J. LaViola, Jr, and Interest Group on Computer–Human Interaction,
P. J. Wisniewski, “Safety vs. surveillance: What children have to iSchool, and Association for Information Sci-
say about mobile apps for parental control,” in Proc. CHI Conf. ence and Technology communities. Contact her at
Human Factors Comput. Syst. (CHI), 2018, pp. 1–14. jinkyung.park@vanderbilt.edu.
8. K. Davis et al., “Supporting Teens’ intentional social
media use through interaction design: An exploratory Mamtaj Akter is an incoming assistant professor at
proof-of-concept study,” in Proc. 22nd Annu. Conf. ACM the New York Institute of Technology, Manhattan,
Interaction Design Children Conf., 2023, pp. 322–334. Manhattan, NY 10023 USA. Her research interests
9. A. Razi et al., “Instagram data donation: A case study on include human–computer interaction, adolescent
collecting ecologically valid social media data for the pur- online safety, and usable privacy and security. Akter
pose of adolescent online risk detection,” in Proc. CHI received a Ph.D. in computer science from Vanderbilt
Conf. Human Factors Comput. Syst. Extended Abstract (CHI University. She is an active member of the Associa-
EA), 2022, pp. 1–9, doi: 10.1145/3491101.3503569. tion for Computing Machinery SIGCHI. Contact her
10. N. S. Ali, Z. Agha, N. Chatlani, J. Park, and P. J. Wisniewski, “A at mamtaj.akter@vanderbilt.edu.
case study on facilitating a long-term youth advisory board to
involve youth in adolescent online safety research,” in Proc. Pamela Wisniewski is an associate professor and a Flow-
CHI Conf. Human Factors Comput. Syst. Extended Abstract ers Family Chancellor Faculty Fellow of Computer
(CHI EA), 2024, pp. 1–8, doi: 10.1145/3613905.3637121. Science at Vanderbilt University, Nashville, TN 37235
11. Z. Agha et al., “Tricky vs. transparent: Towards an ecologically USA. Her research interests include human–com-
valid and safe approach for evaluating online safety nudges puter interaction, social computing, and adolescent
for teens,” in Proc. CHI Conf. Human Factors Comput. Syst. online safety. Wisniewski received a Ph.D. in com-
(CHI), 2024, pp. 1–20, doi: 10.1145/3613904.3642313. puter and information systems from the University of
12. M. Akter, A. J. Godfrey, J. Kropczynski, H. R. Lipford, and P. North Carolina at Charlotte. She is an Association for
J. Wisniewski, “From parental control to joint family over- Computing Machinery senior member. Contact her
sight: Can parents and teens manage mobile online safety at pam.wisniewski@vanderbilt.edu.
and privacy as equals?,” in Proc. ACM Human-Comput.
Interaction, vol. 6, 2022, pp. 1–28, doi: 10.1145/3512904. Karla Badillo-Urquiola is a Clare Boothe Luce Assis-
13. K. Badillo-Urquiola, Z. Agha, D. Abaquita, S. B. Harpin, tant Professor of Computer Science and Engineer-
and P. J. Wisniewski, “Towards a social ecological approach ing at the University of Notre Dame, Notre Dame, IN
to supporting caseworkers in promoting the online safety 46556 USA. Her current research interests include
of youth in foster care,” in Proc. ACM Hum.-Comput. Inter- human–computer interaction, social computing,
action, vol. 8, 2024, pp. 135–28, doi: 10.1145/3637412. adolescent online safety, and vulnerable/marginal-
14. D. Smahel et al. “EU Kids Online 2020: Survey results from ized populations. Badillo-Urquiola received a Ph.D. in
19 countries.” EU KIDS ONLINE. Accessed: May 15, 2024. modeling and simulation from the University of Cen-
[Online]. Available: https://eprints.lse.ac.uk/103294/ tral Florida. She is an active member of the Associa-
15. K. Badillo-Urquiola, Z. Shea, Z. Agha, I. Lediaeva, and tion for Computing Machinery SIGCHI community.
P. J. Wisniewski, “Conducting risky research with teens: Contact her at kbadillou@nd.edu.