research-article

Open access

Understanding Challenges for Developers to Create Accurate Privacy Nutrition Labels

Authors:

Yuvraj Agarwal,

Lorrie Faith Cranor,

Jason I. HongAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Article No.: 588, Pages 1 - 24

https://doi.org/10.1145/3491102.3502012

Published: 29 April 2022 Publication History

All formats PDF

Abstract

Apple announced the introduction of app privacy details to their App Store in December 2020, marking the first ever real-world, large-scale deployment of the privacy nutrition label concept, which had been introduced by researchers over a decade earlier. The Apple labels are created by app developers, who self-report their app’s data practices. In this paper, we present the first study examining the usability and understandability of Apple’s privacy nutrition label creation process from the developer’s perspective. By observing and interviewing 12 iOS app developers about how they created the privacy label for a real-world app that they developed, we identified common challenges for correctly and efficiently creating privacy labels. We discuss design implications both for improving Apple’s privacy label design and for future deployment of other standardized privacy notices.

References

[1]

Yasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim, Michelle L. Mazurek, and Christian Stransky. 2016. You Get Where You're Looking for: The Impact of Information Sources on Code Security. In 2016 IEEE Symposium on Security and Privacy (SP). IEEE. https://doi.org/10.1109/sp.2016.25

[2]

Yuvraj Agarwal and Malcolm Hall. 2013. ProtectMyPrivacy: detecting and mitigating privacy leaks on iOS devices using crowdsourcing. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services - MobiSys '13. ACM Press. https://doi.org/10.1145/2462456.2464460

Digital Library

[3]

Nitin Agrawal, Reuben Binns, Max Van Kleek, Kim Laine, and Nigel Shadbolt. 2021. Exploring Design and Governance Challenges in the Development of Privacy-Preserving Computation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM. https://doi.org/10.1145/3411764.3445677

Digital Library

[4]

Rebecca Balebako and Lorrie Cranor. 2014. Improving App Privacy: Nudging App Developers to Protect User Privacy. IEEE Security & Privacy 12, 4 (jul 2014), 55–58. https://doi.org/10.1109/msp.2014.70

[5]

Rebecca Balebako, Abigail Marsh, Jialiu Lin, Jason Hong, and Lorrie Faith Cranor. 2014. The Privacy and Security Behaviors of Smartphone App Developers. In Proceedings 2014 Workshop on Usable Security. Internet Society. https://doi.org/10.14722/usec.2014.23006

[6]

Rebecca Balebako, Florian Schaub, Idris Adjerid, Alessandro Acquisti, and Lorrie Cranor. 2015. The Impact of Timing on the Salience of Smartphone App Privacy Notices. In Proceedings of the 5th Annual ACM CCS Workshop on Security and Privacy in Smartphones and Mobile Devices. ACM. https://doi.org/10.1145/2808117.2808119

Digital Library

[7]

Rebecca Balebako, Richard Shay, and Lorrie Faith Cranor. 2014. Is Your Inseam a Biometric? A Case Study on the Role of Usability Studies in Developing Public Policy. In Proceedings 2014 Workshop on Usable Security. Internet Society. https://doi.org/10.14722/usec.2014.23039

[8]

Kai Chen, Xueqiang Wang, Yi Chen, Peng Wang, Yeonjoon Lee, XiaoFeng Wang, Bin Ma, Aohui Wang, Yingjun Zhang, and Wei Zou. 2016. Following Devil's Footprints: Cross-Platform Analysis of Potentially Harmful Libraries on Android and iOS. In 2016 IEEE Symposium on Security and Privacy (SP). IEEE. https://doi.org/10.1109/sp.2016.29

[9]

Saksham Chitkara, Nishad Gothoskar, Suhas Harish, Jason I. Hong, and Yuvraj Agarwal. 2017. Does this App Really Need My Location?: Context-Aware Privacy Management for Smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (sep 2017), 1–22. https://doi.org/10.1145/3132029

Digital Library

[10]

Lorrie Faith Cranor. 2012. Necessary but not sufficient: Standardized mechanisms for privacy notice and choice. J. on Telecomm. & High Tech. L. 10 (2012), 273.

[11]

Manuel Egele, Christopher Kruegel, Engin Kirda, and Giovanni Vigna. 2011. PiOS: Detecting Privacy Leaks in iOS Applications. In NDSS. 177–183.

[12]

Serge Egelman, Janice Tsai, Lorrie Faith Cranor, and Alessandro Acquisti. 2009. Timing is everything? The effects of timing and placement of online privacy indicators. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 319–328. https://doi.org/10.1145/1518701.1518752

Digital Library

[13]

Pardis Emami-Naeini, Yuvraj Agarwal, Lorrie Faith Cranor, and Hanan Hibshi. 2020. Ask the Experts: What Should Be on an IoT Privacy and Security Label?. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE. https://doi.org/10.1109/sp40000.2020.00043

[14]

Daniel Greene and Katie Shilton. 2017. Platform privacies: Governance, collaboration, and the different meanings of “privacy” in iOS and Android development. New Media & Society 20, 4 (apr 2017), 1640–1657. https://doi.org/10.1177/1461444817702397

[15]

Irit Hadar, Tomer Hasson, Oshrat Ayalon, Eran Toch, Michael Birnhack, Sofia Sherman, and Arod Balissa. 2017. Privacy by designers: software developers’ privacy mindset. Empirical Software Engineering 23, 1 (apr 2017), 259–289. https://doi.org/10.1007/s10664-017-9517-1

Digital Library

[16]

Carlos Jensen and Colin Potts. 2004. Privacy policies as decision-making tools: an evaluation of online privacy notices. In Proceedings of the 2004 conference on Human factors in computing systems - CHI '04. ACM Press. https://doi.org/10.1145/985692.985752

Digital Library

[17]

Patrick Gage Kelley, Joanna Bresee, Lorrie Faith Cranor, and Robert W. Reeder. 2009. A ”nutrition label” for privacy. In Proceedings of the 5th Symposium on Usable Privacy and Security - SOUPS '09. ACM Press. https://doi.org/10.1145/1572532.1572538

Digital Library

[18]

Patrick Gage Kelley, Lucian Cesca, Joanna Bresee, and Lorrie Faith Cranor. 2010. Standardizing privacy notices: an online study of the nutrition label approach. In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. ACM Press. https://doi.org/10.1145/1753326.1753561

Digital Library

[19]

Patrick Gage Kelley, Lorrie Faith Cranor, and Norman Sadeh. 2013. Privacy as part of the app decision-making process. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM. https://doi.org/10.1145/2470654.2466466

Digital Library

[20]

Tianshi Li, Yuvraj Agarwal, and Jason I. Hong. 2018. Coconut: An IDE Plugin for Developing Privacy-Friendly Apps. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (dec 2018), 1–35. https://doi.org/10.1145/3287056

Digital Library

[21]

Tianshi Li, Elizabeth Louie, Laura Dabbish, and Jason I. Hong. 2021. How Developers Talk About Personal Data and What It Means for User Privacy: A Case Study of a Developer Forum on Reddit. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3 (jan 2021), 1–28. https://doi.org/10.1145/3432919

Digital Library

[22]

Tianshi Li, Elijah B. Neundorfer, Yuvraj Agarwal, and Jason I. Hong. 2021. Honeysuckle: Annotation-Guided Code Generation of In-App Privacy Notices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (sep 2021), 1–27. https://doi.org/10.1145/3478097

Digital Library

[23]

Jialiu Lin, Norman Sadeh, Shahriyar Amini, Janne Lindqvist, Jason I. Hong, and Joy Zhang. 2012. Expectation and purpose: understanding users’ mental models of mobile app privacy through crowdsourcing. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing - UbiComp '12. ACM Press. https://doi.org/10.1145/2370216.2370290

Digital Library

[24]

Aleecia M McDonald and Lorrie Faith Cranor. 2008. The cost of reading privacy policies. I/S: A Journal of Law and Policy for the Information Society 4 (2008), 543.

[25]

Aleecia M. McDonald, Robert W. Reeder, Patrick Gage Kelley, and Lorrie Faith Cranor. 2009. A comparative study of online privacy policies and formats. In Proceedings of the 5th Symposium on Usable Privacy and Security - SOUPS '09. ACM Press. https://doi.org/10.1145/1572532.1572586

Digital Library

[26]

Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and Inter-rater Reliability in Qualitative Research: Norms and Guidelines for CSCW and HCI Practice. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (nov 2019), 1–23. https://doi.org/10.1145/3359174

Digital Library

[27]

Abraham H. Mhaidli, Yixin Zou, and Florian Schaub. 2019. ”We Can’t Live without Them!” App Developers’ Adoption of Ad Networks and Their Considerations of Consumer Risks(SOUPS’19). USENIX Association, USA, 225–244.

[28]

Victor Morel and Raúl Pardo. 2020. SoK: Three Facets of Privacy Policies. In Proceedings of the 19th Workshop on Privacy in the Electronic Society. 41–56.

Digital Library

[29]

Duc Cuong Nguyen, Dominik Wermke, Yasemin Acar, Michael Backes, Charles Weir, and Sascha Fahl. 2017. A Stitch in Time: Supporting Android Developers in WritingSecure Code. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM. https://doi.org/10.1145/3133956.3133977

Digital Library

[30]

Johnny Saldaña. 2015. The coding manual for qualitative researchers. Sage.

[31]

Florian Schaub, Rebecca Balebako, Adam L. Durity, and Lorrie Faith Cranor. 2015. A Design Space for Effective Privacy Notices. In Eleventh Symposium On Usable Privacy and Security (SOUPS 2015). USENIX Association, Ottawa, 1–17. https://www.usenix.org/conference/soups2015/proceedings/presentation/schaub

[32]

Awanthika Senarath and Nalin A. G. Arachchilage. 2018. Why developers cannot embed privacy into software systems?: An empirical investigation. In Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018. ACM. https://doi.org/10.1145/3210459.3210484

Digital Library

[33]

Swapneel Sheth, Gail Kaiser, and Walid Maalej. 2014. Us and them: a study of privacy requirements across north america, asia, and europe. In Proceedings of the 36th International Conference on Software Engineering. ACM. https://doi.org/10.1145/2568225.2568244

Digital Library

[34]

FTC Staff. 2011. Protecting Consumer Privacy in an Era of Rapid Change–A Proposed Framework for Businesses and Policymakers. Journal of Privacy and Confidentiality (jun 2011). https://doi.org/10.29012/jpc.v3i1.596

[35]

Mohammad Tahaei, Tianshi Li, and Kami Vaniea. 2022. Understanding Privacy-Related Advice on Stack Overflow. In Proceedings on Privacy Enhancing Technologies. 1––18. https://doi.org/10.2478/popets-2022-0032

[36]

Mohammad Tahaei, Kami Vaniea, and Naomi Saphra. 2020. Understanding Privacy-Related Questions on Stack Overflow. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376768

Digital Library

[37]

Sebastian Zimmeck, Rafael Goldstein, and David Baraka. 2021. PrivacyFlash Pro: Automating Privacy Policy Generation for Mobile Apps. In Proceedings 2021 Network and Distributed System Security Symposium. Internet Society. https://doi.org/10.14722/ndss.2021.24100

Cited By

Pan SZhang DStaples MXing ZChen JXu XHoang TBalzarotti DXu W(2024)Is it a trap? a large-scale empirical study and comprehensive assessment of online automated privacy policy generators for mobile appsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699218(5681-5698)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699218
Liu DXiao YZhang CXie KBai XZhang SXing LBalzarotti DXu W(2024)iHunterProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699217(5663-5680)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699217
Mandal PAmi AOlaiya VRazmjo SNadkarni ABalzarotti DXu W(2024)"Belt and suspenders" or "just red tape"?Proceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699176(4927-4944)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699176
Show More Cited By

Index Terms

Understanding Challenges for Developers to Create Accurate Privacy Nutrition Labels
1. Security and privacy
  1. Human and societal aspects of security and privacy
    1. Usability in security and privacy
2. Software and its engineering

Recommendations

Understanding iOS Privacy Nutrition Labels: An Exploratory Large-Scale Analysis of App Store Data
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

Since December 2020, the Apple App Store has required all developers to create a privacy label when submitting new apps or app updates. However, there has not been a comprehensive study on how developers responded to this requirement. We present the ...
Goodbye Tracking? Impact of iOS App Tracking Transparency and Privacy Labels
FAccT '22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency

Tracking is a highly privacy-invasive data collection practice that has been ubiquitous in mobile apps for many years due to its role in supporting advertising-based revenue models. In response, Apple introduced two significant changes with iOS 14: App ...
Privacy Capsules: Preventing Information Leaks by Mobile Apps
MobiSys '16: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services

Preventing the leakage of user information via untrusted third-party apps is a key challenge in mobile privacy. We propose and evaluate privacy capsules (PCs), a platform execution model for mobile apps that prevents the flow of private information to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Air Force Research Laboratory
National Science Foundation

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
4,871
Total Downloads

Downloads (Last 12 months)2,404
Downloads (Last 6 weeks)162

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pan SZhang DStaples MXing ZChen JXu XHoang TBalzarotti DXu W(2024)Is it a trap? a large-scale empirical study and comprehensive assessment of online automated privacy policy generators for mobile appsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699218(5681-5698)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699218
Liu DXiao YZhang CXie KBai XZhang SXing LBalzarotti DXu W(2024)iHunterProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699217(5663-5680)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699217
Mandal PAmi AOlaiya VRazmjo SNadkarni ABalzarotti DXu W(2024)"Belt and suspenders" or "just red tape"?Proceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699176(4927-4944)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699176
Lee HGao LYang SForlizzi JDas SBalzarotti DXu W(2024)"I don't know if we're doing good. I don't know if we're doing bad": investigating how practitioners scope, motivate, and conduct privacy work when developing AI productsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699173(4873-4890)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699173
Khandelwal RNayak AChung PFawaz KBalzarotti DXu W(2024)Unpacking privacy labelsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699059(2831-2848)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699059
Balash DAli MKanich CAviv AKelley PKapadia A(2024)"I would not install an app with this label"Proceedings of the Twentieth USENIX Conference on Usable Privacy and Security10.5555/3696899.3696921(413-432)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.5555/3696899.3696921
Gopi GMaddi AArasaratnam OFanti GKelley PKapadia A(2024)Privacy requirements and realities of digital public goodsProceedings of the Twentieth USENIX Conference on Usable Privacy and Security10.5555/3696899.3696908(159-177)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.5555/3696899.3696908
Zhang SKlucinec LNorton KSadeh NCranor LKelley PKapadia A(2024)Exploring expandable-grid designs to make iOS app privacy labels more usableProceedings of the Twentieth USENIX Conference on Usable Privacy and Security10.5555/3696899.3696907(139-157)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.5555/3696899.3696907
Levi-Eshkol KRibak R(2024)“Track every move”: Analyzing developers’ privacy discourse in GitHub README filesNew Media & Society10.1177/14614448241270541Online publication date: 28-Aug-2024
https://doi.org/10.1177/14614448241270541
Xiao YNadkarni ALiao XTorres-Arias SDe Carli LZhang Y(2024)Enhancing Transparency and Accountability of TPLs with PBOM: A Privacy Bill of MaterialsProceedings of the 2024 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses10.1145/3689944.3696159(1-11)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3689944.3696159
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten