GenAIPABench: A Benchmark for Generative AI-based Privacy Assistants

2016

Privacy policies written in natural language are the predominant method that operators of websites and online services use to communicate privacy practices to their users. However, these documents are infrequently read by Internet users, due in part to the length and complexity of the text. These factors also inhibit the efforts of regulators to assess privacy practices or to enforce standards. One proposed approach to improving the status quo is to use a combination of methods from crowdsourcing, natural language processing, and machine learning to extract details from privacy policies and present them in an understandable fashion. We sketch out this vision and describe our ongoing work to bring it to fruition. Further, we discuss challenges associated with bridging the gap between the contents of privacy policy text and website users’ abilities to understand those policies. These challenges are motivated by the rich interconnectedness of the problems as well as the broader impact ...

Findings of the Association for Computational Linguistics: EMNLP 2020

Proceedings of the 25th International Conference on World Wide Web, 2016

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016

Proceedings of the SIGCHI conference on Human Factors in computing systems - CHI '06, 2006

ArXiv, 2021

Understanding privacy policies is crucial for users as it empowers them to learn about the information that matters to them. Sentences written in a privacy policy document explain privacy practices, and the constituent text spans convey further specific information about that practice. We refer to predicting the privacy practice explained in a sentence as intent classification and identifying the text spans sharing specific information as slot filling. In this work, we propose PolicyIE, an English corpus consisting of 5,250 intent and 11,788 slot annotations spanning 31 privacy policies of websites and mobile applications. PolicyIE corpus is a challenging real-world benchmark with limited labeled examples reflecting the cost of collecting large-scale annotations from domain experts. We present two alternative neural approaches as baselines, (1) intent classification and slot filling as a joint sequence tagging and (2) modeling them as a sequence-tosequence (Seq2Seq) learning task. T...

Benchmarks for general language understanding have been rapidly developing in recent years of NLP research, particularly because of their utility in choosing strong-performing models for practical downstream applications. While benchmarks have been proposed in the legal language domain, virtually no such benchmarks exist for privacy policies despite their increasing importance in modern digital life. This could be explained by privacy policies falling under the legal language domain, but we find evidence to the contrary that motivates a separate benchmark for privacy policies. Consequently, we propose PrivacyGLUE as the first comprehensive benchmark of relevant and high-quality privacy tasks for measuring general language understanding in the privacy language domain. Furthermore, we release performances from multiple transformer language models and perform model-pair agreement analysis to detect tasks where models benefited from domain specialization. Our findings show the importanc...

2017

Privacy policies are the locus where consequences concerning privacy and personal data are produced, but content features explain why they are largely ignored by its addressees. To abridge users with policies, we propose a policybased system that identifies potential pitfalls in the privacy policies of companies on the Web. It will then suggest clarification of terms by suggesting removal or replacement of defective terms, in order to foster accountable policymaking and compliance. The proposed methods are based on extracting knowledge from natural language texts of a small sample size, and on semantic representations of the policy expression.

2020 IEEE 28th International Requirements Engineering Conference (RE), 2020

https://youtu.be/JPRyHL846us, 2024

Roteiro elaborado para as classes de Teoria Musical e Harmonia ministradas nos cursos de Música da Udesc. Correlacionando questões de harmonia, melodia e forma, o vídeo procura reproduzir a prática em sala de aula. Em comemoração aos 150 anos de nascimento de Arnold Schoenberg" (setembro de 1874), os comentários guardam um tom bastante schoenberguiano e sugerem uma improvável conversa (que, pelas datas, bem poderia ter acontecido) entre aquele famoso professor austríaco e o nosso mestre Pixinguinha (de 1897). Recomenda-se assistir em tela grande.