research-article

Framing the News: From Human Perception to Large Language Model Inferences

Authors:

David Alonso del Barrio,

Daniel Gatica-PerezAuthors Info & Claims

ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

Pages 627 - 635

https://doi.org/10.1145/3591106.3592278

Published: 12 June 2023 Publication History

Abstract

Identifying the frames of news is important to understand the articles’ vision, intention, message to be conveyed, and which aspects of the news are emphasized. Framing is a widely studied concept in journalism, and has emerged as a new topic in computing, with the potential to automate processes and facilitate the work of journalism professionals. In this paper, we study this issue with articles related to the Covid-19 anti-vaccine movement. First, to understand the perspectives used to treat this theme, we developed a protocol for human labeling of frames for 1786 headlines of No-Vax movement articles of European newspapers from 5 countries. Headlines are key units in the written press, and worth of analysis as many people only read headlines (or use them to guide their decision for further reading.) Second, considering advances in Natural Language Processing (NLP) with large language models, we investigated two approaches for frame inference of news headlines: first with a GPT-3.5 fine-tuning approach, and second with GPT-3.5 prompt-engineering. Our work contributes to the study and analysis of the performance that these models have to facilitate journalistic tasks like classification of frames, while understanding whether the models are able to replicate human perception in the identification of these frames.

References

[1]

Justito Adiprasetio and Annissa Winda Larasati. 2020. Pandemic crisis in online media: Quantitative framing analysis on Detik. com’s coverage of Covid-19. Jurnal Ilmu Sosial Dan Ilmu Politik 24, 2 (2020), 153–170.

[2]

Neel Alex, Eli Lifland, Lewis Tunstall, Abhishek Thakur, Pegah Maham, C Jess Riedel, Emmie Hine, Carolyn Ashurst, Paul Sedille, Alexis Carlier, 2021. RAFT: A real-world few-shot text classification benchmark. arXiv preprint arXiv:2109.14076 (2021).

[3]

David Alonso del Barrio and Daniel Gatica-Perez. 2022. How Did Europe’s Press Cover Covid-19 Vaccination News? A Five-Country Analysis. (2022), 35–43. https://doi.org/10.1145/3512732.3533588

Digital Library

[4]

Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? (2021), 610–623.

Digital Library

[5]

Santosh Kumar Biswal and Nikhil Kumar Gouda. 2020. Artificial intelligence in journalism: A boon or bane? In Optimization in machine learning and applications. Springer, 155–167.

[6]

Erik Bleich, Hannah Stonebraker, Hasher Nisar, and Rana Abdelhamid. 2015. Media portrayals of minorities: Muslims in British newspaper headlines, 2001–2012. Journal of Ethnic and Migration Studies 41, 6 (2015), 942–962.

[7]

Michael Bommarito and Daniel Martin Katz. 2022. GPT Takes the Bar Exam. https://doi.org/10.48550/ARXIV.2212.14402

[8]

Meredith Broussard, Nicholas Diakopoulos, Andrea L Guzman, Rediet Abebe, Michel Dupagne, and Ching-Hua Chuan. 2019. Artificial intelligence and journalism. Journalism & Mass Communication Quarterly 96, 3 (2019), 673–695.

[9]

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.

[10]

Björn Burscher, Daan Odijk, Rens Vliegenthart, Maarten De Rijke, and Claes H De Vreese. 2014. Teaching the computer to code frames in news: Comparing two supervised machine learning approaches to frame analysis. Communication Methods and Measures 8, 3 (2014), 190–206.

[11]

Bjorn Burscher, Rens Vliegenthart, and Claes H de Vreese. 2016. Frames beyond words: Applying cluster and sentiment analysis to news coverage of the nuclear power issue. Social Science Computer Review 34, 5 (2016), 530–545.

Digital Library

[12]

Dallas Card, Amber Boydstun, Justin Gross, Philip Resnik, and Noah Smith. 2015. The Media Frames Corpus: Annotations of Frames Across Issues. 2 (01 2015), 438–444. https://doi.org/10.3115/v1/P15-2072

[13]

Daniel Catalan-Matamoros and Carlos Elías. 2020. Vaccine hesitancy in the age of coronavirus and fake news: analysis of journalistic sources in the Spanish quality press. International Journal of Environmental Research and Public Health 17, 21 (2020), 8136.

[14]

Daniel Catalán-Matamoros and Carmen Peñafiel-Saiz. 2019. Media and mistrust of vaccines: a content analysis of press headlines. Revista latina de comunicación social74 (2019), 786–802.

[15]

Mark Coddington. 2015. Clarifying journalism’s quantitative turn: A typology for evaluating data journalism, computational journalism, and computer-assisted reporting. Digital journalism 3, 3 (2015), 331–348.

[16]

Stephen D Cooper. 2010. The oppositional framing of bloggers. In Doing News Framing Analysis. Routledge, 151–172.

[17]

Robert Dale. 2021. GPT-3: What’s it good for?Natural Language Engineering 27, 1 (2021), 113–118.

[18]

Astrid Dirikx and Dave Gelders. 2010. To frame is to explain: A deductive frame-analysis of Dutch and French climate change coverage during the annual UN Conferences of the Parties. Public Understanding of Science 19, 6 (2010), 732–742. https://doi.org/10.1177/0963662509352044 arXiv:https://doi.org/10.1177/0963662509352044PMID: 21560546.

[19]

Astrid Dirikx and Dave Gelders. 2010. To frame is to explain: A deductive frame-analysis of Dutch and French climate change coverage during the annual UN Conferences of the Parties. Public understanding of science 19, 6 (2010), 732–742.

[20]

Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. 2020. Gsum: A general framework for guided neural abstractive summarization. arXiv preprint arXiv:2010.08014 (2020).

[21]

Sumayya Ebrahim. 2022. The corona chronicles: Framing analysis of online news headlines of the COVID-19 pandemic in Italy, USA and South Africa. Health SA Gesondheid (Online) 27 (2022), 1–8.

[22]

Hend Abdelgaber Ahmed El-Behary. 2021. A Feverish Spring: A Comparative Analysis of COVID-19 News Framing in Sweden, the UK, and Egypt. (2021).

[23]

Robert M Entman. 1993. Framing: Towards clarification of a fractured paradigm. McQuail’s reader in mass communication theory 390 (1993), 397.

[24]

Tianyu Gao, Adam Fisch, and Danqi Chen. 2020. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723 (2020).

[25]

Piyush Ghasiya and Koji Okamura. 2021. Investigating COVID-19 news across four nations: a topic modeling and sentiment analysis approach. Ieee Access 9 (2021), 36645–36656.

[26]

Robert Gifford. 1994. A Lens-Mapping Framework for Understanding the Encoding and Decoding of Interpersonal Dispositions in Nonverbal Behavior. Journal of Personality and Social Psychology 66 (02 1994), 398–412. https://doi.org/10.1037//0022-3514.66.2.398

[27]

Quentin Grail, Julien Perez, and Eric Gaussier. 2021. Globalizing BERT-based transformer architectures for long document summarization. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 1792–1810.

[28]

Anushka Gupta, Diksha Chugh, Rahul Katarya, 2022. Automated news summarization using transformers. In Sustainable Advanced Computing. Springer, 249–259.

[29]

Alfred Hermida and Mary Lynn Young. 2017. Finding the data unicorn: A hierarchy of hybridity in data and computational journalism. Digital Journalism 5, 2 (2017), 159–176.

[30]

Karoliina Isoaho, Daria Gritsenko, and Eetu Mäkelä. 2021. Topic modeling and text analysis for qualitative policy research. Policy Studies Journal 49, 1 (2021), 300–324.

[31]

Carina Jacobi, Wouter Van Atteveldt, and Kasper Welbers. 2016. Quantitative analysis of large amounts of journalistic texts using topic modelling. Digital journalism 4, 1 (2016), 89–106.

[32]

Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know?Transactions of the Association for Computational Linguistics 8 (2020), 423–438.

[33]

Shima Khanehzar, Andrew Turpin, and Gosia Mikołajczak. 2019. Modeling Political Framing Across Policy Issues and Contexts. In ALTA.

[34]

Jeesun Kim and Wayne Wanta. 2018. News framing of the US immigration debate during election years: Focus on generic frames. The Communication Review 21, 2 (2018), 89–115.

[35]

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110 (2022).

[36]

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. (2021). https://doi.org/10.48550/ARXIV.2107.13586

[37]

Siyi Liu, Lei Guo, Kate Mays, Margrit Betke, and Derry Tanti Wijaya. 2019. Detecting frames in news headlines and its application to analyzing news framing trends surrounding US gun violence. In Proceedings of the 23rd conference on computational natural language learning (CoNLL).

[38]

Jörg Matthes and Matthias Kohring. 2008. The Content Analysis of Media Frames: Toward Improving Reliability and Validity. Journal of Communication 58 (06 2008). https://doi.org/10.1111/j.1460-2466.2008.00384.x

[39]

Selina Meyer, David Elsweiler, Bernd Ludwig, Marcos Fernandez-Pichel, and David E Losada. 2022. Do We Still Need Human Assessors? Prompt-Based GPT-3 User Simulation in Conversational AI. In Proceedings of the 4th Conference on Conversational User Interfaces. 1–6.

Digital Library

[40]

Stuart E Middleton, Symeon Papadopoulos, and Yiannis Kompatsiaris. 2018. Social computing for verifying social media content in breaking news. IEEE Internet Computing 22, 2 (2018), 83–89.

[41]

Marko Milosavljević and Igor Vobič. 2021. ‘Our task is to demystify fears’: Analysing newsroom management of automation in journalism. Journalism 22, 9 (2021), 2203–2221.

[42]

R. Monarch. 2021. Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-centered AI. Manning. https://books.google.ch/books?id=LCh0zQEACAAJ

[43]

Tom Nicholls and Pepper D Culpepper. 2021. Computational identification of media frames: Strengths, weaknesses, and opportunities. Political Communication 38, 1-2 (2021), 159–181.

[44]

Zhongdang Pan and Gerald M Kosicki. 1993. Framing analysis: An approach to news discourse. Political communication 10, 1 (1993), 55–75.

[45]

Raul Puri and Bryan Catanzaro. 2019. Zero-shot text classification with generative language models. arXiv preprint arXiv:1912.10165 (2019).

[46]

Guanghui Qin and Jason Eisner. 2021. Learning how to ask: Querying lms with mixtures of soft prompts. arXiv preprint arXiv:2104.06599 (2021).

[47]

Rabindra Lamsal. 2021. Sentiment Analysis of English Tweets with BERTsent. https://huggingface.co/rabindralamsal/finetuned-bertweet-sentiment-analysis.

[48]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.

[49]

Nishant Rai, Deepika Kumar, Naman Kaushik, Chandan Raj, and Ahad Ali. 2022. Fake News Classification using transformer based enhanced LSTM and BERT. International Journal of Cognitive Computing in Engineering 3 (2022), 98–105. https://doi.org/10.1016/j.ijcce.2022.03.003

[50]

Frida V Rodelo. 2021. Framing of the Covid-19 pandemic and its organizational predictors. Cuadernos. info50 (2021), 91–112.

[51]

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 (2022).

[52]

Holli Semetko and Patti Valkenburg. 2000. Framing European Politics: A Content Analysis of Press and Television News. Journal of Communication 50 (06 2000), 93 – 109. https://doi.org/10.1111/j.1460-2466.2000.tb02843.x

[53]

Richard Shin, Christopher H Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, and Benjamin Van Durme. 2021. Constrained language models yield few-shot semantic parsers. arXiv preprint arXiv:2104.08768 (2021).

[54]

Efstathios Sidiropoulos and Andreas Veglis. 2017. Computer Supported Collaborative Work trends on Media Organizations: Mixing Qualitative and Quantitative Approaches. Studies in Media and Communication 5 (04 2017), 63. https://doi.org/10.11114/smc.v5i1.2279

[55]

Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243 (2019).

[56]

Alex Tamkin, Miles Brundage, Jack Clark, and Deep Ganguli. 2021. Understanding the capabilities, limitations, and societal impact of large language models. arXiv preprint arXiv:2102.02503 (2021).

[57]

Trieu H Trinh and Quoc V Le. 2018. A simple method for commonsense reasoning. arXiv preprint arXiv:1806.02847 (2018).

[58]

Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. 2021. Multimodal few-shot learning with frozen language models. Advances in Neural Information Processing Systems 34 (2021), 200–212.

[59]

Sandra A Vannoy and Prashant Palvia. 2010. The social influence model of technology adoption. Commun. ACM 53, 6 (2010), 149–153.

Digital Library

[60]

Tuukka Ylä-Anttila, Veikko Eranti, and Anna Kukkonen. 2022. Topic modeling for frame analysis: A study of media debates on climate change in India and USA. Global Media and Communication 18, 1 (2022), 91–112.

Cited By

Castillo-Campos MVarona-Aramburu DBecerra-Alonso D(2024)Artificial Intelligence Tools and Bias in Journalism-related Content Generation: Comparison Between Chat GPT-3.5, GPT-4 and BingTripodos10.51698/tripodos.2024.55.06(06)Online publication date: 24-Jun-2024
https://doi.org/10.51698/tripodos.2024.55.06
Alonso del Barrio DTiel MGatica-Perez D(2024)Human Interest or Conflict? Leveraging LLMs for Automated Framing Analysis in TV ShowsProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3656308(157-167)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3639701.3656308
Sufi F(2024)Advanced Computational Methods for News Classification: A Study in Neural Networks and CNN integrated with GPTJournal of Economy and Technology10.1016/j.ject.2024.09.001Online publication date: Sep-2024
https://doi.org/10.1016/j.ject.2024.09.001
Show More Cited By

Index Terms

Framing the News: From Human Perception to Large Language Model Inferences
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Information extraction
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Text input

Recommendations

Human Interest or Conflict? Leveraging LLMs for Automated Framing Analysis in TV Shows
IMX '24: Proceedings of the 2024 ACM International Conference on Interactive Media Experiences

In the current media landscape, understanding the framing of information is crucial for critical consumption and informed decision making. Framing analysis is a valuable tool for identifying the underlying perspectives used to present information, and ...
Co-Writing with Opinionated Language Models Affects Users’ Views
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

If large language models like GPT-3 preferably produce a particular point of view, they may influence people’s opinions on an unknown scale. This study investigates whether a language-model-powered writing assistant that generates some opinions more ...
Mental Health News Coverage in Turkish National Newspapers with Respect to Episodic and Thematic Framing: A Retrospective Content Analysis
Computational Science and Its Applications – ICCSA 2024 Workshops
Abstract
News framing is a process by which news media highlight, emphasize and place prominence to a certain aspect of a news story. This process is comprised of constructing news narratives to shape the perceptions and attitudes of the audience within ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

June 2023

694 pages

ISBN:9798400701788

DOI:10.1145/3591106

Editors:
Ioannis (Yiannis) Kompatsiaris
Centre for Research and Technology Hellas, Greece
,
Jiebo Luo
University of Rochester,USA
,
Nicu Sebe
University of Trento, Italy
,
Angela Yao
National University of Singapore, Singapore
,
Vasileios Mezaris
Centre for Research and Technology Hellas, Greece
,
Symeon Papadopoulos
Centre for Research and Technology Hellas, Greece
,
Adrian Popescu
CEA LIST, France
,
Zi (Helen) Huang
University of Queensland, Australia

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

European Commission

Conference

ICMR '23

Sponsor:

SIGMM

ICMR '23: International Conference on Multimedia Retrieval

June 12 - 15, 2023

Thessaloniki, Greece

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
251
Total Downloads

Downloads (Last 12 months)157
Downloads (Last 6 weeks)20

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Castillo-Campos MVarona-Aramburu DBecerra-Alonso D(2024)Artificial Intelligence Tools and Bias in Journalism-related Content Generation: Comparison Between Chat GPT-3.5, GPT-4 and BingTripodos10.51698/tripodos.2024.55.06(06)Online publication date: 24-Jun-2024
https://doi.org/10.51698/tripodos.2024.55.06
Alonso del Barrio DTiel MGatica-Perez D(2024)Human Interest or Conflict? Leveraging LLMs for Automated Framing Analysis in TV ShowsProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3656308(157-167)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3639701.3656308
Sufi F(2024)Advanced Computational Methods for News Classification: A Study in Neural Networks and CNN integrated with GPTJournal of Economy and Technology10.1016/j.ject.2024.09.001Online publication date: Sep-2024
https://doi.org/10.1016/j.ject.2024.09.001
Li JLi JSu Y(2024)A Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and CreativityArtificial Intelligence in HCI10.1007/978-3-031-60615-1_5(60-85)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-60615-1_5
Kille BLommatzsch AÖzgöbek ÖLiu PEide SZhang L(2023)The Eleventh International Workshop on News Recommendation and Analytics (INRA’23)Proceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608760(1263-1266)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608760

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents