Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Using word embeddings to probe sentiment associations of politically loaded terms in news and opinion articles from news media outlets

  • Research Article
  • Published:
Journal of Computational Social Science Aims and scope Submit manuscript

Abstract

This work describes an analysis of political associations in 27 million diachronic (1975–2019) news and opinion articles from 47 news media outlets popular in the United States. We use embedding models trained on individual outlets content to quantify outlet-specific latent associations between positive/negative sentiment words and terms loaded with political connotations such as those describing political orientation, party affiliation, names of influential politicians, and ideologically aligned public figures. We observe that both left- and right-leaning news media tend to associate positive sentiment words with terms used to refer to members of their own political in-group and negative sentiment words with terms used to denote members of their ideological outgroup. Outlets rated as centrist by humans display political associations that are often milder but similar in orientation to those of left-leaning news organizations. A weighted average of political associations by outlets’ readership volume hints that political associations embedded in left of center news outlets might have larger societal reach. A chronological analysis of political associations through time suggests that political sentiment polarization is increasing in both left- and right-leaning news media contents. Our approach for measuring sentiment associations of words denoting political orientation in outlet-specific embedding models correlates substantially with external human ratings of outlet ideological bias (r > 0.7). Yet, specific sentiment associations are sometimes multifaceted and challenging to interpret. Overall, our work signals the potential of machine learning models derived from news media language usage to quantify the ideological bias embedded in news outlet content.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Neuendorf, K. A. (2001). The Content Analysis Guidebook (1st ed.). SAGE Publications Inc.

    Google Scholar 

  2. Rozado, D. (2019). Using word embeddings to analyze how universities conceptualize ‘diversity’ in their online institutional presence. Society, 56(3), 256–266. https://doi.org/10.1007/s12115-019-00362-9

    Article  Google Scholar 

  3. Setting the Agenda: Mass Media and Public Opinion, 2 edition. Polity, 2014

  4. Rozado, D. (2020). Prejudice and victimization themes in New York times discourse: a chronological analysis. Academic Questions, 33(1), 89–100. https://doi.org/10.1007/s12129-019-09857-7

    Article  Google Scholar 

  5. Gorp, B. V. (2007). The constructionist approach to framing: bringing culture back in. The Journal of Communication, 57(1), 60–78. https://doi.org/10.1111/j.0021-9916.2007.00329.x

    Article  Google Scholar 

  6. Gadarian, S. K. (2010). The politics of threat: how terrorism news shapes foreign policy attitudes. The Journal of Political, 72(2), 469–483. https://doi.org/10.1017/S0022381609990910

    Article  Google Scholar 

  7. “Media Bias (Real and Perceived) and the Rise of Partisan Media,” Niskanen Center, Nov. 06, 2017. https://www.niskanencenter.org/media-bias-real-perceived-rise-partisan-media/ (accessed May 03, 2021)

  8. Grabe, M. E., Bucy, E. P. Image Bite Politics: News and the Visual Framing of Elections. Oxford University Press. Accessed: May 03, 2021. [Online]. https://www.oxford.universitypressscholarship.com/view/10.1093/acprof:oso/9780195372076.001.0001/acprof-9780195372076

  9. Groeling, T. (2008). Who’s the Fairest of them All? An empirical test for partisan bias on ABC, CBS, NBC, and Fox News. Presidential Studies Quarterly, 38(4), 631–657. https://doi.org/10.1111/j.1741-5705.2008.02668.x

    Article  Google Scholar 

  10. Schiffer, A. J. (2006). Assessing partisan bias in political news: the case(s) of local senate election coverage. Political Communication, 23(1), 23–39. https://doi.org/10.1080/10584600500476981

    Article  Google Scholar 

  11. Groseclose, T., & Milyo, J. (2005). A measure of media bias. The Quarterly Journal of Economics, 120(4), 1191–1237.

    Article  Google Scholar 

  12. Gasper, J. (2011). Shifting ideologies? Re-examining media bias. Quarterly Journal of Political Science, 6, 357–370. https://doi.org/10.1561/100.00010006

    Article  Google Scholar 

  13. Martin, G. J., & Yurukoglu, A. (2017). Bias in cable news: persuasion and polarization. The American Economic Review, 107(9), 2565–2599. https://doi.org/10.1257/aer.20160812

    Article  Google Scholar 

  14. Larcinese, V., Puglisi, R., & Snyder, J. M., Jr. (2007). Partisan bias in economic news: evidence on the agenda-setting behavior of US newspapers. National Bureau of Economic Research. https://doi.org/10.3386/w13378

    Article  Google Scholar 

  15. Waldman, P., & Devitt, J. (1998). Newspaper photographs and the 1996 presidential election: the question of bias. Journalism & Mass Communication Quarterly, 75(2), 302–311. https://doi.org/10.1177/107769909807500206

    Article  Google Scholar 

  16. Barrett, A. W., & Barrington, L. W. (2005). Bias in newspaper photograph selection. Political Research Quarterly, 58(4), 609–618. https://doi.org/10.1177/106591290505800408

    Article  Google Scholar 

  17. Kahn, K. F., & Kenney, P. (2002). The slant of the news: how editorial endorsements influence campaign coverage and citizens’ views of candidates. American Political Science Review., 96(2), 381–394. https://doi.org/10.1017/S0003055402000230

    Article  Google Scholar 

  18. Media Bias in Presidential Election Coverage 1948–2008: Evaluation via Formal Measurement. Accessed: May 03, 2021. [Online]. https://rowman.com/isbn/0739164740

  19. Niven, D. (2003). Objective evidence on media bias: newspaper coverage of congressional party switchers. Journalism & Mass Communication Quarterly, 80(2), 311–326. https://doi.org/10.1177/107769900308000206

    Article  Google Scholar 

  20. Dalal, S., Adlim, B., & Lesk, M. (2019). How to measure relative bias in media coverage? Significance, 16(5), 18–23. https://doi.org/10.1111/j.1740-9713.2019.01316.x

    Article  Google Scholar 

  21. Mitchell, A., Gottfried, J., Kiley, J., Matsa, K. E. “Political Polarization & Media Habits,” Pew Research Center’s Journalism Project, Oct. 21, 2014. https://www.journalism.org/2014/10/21/political-polarization-media-habits/ (accessed May 06, 2021)

  22. Eveland, W. P., & Shah, D. V. (2003). The impact of individual and interpersonal factors on perceived news media bias. Political Psychology, 24(1), 101–117. https://doi.org/10.1111/0162-895X.00318

    Article  Google Scholar 

  23. Baum, M. A., & Gussin, P. (2008). In the eye of the beholder: how information shortcuts shape individual perceptions of bias in the media. Quarterly Journal of Political Science, 3(1), 1–31. https://doi.org/10.1561/100.00007010

    Article  Google Scholar 

  24. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J. “Distributed Representations of Words and Phrases and their Compositionality,” in Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2013, pp. 3111–3119. Accessed: May 11, 2017. [Online]. http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf

  25. Mikolov, T., Yih, W., Zweig, G. “Linguistic Regularities in Continuous Space Word Representations,” in Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, Georgia, 2013, pp. 746–751. Accessed: Mar. 19, 2019. [Online]. http://aclweb.org/anthology/N13-1090

  26. Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186. https://doi.org/10.1126/science.aal4230

    Article  Google Scholar 

  27. Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018). Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16), E3635–E3644. https://doi.org/10.1073/pnas.1720347115

    Article  Google Scholar 

  28. Kozlowski, A. C., Taddy, M., & Evans, J. A. (2019). The geometry of culture: analyzing the meanings of class through word embeddings. American Sociological Review, 84(5), 905–949. https://doi.org/10.1177/0003122419877135

    Article  Google Scholar 

  29. “AllSides Media Bias Ratings,” AllSides. https://www.allsides.com/blog/updated-allsides-media-bias-chart-version-11 (accessed May 10, 2020)

  30. Rozado, D. (2020). Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types. PLoS ONE, 15(4), e0231189. https://doi.org/10.1371/journal.pone.0231189

    Article  Google Scholar 

  31. Notess, G. R. “The Wayback Machine: The Web’s Archive.,” Online, vol. 26, no. 2, 2002, Accessed: Mar. 14, 2021. [Online]. https://www.elibrary.ru/item.asp?id=4370136

  32. Mehmood, M. A., Shafiq, H. M., Waheed, A. “Understanding regional context of World Wide Web using common crawl corpus,” in 2017 IEEE 13th Malaysia International Conference on Communications (MICC), 2017, pp. 164–169. https://doi.org/10.1109/MICC.2017.8311752

  33. R. Řehůřek and P. Sojka, “Software Framework for Topic Modelling with Large Corpora,” in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, Valletta, Malta, May 2010, pp. 45–50.

  34. Gladkova, A., Drozd, A., Matsuoka, S. “Analogy-based detection of morphological and semantic relations with word embeddings: what works and what doesn’t.,” in Proceedings of the NAACL Student Research Workshop, San Diego, California, Jun. 2016, pp. 8–15. Accessed: Mar. 25, 2019. [Online]. http://www.aclweb.org/anthology/N16-2002

  35. Byers, D. “Twitter’s most influential political journalists,” POLITICO. https://www.politico.com/blogs/media/2015/04/twitters-most-influential-political-journalists-205510.html (accessed Aug. 02, 2020)

  36. Harnden, T. “The most influential US liberals: 20–1,” Jan. 15, 2010. Accessed: Aug. 02, 2020. [Online]. https://www.telegraph.co.uk/news/worldnews/northamerica/usa/6991000/The-most-influential-US-liberals-20-1.html

  37. Harnden, T. “The most influential US conservatives: 20–1,” Jan. 15, 2010. Accessed: Aug. 02, 2020. [Online]. https://www.telegraph.co.uk/news/worldnews/northamerica/usa/6990965/The-most-influential-US-conservatives-20-1.html

  38. Gunkel, P. “Ideonomy Primary Personality Traits.” http://ideonomy.mit.edu/essays/traits.html (accessed Aug. 08, 2020)

  39. “Similarweb.com—Digital World Market Intelligence Platform,” SimilarWeb.com. https://www.similarweb.com/ (accessed Nov. 19, 2020)

  40. Stone, P. J., Dunphy, D. C., Smith, M. S. (1966). The general inquirer: A computer approach to content analysis. M.I.T. Press, 1966. [Online]. http://wjh.harvard.edu/~inquirer/

  41. Enten, H. “How Roger Ailes Polarized TV News,” FiveThirtyEight, May 19, 2017. https://fivethirtyeight.com/features/how-roger-ailes-polarized-tv-news/ (accessed May 20, 2021)

  42. Mann, T. E. “Asymmetrical Polarization Undermined? Thoughts on the New Pew Research Center’s Report on Political Polarization,” Brookings, Nov. 30, 1AD. https://www.brookings.edu/blog/fixgov/2014/06/13/asymmetrical-polarization-undermined-thoughts-on-the-new-pew-research-centers-report-on-political-polarization/ (accessed May 20, 2021)

  43. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. https://doi.org/10.18653/v1/N19-1423

  44. G. Inc, “Americans’ Trust in Mass Media Sinks to New Low,” Gallup.com, Sep. 14, 2016. https://news.gallup.com/poll/195542/americans-trust-mass-media-sinks-new-low.aspx (accessed May 06, 2021)

  45. G. Inc, “Americans Remain Distrustful of Mass Media,” Gallup.com, Sep. 30, 2020. https://news.gallup.com/poll/321116/americans-remain-distrustful-mass-media.aspx (accessed May 06, 2021)

  46. al-Gharbi, M. “The New York Times’ obsession with Trump, quantified,” Columbia Journalism Review. https://www.cjr.org/covering_the_election/new-york-times-trump.php (accessed May 09, 2020)

  47. al-Gharbi, M. “Cable news profits from its obsession with Trump. Viewers are the only victims.,” Columbia Journalism Review. https://www.cjr.org/politics/cable-news-trump-obsession.php (accessed Mar. 22, 2021)

Download references

Author information

Authors and Affiliations

  1. School of Engineering, Construction and Living Sciences, Otago Polytechnic, Dunedin, New Zealand

    David Rozado

  2. Department of Sociology, Columbia University, New York, USA

    Musa al-Gharbi

Authors
  1. David Rozado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Musa al-Gharbi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Rozado.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 1329 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rozado, D., al-Gharbi, M. Using word embeddings to probe sentiment associations of politically loaded terms in news and opinion articles from news media outlets. J Comput Soc Sc 5, 427–448 (2022). https://doi.org/10.1007/s42001-021-00130-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42001-021-00130-y

Keywords

Search

Navigation