research-article

An Empirical Investigation of Personalization Factors on TikTok

Authors:

Maximilian Boeker,

Aleksandra UrmanAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 2298 - 2309

https://doi.org/10.1145/3485447.3512102

Published: 25 April 2022 Publication History

Abstract

TikTok currently is the fastest growing social media platform with over 1 billion active monthly users of which the majority is from generation Z. Arguably, its most important success driver is its recommendation system. Despite the importance of TikTok’s algorithm to the platform’s success and content distribution, little work has been done on the empirical analysis of the algorithm. Our work lays the foundation to fill this research gap. Using a sock-puppet audit methodology with a custom algorithm developed by us, we tested and analysed the effect of the language and location used to access TikTok, follow- and like-feature, as well as how the recommended content changes as a user watches certain posts longer than others. We provide evidence that all the tested factors influence the content recommended to TikTok users. Further, we identified that the follow-feature has the strongest influence, followed by the like-feature and video view rate. We also discuss the implications of our findings in the context of the formation of filter bubbles on TikTok and the proliferation of problematic content.

References

[1]

2020. Terms of Service | TikTok. https://www.tiktok.com/legal/terms-of-service?lang=en#terms-eea

[2]

Gediminas Adomavicius, Jesse Bockstedt, Shawn P Curley, Jingjing Zhang, and Sam Ransbotham. 2019. The hidden side effects of recommendation systems. MIT Sloan Management Review 60, 2 (2019), 1.

[3]

Oscar Alvarado, Hendrik Heuer, Vero Vanden Abeele, Andreas Breiter, and Katrien Verbert. 2020. Middle-Aged Video Consumers’ Beliefs About Algorithmic Recommendations on YouTube. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2(2020), 1–24.

Digital Library

[4]

Katie Anderson. 2020. Getting acquainted with social networks and apps: it is time to talk about TikTok. Library Hi Tech News ahead-of-print (02 2020). https://doi.org/10.1108/LHTN-01-2020-0001

[5]

Salman Aslam. 2021. TikTok by the Numbers: Stats, Demographics & Fun Facts. https://www.omnicoreagency.com/tiktok-statistics/

[6]

Ricardo Baeza-Yates. 2020. Bias in Search and Recommender Systems. In Fourteenth ACM Conference on Recommender Systems(Virtual Event, Brazil) (RecSys ’20). Association for Computing Machinery, New York, NY, USA, 2. https://doi.org/10.1145/3383313.3418435

Digital Library

[7]

Jack Bandy. 2021. Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–34.

Digital Library

[8]

Jack Bandy and Nicholas Diakopoulos. 2020. # TulsaFlop: A Case Study of Algorithmically-Influenced Collective Action on TikTok. arXiv preprint arXiv:2012.07716(2020).

[9]

Jack Bandy and Nicholas Diakopoulos. 2021. More Accounts, Fewer Links: How Algorithmic Curation Impacts Media Exposure in Twitter Timelines. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–28.

Digital Library

[10]

Corey H Basch, Grace C Hillyer, and Christie Jaime. 2020. COVID-19 on TikTok: harnessing an emerging social media platform to convey important public health messages. International journal of adolescent medicine and health (2020).

[11]

BBC. 2021. TikTok named as the most downloaded app of 2020. https://www.bbc.com/news/business-58155103

[12]

Zhuang Chen, Qian He, Zhifei Mao, Hwei-Ming Chung, and Sabita Maharjan. 2019. A study on the characteristics of douyin short videos and implications for edge caching. In Proceedings of the ACM Turing Celebration Conference-China. 1–6.

Digital Library

[13]

Patricio Domingues, Ruben Nogueira, José Carlos Francisco, and Miguel Frade. 2020. Post-Mortem Digital Forensic Artifacts of TikTok Android App. In Proceedings of the 15th International Conference on Availability, Reliability and Security (Virtual Event, Ireland) (ARES ’20). Association for Computing Machinery, New York, NY, USA, Article 42, 8 pages. https://doi.org/10.1145/3407023.3409203

Digital Library

[14]

Douyin. 2019. Douyin Official Data Report. https://static1.squarespace.com/static/5ac136ed12b13f7c187bdf21/t/5e13ba8db3528b5c1d4fada0/1578351246398/douyin+data+report.pdf

[15]

Facebook. [n.d.]. How News Feed Works. https://www.facebook.com/help/1155510281178725/?helpref=hc_fnav

[16]

Martin Feuz, Matthew Fuller, and Felix Stalder. 2011. Personal Web searching in the age of semantic capitalism: Diagnosing the mechanisms of personalisation. First Monday 16, 2 (Feb. 2011). https://doi.org/10.5210/fm.v16i2.3344

[17]

Mario Haim, Andreas Graefe, and Hans-Bernd Brosius. 2018. Burst of the Filter Bubble?Digital Journalism 6, 3 (2018), 330–343. https://doi.org/10.1080/21670811.2017.1338145 arXiv:https://doi.org/10.1080/21670811.2017.1338145

[18]

Aniko Hannak, Piotr Sapiezynski, Arash Molavi Kakhki, Balachander Krishnamurthy, David Lazer, Alan Mislove, and Christo Wilson. 2013. Measuring Personalization of Web Search. In Proceedings of the 22nd International Conference on World Wide Web (Rio de Janeiro, Brazil) (WWW ’13). Association for Computing Machinery, New York, NY, USA, 527–538. https://doi.org/10.1145/2488388.2488435

Digital Library

[19]

Aniko Hannak, Gary Soeller, David Lazer, Alan Mislove, and Christo Wilson. 2014. Measuring price discrimination and steering on e-commerce web sites. In Proceedings of the 2014 conference on internet measurement conference. 305–318.

Digital Library

[20]

Aniko Hannak, Gary Soeller, David Lazer, Alan Mislove, and Christo Wilson. 2014. Measuring Price Discrimination and Steering on E-Commerce Web Sites. In Proceedings of the 2014 Conference on Internet Measurement Conference (Vancouver, BC, Canada) (IMC ’14). Association for Computing Machinery, New York, NY, USA, 305–318. https://doi.org/10.1145/2663716.2663744

Digital Library

[21]

YouTube Help. [n.d.]. About video ad metrics and reporting. https://support.google.com/youtube/answer/2375431?hl=en

[22]

Hendrik Heuer. 2020. Users & Machine Learning-Based Curation Systems. Ph.D. Dissertation. Universität Bremen.

[23]

Jeff Horowitz and Deepa Seetharaman. 2020. Facebook Executives Shut Down Efforts to Make the Site Less Divisive. https://www.wsj.com/articles/facebook-knows-it-encourages-division-top-executives-nixed-solutions-11590507499

[24]

Philip N Howard and Bence Kollanyi. 2016. Bots,# strongerin, and# brexit: Computational propaganda during the uk-eu referendum. Available at SSRN 2798311(2016).

[25]

Eslam Hussein, Prerna Juneja, and Tanushree Mitra. 2020. Measuring Misinformation in Video Search Platforms: An Audit Study on YouTube. Proc. ACM Hum.-Comput. Interact. 4, CSCW1, Article 048 (May 2020), 27 pages. https://doi.org/10.1145/3392854

Digital Library

[26]

Mansoor Iqbal. 2021. TikTok Revenue and Usage Statistics (2021). https://www.businessofapps.com/data/tik-tok-statistics/

[27]

Wall Street Journal. 2021. Investigation: How TikTok’s Algorithm Figures Out Your Deepest Desires. https://www.wsj.com/video/series/inside-tiktoks-highly-secretive-algorithm/investigation-how-tiktok-algorithm-figures-out-your-deepest-desires/6C0C2040-FF25-4827-8528-2BD6612E3796

[28]

Chloe Kliman-Silver, Aniko Hannak, David Lazer, Christo Wilson, and Alan Mislove. 2015. Location, location, location: The impact of geolocation on web search personalization. In Proceedings of the 2015 internet measurement conference. 121–127.

Digital Library

[29]

Louis Klimek. 2021. 12 Ways to hide your Bot Automation from Detection | How to make Selenium undetectable and stealth. https://piprogramming.org/articles/How-to-make-Selenium-undetectable-and-stealth--7-Ways-to-hide-your-Bot-Automation-from-Detection-0000000017.html

[30]

Daniel Klug, Yiluo Qin, Morgan Evans, and Geoff Kaufman. 2021. Trick and Please. A Mixed-Method Study On User Assumptions About the TikTok Algorithm. In 13th ACM Web Science Conference 2021. 84–92.

Digital Library

[31]

Mykola Makhortykh, Aleksandra Urman, and Ulloa Roberto. 2020. How search engines disseminate information about COVID-19 and why they should do better. The Harvard Kennedy School (HKS) Misinformation Review 1 (2020).

[32]

Louise Matsakis. 2020. TikTok Finally Explains How the ‘For You’ Algorithm Works. https://www.wired.com/story/tiktok-finally-explains-for-you-algorithm-works/

[33]

Christian Matt, Alexander Benlian, Thomas Hess, and Christian Weiß. 2014. Escaping from the filter bubble? The effects of novelty and serendipity on users’ evaluations of online recommendations. (2014).

[34]

Juan Carlos Medina Serrano, Orestis Papakyriakopoulos, and Simon Hegelich. 2020. Dancing to the Partisan Beat: A First Analysis of Political Communication on TikTok. In 12th ACM Conference on Web Science (Southampton, United Kingdom) (WebSci ’20). Association for Computing Machinery, New York, NY, USA, 257–266. https://doi.org/10.1145/3394231.3397916

Digital Library

[35]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781(2013).

[36]

Brent Mittelstadt. 2016. Automation, algorithms, and politics| Auditing for transparency in content personalization systems. International Journal of Communication 10 (2016), 12.

[37]

Maryam Mohsin. 2021. 10 TIKTOK STATISTICS THAT YOU NEED TO KNOW IN 2021 [INFOGRAPHIC]. https://www.oberlo.com/blog/tiktok-statistics

[38]

Philip M Napoli. 2018. What Social Media Platforms Can Learn from Audience Measurement: Lessons in the Self-Regulation of’Black Boxes’. TPRC.

[39]

Casey Newton. 2017. How YouTube Perfected The Feed. https://www.theverge.com/2017/8/30/16222850/youtube-google-brain-algorithm-video-recommendation-personalized-feed

[40]

Marije Nouwen and Mathilde Hermine Christine Marie Ghislaine Duflos. 2021. TikTok as a Data Gathering Space: The Case of Grandchildren and Grandparents during the COVID-19 Pandemic. In Interaction Design and Children (Athens, Greece) (IDC ’21). Association for Computing Machinery, New York, NY, USA, 498–502. https://doi.org/10.1145/3459990.3465201

Digital Library

[41]

TikTok Blog Post. 2020. How TikTok recommends videos #ForYou. https://newsroom.tiktok.com/en-us/how-tiktok-recommends-videos-for-you

[42]

TikTok Blog Post. 2020. TikTok by the Numbers: Stats, Demographics & Fun Facts. https://newsroom.tiktok.com/en-us/how-tiktok-recommends-videos-for-you

[43]

Manoel Horta Ribeiro, Raphael Ottoni, Robert West, Virgílio AF Almeida, and Wagner Meira Jr. 2020. Auditing radicalization pathways on YouTube. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 131–141.

Digital Library

[44]

Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender systems handbook. Springer.

[45]

Camille L Ryan. 2013. Language use in the United States: 2011. (2013).

[46]

Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry 22(2014), 4349–4357.

[47]

Kyla Scanlon. 2020. The App That Knows You Better than You Know Yourself: An Analysis of the TikTok Algorithm. https://chatbotslife.com/the-app-that-knows-you-better-than-you-know-yourself-an-analysis-of-the-tiktok-algorithm-be12eefaab5a

[48]

Rashmi Sinha and Kirsten Swearingen. 2002. The role of transparency in recommender systems. In CHI’02 extended abstracts on Human factors in computing systems. 830–831.

[49]

Li Sun, Haoqi Zhang, Songyang Zhang, and Jiebo Luo. 2020. Content-based Analysis of the Cultural Differences between TikTok and Douyin. In 2020 IEEE International Conference on Big Data. https://doi.org/10.1109/BigData50022.2020.9378032

[50]

TikTok. 2021. Thanks a billion!https://newsroom.tiktok.com/en-us/1-billion-people-on-tiktok

[51]

Aleksandra Urman, Mykola Makhortykh, and Roberto Ulloa. 2021. The Matter of Chance: Auditing Web Search Results Related to the 2020 US Presidential Primary Elections Across Six Search Engines. Social science computer review(2021), 08944393211006863.

[52]

Jorge Vázquez-Herrero, María-Cruz Negreira-Rey, and Xosé López-García. 2020. Let’s dance the news! How the news media are adapting to the logic of TikTok. Journalism (2020), 1464884920969092.

[53]

Catherine Wang. 2020. Why TikTok made its user so obsessive? The AI Algorithm that got you hooked.https://towardsdatascience.com/why-tiktok-made-its-user-so-obsessive-the-ai-algorithm-that-got-you-hooked-7895bb1ab423

[54]

Gabriel Weimann and Natalie Masri. 2020. Research note: spreading hate on TikTok. Studies in Conflict & Terrorism(2020), 1–14.

[55]

Samuel C Woolley. 2016. Automating power: Social bot interference in global politics. First Monday (2016).

[56]

Xing Yi, Hema Raghavan, and Chris Leggetter. 2009. Discovering Users’ Specific Geo Intention in Web Search. In Proceedings of the 18th International Conference on World Wide Web (Madrid, Spain) (WWW ’09). Association for Computing Machinery, New York, NY, USA, 481–490. https://doi.org/10.1145/1526709.1526774

Digital Library

[57]

Zhengwei Zhao. 2021. Analysis on the “Douyin (Tiktok) Mania” Phenomenon Based on Recommendation Algorithms. In E3S Web of Conferences, Vol. 235. EDP Sciences, 03029.

[58]

Xujuan Zhou, Yue Xu, Yuefeng Li, Audun Josang, and Clive Cox. 2012. The state-of-the-art in personalized recommender systems for social networking. Artificial Intelligence Review 37, 2 (2012), 119–132.

Digital Library

Cited By

Mutiara PPutri K(2024)The Study of Uses and Gratification Theory of TikTok as A Shopping Platform Seen from Buyers’ ViewJournal of Digital Marketing and Communication10.53623/jdmc.v4i1.4294:1(7-18)Online publication date: 20-May-2024
https://doi.org/10.53623/jdmc.v4i1.429
García-Prieto VBonilla-del-Río MFiguereo-Benítez J(2024)Discapacidad, discursos de odio y redes sociales: video-respuestas a los haters en TikTokRevista Latina de Comunicación Social10.4185/rlcs-2024-2258(1-21)Online publication date: 12-Feb-2024
https://doi.org/10.4185/rlcs-2024-2258
Kaplan LSapiezynski PChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Comprehensively Auditing the TikTok Mobile AppCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3651260(1198-1201)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651260
Show More Cited By

Index Terms

An Empirical Investigation of Personalization Factors on TikTok
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information systems applications
  2. World Wide Web

Index terms have been assigned to the content through auto-classification.

Recommendations

TikTok and the Art of Personalization: Investigating Exploration and Exploitation on Social Media Feeds
WWW '24: Proceedings of the ACM on Web Conference 2024

Recommendation algorithms for social media feeds often function as black boxes from the perspective of users. We aim to detect whether social media feed recommendations are personalized to users, and to characterize the factors contributing to ...
Comprehensively Auditing the TikTok Mobile App
WWW '24: Companion Proceedings of the ACM on Web Conference 2024

TikTok has become a dominant force in the social media landscape of the United States, and has spawned other social media sites emulating their algorithmically-driven short form content recommendation platform (e.g. Youtube Shorts and Instagram Reels). ...
Social media recommendation based on people and tags
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

We study personalized item recommendation within an enterprise social media application suite that includes blogs, bookmarks, communities, wikis, and shared files. Recommendations are based on two of the core elements of social media - people and tags. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
2,337
Total Downloads

Downloads (Last 12 months)1,062
Downloads (Last 6 weeks)64

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mutiara PPutri K(2024)The Study of Uses and Gratification Theory of TikTok as A Shopping Platform Seen from Buyers’ ViewJournal of Digital Marketing and Communication10.53623/jdmc.v4i1.4294:1(7-18)Online publication date: 20-May-2024
https://doi.org/10.53623/jdmc.v4i1.429
García-Prieto VBonilla-del-Río MFiguereo-Benítez J(2024)Discapacidad, discursos de odio y redes sociales: video-respuestas a los haters en TikTokRevista Latina de Comunicación Social10.4185/rlcs-2024-2258(1-21)Online publication date: 12-Feb-2024
https://doi.org/10.4185/rlcs-2024-2258
Kaplan LSapiezynski PChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Comprehensively Auditing the TikTok Mobile AppCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3651260(1198-1201)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651260
Urman ASmirnov ILasser J(2024)The right to audit and power asymmetries in algorithm auditingEPJ Data Science10.1140/epjds/s13688-024-00454-513:1Online publication date: 7-Mar-2024
https://doi.org/10.1140/epjds/s13688-024-00454-5
Nowacki L(2024)The old king is dead, long live the algorithmic king – the decline of Facebook and the rise of TikTok – comparative study of algorithmic design of social media platformsInternational Social Science Journal10.1111/issj.12513Online publication date: 31-May-2024
https://doi.org/10.1111/issj.12513
Bunza R(2024)Duration-Based Investigation of User Content Choices in the Exit of Filter Bubbles2024 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA)10.1109/HORA61326.2024.10550588(1-8)Online publication date: 23-May-2024
https://doi.org/10.1109/HORA61326.2024.10550588
McLaughlin BCloudy JHunter JPotter B(2024)Stitch incoming: political engagement and aggression on TikTokBehaviour & Information Technology10.1080/0144929X.2024.2354436(1-14)Online publication date: 16-May-2024
https://doi.org/10.1080/0144929X.2024.2354436
Schwabl PHaim MUnkel J(2024)Aligning agent-based testing (ABT) with the experimental research paradigm: a literature review and best practicesJournal of Computational Social Science10.1007/s42001-024-00283-6Online publication date: 16-May-2024
https://doi.org/10.1007/s42001-024-00283-6
Yang YZeng ZJiang SChen MShang R(2024)DEKGCI: A double-ended recommendation model for integrating knowledge graph and user–item interaction graphThe Journal of Supercomputing10.1007/s11227-024-06344-xOnline publication date: 8-Jul-2024
https://doi.org/10.1007/s11227-024-06344-x
Ionescu CLicu M(2023)Are TikTok Algorithms Influencing Users’ Self-Perceived Identities and Personal Values? A Mini ReviewSocial Sciences10.3390/socsci1208046512:8(465)Online publication date: 21-Aug-2023
https://doi.org/10.3390/socsci12080465
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents