Best-Effort Scan of Twitch’s CDN in Europe
Pages 10 - 18
Abstract
Twitch is a live video service that is a leader in game streaming with 20 billion annual viewing hours and an annual viewer growth rate of +26%. To sustain this load and growth rate, Twitch maintains a content distribution network (CDN) to ensure high service quality and to distribute load. Its CDN is a critical online infrastructure that should be better understood. To supplement recent works attempting to measure Twitch’s CDN that crawled few active channels (~2%) and focused on mapping the CDN, we aimed to (1) scan the channels as complete as possible, which we refer to as a best-effort scan and (2) infer the server allocation scheme behind.
To contain the crawling traffic, we scope the scan within Europe. We observed 1045 servers in 24 server clusters which are located in 17 cities across 13 countries in Europe. We found that Twitch’s server allocation scheme has two phases: In the first phase, a viewer’s location (i.e., a vantage point or VP) is determined from their IP prefix and a pool of nearby server cluster(s) is selected. In the second phase, when a new channel is requested, a server cluster is selected and a server in the cluster is randomly selected to handle the load.
References
[1]
V. K. Adhikari, Y. Guo, F. Hao, V. Hilt, and Z. Zhang. 2012. A tale of three CDNs: An active measurement study of Hulu and its CDNs. In 2012 Proceedings IEEE INFOCOM Workshops. 7–12. https://doi.org/10.1109/INFCOMW.2012.6193524
[2]
V. K. Adhikari, Yang Guo, Fang Hao, M. Varvello, V. Hilt, M. Steiner, and Z. Zhang. 2012. Unreeling netflix: Understanding and improving multi-CDN movie delivery. In 2012 Proceedings IEEE INFOCOM. 1620–1628. https://doi.org/10.1109/INFCOM.2012.6195531
[3]
Adobe. 2012. Adobe’s Real Time Messaging Protocol. Retrieved July 30, 2023 from https://www.adobe.com/devnet/rtmp.html
[4]
Apple. 2021. HTTP Live Streaming. Retrieved July 30, 2023 from https://developer.apple.com/streaming/
[5]
Matt Calder, Xun Fan, Zi Hu, Ethan Katz-Bassett, John Heidemann, and Ramesh Govindan. 2013. Mapping the Expansion of Google’s Serving Infrastructure. In Proceedings of the 2013 Conference on Internet Measurement Conference (Barcelona, Spain) (IMC ’13). Association for Computing Machinery, New York, NY, USA, 313–326. https://doi.org/10.1145/2504730.2504754
[6]
Matt Calder, Ashley Flavel, Ethan Katz-Bassett, Ratul Mahajan, and Jitendra Padhye. 2015. Analyzing the Performance of an Anycast CDN. In Proceedings of the 2015 Internet Measurement Conference (Tokyo, Japan) (IMC ’15). Association for Computing Machinery, New York, NY, USA, 531–537. https://doi.org/10.1145/2815675.2815717
[7]
Fangfei Chen, Ramesh K. Sitaraman, and Marcelo Torres. 2015. End-User Mapping: Next Generation Request Routing for Content Delivery. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (London, United Kingdom) (SIGCOMM ’15). Association for Computing Machinery, New York, NY, USA, 167–181. https://doi.org/10.1145/2785956.2787500
[8]
Hsuan-Yu Chou. 2023. Kukudy. https://github.com/hy-chou/kukudy
[9]
GraphQL Community. 2023. GraphQL: A query language for your API. Retrieved July 30, 2023 from https://graphql.org
[10]
Twitch Developer. 2023. How many API calls can you make to Twitch in 24 hours?Retrieved July 30, 2023 from https://discuss.dev.twitch.tv/t/how-many-api-calls-can-you-make-to-twitch-in-24-hours/19385/7
[11]
Gavin Divers. 2023. Gaming Industry Dominates as the Highest-Grossing Entertainment Industry. Retrieved July 30, 2023 from https://gamerhub.co.uk/gaming-industry-dominates-as-the-highest-grossing-entertainment-industry/
[12]
Deng et al.2017. Internet Scale User-Generated Live Video Streaming: The Twitch Case. In Proceedings of PAM (Sydney, Australia). Springer, New York, NY, 60–71.
[13]
Jeff Gong, Sahil Dhanju, Chih-Chiang Lu, and Yueshi Shen. 2017. Live Video Transmuxing/Transcoding: FFmpeg vs. Twitch Transcoder Part I. Retrieved July 30, 2023 from https://blog.twitch.tv/en/2017/10/10/live-video-transmuxing-transcoding-f-fmpeg-vs-twitch-transcoder-part-i-489c1c125f28/
[14]
Xinjie Guan and Baek-Young Choi. 2011. Push or Pull?: Toward Optimal Content Delivery. IEEE International Conference on Communications, 1–5. https://doi.org/10.1109/icc.2011.5963299
[15]
Apple Inc.2021. Creating a Master Playlist. Retrieved July 30, 2023 from https://developer.apple.com/documentation/http_live_streaming/example_playlists_for_http_live_streaming/creating_a_master_playlist
[16]
Apple Inc.2021. Live Playlist (Sliding Window) Construction. Retrieved July 30, 2023 from https://developer.apple.com/documentation/http_live_streaming/example_playlists_for_http_live_streaming/live_playlist_sliding_window_construction
[17]
Imarc Impactful Insights. 2022. Gaming Market Report by Device Type, Platform, Revenue, Type, Age Group, and Region 2023-2028. Retrieved July 30, 2023 from https://www.imarcgroup.com/gaming-market
[18]
ipinfo.io. Retrieved 2023. AS46489 Twitch Interactive Inc.Retrieved July 30, 2023 from https://ipinfo.io/AS46489
[19]
Mansoor Iqbal. 2023. Twitch Revenue and Usage Statistics. Retrieved July 30, 2023 from https://www.businessofapps.com/data/twitch-statistics/
[20]
Nick. 2021. Which 15 Countries Have The Most Twitch Viewers?Retrieved July 30, 2023 from https://visualsbyimpulse.com/countries-most-twitch-viewers-top-15/
[21]
NordVPN. 2023. NordVPN Servers. Retrieved July 30, 2023 from https://nordvpn.com/servers
[22]
NordVPN. 2023. NordVPN Website. Retrieved July 30, 2023 from https://www.nordvpn.com
[23]
R. Torres, A. Finamore, J. R. Kim, M. Mellia, M. M. Munafo, and S. Rao. 2011. Dissecting Video Server Selection Strategies in the YouTube CDN. In 2011 31st International Conference on Distributed Computing Systems. 248–257. https://doi.org/10.1109/ICDCS.2011.43
[24]
Twitch. 2015. Video Broadcasting Overview. Retrieved July 30, 2023 from https://blog.twitch.tv/en/2015/12/18/twitch-engineering-an-introduction-and-overview-a23917b71a25/
[25]
Twitch. 2023. Twitch API Concepts – Twtich Rate Limits. Retrieved July 30, 2023 from https://dev.twitch.tv/docs/api/guide/#twitch-rate-limits
[26]
Twitch. 2023. Twitch API Reference. Retrieved July 30, 2023 from https://dev.twitch.tv/docs/api/reference/
[27]
Twitch. 2023. Using the Twitch API in an Extension Front End - Rate limits. Retrieved July 30, 2023 from https://dev.twitch.tv/docs/extensions/frontend-api-usage/#rate-limits
[28]
TwitchTracker. 2023. TwitchTracker. Retrieved July 30, 2023 from https://twitchtracker.com/statistics/channels
[29]
Maurice Wahba. 2023. Twitch GraphQL API. Retrieved July 30, 2023 from https://github.com/mauricew/twitch-graphql-api
[30]
Bolun Wang, Xinyi Zhang, Gang Wang, Haitao Zheng, and Ben Y. Zhao. 2016. Anatomy of a Personalized Livestreaming System. In Proceedings of the 2016 Internet Measurement Conference (Santa Monica, California, USA) (IMC ’16). Association for Computing Machinery, New York, NY, USA, 485–498. https://doi.org/10.1145/2987443.2987453
[31]
Caleb Wang, Yuan-Tai Liu, and Polly Huang. 2022. Jujuby: Design and Deployment of a Crawler for Twitch CDN Mapping. In Proceedings of the 17th Asian Internet Engineering Conference. 44–52.
[32]
Wikipedia. 2023. Adaptive Bitrate Streaming. Retrieved July 30, 2023 from https://en.wikipedia.org/wiki/Adaptive_bitrate_streaming
[33]
Wikipedia. 2023. Exponential backoff. Retrieved July 30, 2023 from https://en.wikipedia.org/wiki/Exponential_backoff
[34]
Wei-Shiang Wung, Guan-Ting Ting, Ruey-Tzer Hsu, Cheng Hsu, Yu-Chien Tsai, Caleb Wang, Yuan-Tai Liu, Hsi Chen, and Polly Huang. 2021. Twitch’s CDN as an Open Population Ecosystem. In Proceedings of AINTEC’21 (Virtual Event, Japan) (AINTEC ’21). Association for Computing Machinery, New York, NY, USA, 56–63. https://doi.org/10.1145/3497777.3498551
Index Terms
- Best-Effort Scan of Twitch’s CDN in Europe
Recommendations
Jujuby: Design and Deployment of a Crawler for Twitch CDN Mapping
AINTEC '22: Proceedings of the 17th Asian Internet Engineering ConferenceContent Delivery Networks (CDNs) deliver much of the world’s content on the Internet today. As demand for content rises, the role of CDNs is as crucial as ever, making it imperative to understand the internal workings of such systems. Focusing on ...
Twitch’s CDN as an Open Population Ecosystem
AINTEC '21: Proceedings of the 16th Asian Internet Engineering ConferenceThe quality and continuity of the video services such as Twitch depend on the scale and well-being of their content distribution networks (CDNs). Each CDN may consist of 1000s of servers, physically feeding the videos to the clients. Opting for a ...
Streaming on twitch: fostering participatory communities of play within live mixed media
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsPreviously, video streaming sites were at the fringes of online social media. In the past two years, live streams of video games, on sites such as Twitch.tv, have become very popular. Live streams serve as meeting grounds for player communities. The ...
Comments
Information & Contributors
Information
Published In
December 2023
129 pages
ISBN:9798400709395
DOI:10.1145/3630590
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
AINTEC '23
Acceptance Rates
Overall Acceptance Rate 15 of 38 submissions, 39%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 55Total Downloads
- Downloads (Last 12 months)55
- Downloads (Last 6 weeks)4
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format