Towards a traffic map of the Internet: Connecting the dots between popular services and users
Authors: 
Thomas Koch, Weifan Jiang, Tao Luo, 
Petros Gigis, Yunfan Zhang, Kevin Vermeulen, Emile Aben, Matt Calder, Ethan Katz-Bassett, 
Lefteris Manassakis, 
Georgios Smaragdakis, Narseo Vallina-RodriguezAuthors Info & Claims
Thomas Koch, Weifan Jiang, Tao Luo, Petros Gigis, Yunfan Zhang, Kevin Vermeulen, Emile Aben, Matt Calder, Ethan Katz-Bassett, Lefteris Manassakis, Georgios Smaragdakis, Narseo Vallina-RodriguezAuthors Info & ClaimsPages 23 - 30
Abstract
The impact of Internet phenomena depends on how they impact users, but researchers lack visibility into how to translate Internet events into their impact. Distressingly, the research community seems to have lost hope of obtaining this information without relying on privileged viewpoints. We argue for optimism thanks to new network measurement methods and changes in Internet structure which make it possible to construct an "Internet traffic map". This map would identify the locations of users and major services, the paths between them, and the relative activity levels routed along these paths. We sketch our vision for the map, detail new measurement ideas for map construction, and identify key challenges that the research community should tackle. The realization of an Internet traffic map will be an Internet-scale research effort with Internet-scale impacts that reach far beyond the research community, and so we hope our fellow researchers are excited to join us in addressing this challenge.
References
[1]
Archipelago monitor locations, 2015. URL https://caida.org/projects/ark/locations.
[2]
Cloudflare workers, 2021. URL https://workers.cloudflare.com/.
[3]
Bernhard Ager, Wolfgang Mühlbauer, Georgios Smaragdakis, and Steve Uhlig. Web content cartography. In ACM IMC, 2011.
[4]
Bernhard Ager, Nikolaos Chatzis, Anja Feldmann, Nadi Sarrar, Steve Uhlig, and Walter Willinger. Anatomy of a large European IXP. In ACM SIGCOMM, 2012.
[5]
Zahaib Akhtar, Yun Seong Nam, Jessica Chen, Ramesh Govindan, Ethan Katz-Bassett, Sanjay Rao, Jibin Zhan, and Hui Zhang. Understanding video management planes. In ACM IMC, 2018.
[6]
Todd Arnold, Ege Gürmeriçliler, Georgia Essig, Arpit Gupta, Matt Calder, Vasileios Giotsas, and Ethan Katz-Bassett. (How much) does a private WAN improve cloud performance? In IEEE INFOCOM, 2020.
[7]
Todd Arnold, Jia He, Weifan Jiang, Matt Calder, Italo Cunha, Vasileios Giotsas, and Ethan Katz-Bassett. Cloud provider connectivity in the flat Internet. In ACM IMC, 2020.
[8]
Adam Bender, Rob Sherwood, and Neil Spring. Fixing Ally's growing pains with velocity modeling. In ACM IMC, 2008.
[9]
Robert Beverly. Yarrp'ing the Internet: Randomized high-speed active topology discovery. In ACM IMC, 2016.
[10]
A. Bhatia. Mapping Facebook's FNA (CDN) nodes across the world! https://anuragbhatia.com/2018/03/networking/isp-column/mapping-facebooks-fna-cdn-nodes-across-the-world/, 2018.
[11]
A. Bhatia. Facebook FNA node update. https://anuragbhatia.com/2019/11/networking/isp-column/facebook-fna-node-update/, 2019.
[12]
Timm Böttger, Felix Cuadrado, Gareth Tyson, Ignacio Castro, and Steve Uhlig. Open Connect everywhere: A glimpse at the Internet ecosystem through the lens of the Netflix CDN. In ACM SIGCOMM CCR, 2018.
[13]
Matt Calder, Xun Fan, Zi Hu, Ethan Katz-Bassett, John Heidemann, and Ramesh Govindan. Mapping the expansion of Google's serving infrastructure. In ACM IMC, 2013.
[14]
Matt Calder, Ashley Flavel, Ethan Katz-Bassett, Ratul Mahajan, and Jitendra Padhye. Analyzing the performance of an anycast CDN. In ACM IMC, 2015.
[15]
Matt Calder, Ryan Gao, Manuel Schröder, Ryan Stewart, Jitendra Padhye, Ratul Mahajan, Ganesh Ananthanarayanan, and Ethan Katz-Bassett. Odin: Microsoft's scalable fault-tolerant CDN measurement system. In USENIX NSDI, 2018.
[16]
Matt Calder, Xun Fan, and Liang Zhu. A cloud provider's view of EDNS Client-Subnet adoption. In IEEE TMA, 2019.
[17]
Fangfei Chen, Ramesh K Sitaraman, and Marcelo Torres. End-user mapping: Next generation request routing for content delivery. In ACM SIGCOMM, 2015.
[18]
Yi-Ching Chiu, Brandon Schlinker, Abhishek Balaji Radhakrishnan, Ethan Katz-Bassett, and Ramesh Govindan. Are We One Hop Away from a Better Internet? In Proc. ACM IMC, 2015.
[19]
Yi-Ching Chiu, Brandon Schlinker, Abhishek Balaji Radhakrishnan, Ethan Katz-Bassett, and Ramesh Govindan. Are we one hop away from a better Internet? In ACM IMC, 2015.
[20]
David R Choffnes and Fabián E Bustamante. Taming the torrent: a practical approach to reducing cross-ISP traffic in peer-to-peer systems. In ACM SIGCOMM, 2008.
[21]
Wouter B De Vries, Ricardo de O. Schmidt, Wes Hardaker, John Heidemann, Pieter-Tjerk de Boer, and Aiko Pras. Broad and load-aware anycast mapping with Verfploeter. In ACM IMC, 2017.
[22]
Ramakrishnan Durairajan, Paul Barford, Joel Sommers, and Walter Willinger. Intertubes: A study of the US long-haul fiber-optic infrastructure. In ACM SIGCOMM, 2015.
[23]
Rodérick Fanou, Francisco Valera, and Amogh Dhamdhere. Investigating the causes of congestion on the African IXP substrate. In ACM IMC, 2017.
[24]
Petros Gigis, Vasileios Kotronis, Emile Aben, Stephen D Strowes, and Xenofontas Dimitropoulos. Characterizing user-to-user connectivity with RIPE Atlas. In ACM ANRW, 2017.
[25]
Petros Gigis, Matt Calder, Lefteris Manassakis, George Nomikos, Vasileios Kotronis, Xenofontas Dimitropoulos, Ethan Katz-Bassett, and Georgios Smaragdakis. Seven years in the life of hypergiants' off-nets. In ACM SIGCOMM, 2021.
[26]
Vasileios Giotsas, Georgios Smaragdakis, Bradley Huffaker, Matthew Luckie, and KC Claffy. Mapping peering interconnections to a facility. In ACM CoNEXT, 2015
[27]
Ramesh Govindan and Hongsuda Tangmunarunkit. Heuristics for Internet map discovery. In IEEE INFOCOM, 2000.
[28]
Enrico Gregori, Alessandro Improta, and Luca Sani. On the African peering connectivity revealable via BGP route collectors. In EAI AFRICOMM, 2017.
[29]
Arpit Gupta, Matt Calder, Nick Feamster, Marshini Chetty, Enrico Calandro, and Ethan Katz-Bassett. Peering at the Internet's frontier: A first look at ISP interconnectivity in Africa. In PAM, 2014.
[30]
Gonca Gürsun and Mark Crovella. On traffic matrix completion in the Internet. In ACM IMC, 2012.
[31]
Gonca Gürsun, Natali Ruchansky, Evimaria Terzi, and Mark Crovella. Inferring visibility: Who's (not) talking to whom? In ACM SIGCOMM, 2012.
[32]
Cheng Huang, Angela Wang, Jin Li, and Keith W Ross. Measuring and evaluating large-scale CDNs. In ACM IMC, 2008.
[33]
Geoff Huston. How big is that network, 2014. URL https://labs.apnic.net/?p=526.
[34]
Weifan Jiang, Tao Luo, Thomas Koch, Yunfan Zhang, Ethan Katz-Bassett, and Matt Calder. Towards identifying networks with Internet clients using public data. In ACM IMC, 2021.
[35]
Yuchen Jin, Colin Scott, Amogh Dhamdhere, Vasileios Giotsas, Arvind Krishnamurthy, and Scott Shenker. Stable and practical AS relationship inference with ProbLink. In USENIX NSDI, 2019.
[36]
Ethan Katz-Bassett, Harsha V Madhyastha, Vijay Kumar Adhikari, Colin Scott, Justine Sherry, Peter Van Wesep, Thomas E Anderson, and Arvind Krishnamurthy. Reverse traceroute. In USENIX NSDI, 2010.
[37]
Ken Keys, Young Hyun, Matthew Luckie, and Kim Claffy. Internet-scale IPv4 alias resolution with MIDAR. In IEEE/ACM ToN, 2012.
[38]
Thomas Koch, Ke Li, Calvin Ardi, Matt Calder, John Heidemann, and Ethan Katz-Bassett. Anycast in context: A tale of two systems. In ACM SIGCOMM, 2021.
[39]
Vasileios Kotronis, George Nomikos, Lefteris Manassakis, Dimitris Mavrommatis, and Xenofontas Dimitropoulos. Shortcuts through colocation facilities. In ACM IMC, 2017.
[40]
Craig Labovitz, Scott Iekel-Johnson, Danny McPherson, Jon Oberheide, and Farnam Jahanian. Internet inter-domain traffic. In ACM SIGCOMM, 2010.
[41]
Matthew Luckie, Bradley Huffaker, Amogh Dhamdhere, Vasileios Giotsas, and KC Claffy. AS relationships, customer cones, and validation. In ACM IMC, 2013.
[42]
Harsha V Madhyastha, Tomas Isdal, Michael Piatek, Colin Dixon, Thomas Anderson, Arvind Krishnamurthy, and Arun Venkataramani. iplane: An information plane for distributed services. In USENIX OSDI, 2006.
[43]
Zhuoqing Morley Mao, Charles D Cranor, Fred Douglis, Michael Rabinovich, Oliver Spatscheck, and Jia Wang. A precise and efficient evaluation of the proximity between web clients and their local DNS servers. In USENIX ATC, 2002.
[44]
Srdjan Matic, Gareth Tyson, and Gianluca Stringhini. Pythia: a framework for the automated analysis of web hosting environments. In ACM WWW, 2019.
[45]
Prem Melville and Vikas Sindhwani. Recommender systems. Encyclopedia of machine learning, 1:829--838, 2010.
[46]
Foivos Michclinakis, Hossein Doroud, Abbas Razaghpanah, Andra Lutu, Narseo Vallina-Rodriguez, Phillipa Gill, and Joerg Widmer. The cloud that runs the mobile Internet: A measurement study of mobile cloud services. In IEEE INEOCOM, 2018.
[47]
George Nomikos, Vasileios Kotronis, Pavlos Sermpezis, Petros Gigis, Lefteris Manassakis, Christoph Dietzel, Stavros Konstantaras, Xenofontas Dimitropoulos, and Vasileios Giotsas. O peer, where art thou? Uncovering remote peering interconnections at IXPs. In ACM IMC, 2018.
[48]
Ricardo Oliveira, Dan Pei, Walter Willinger, Beichuan Zhang, and Lixia Zhang. The (in) completeness of the observed Internet AS-level structure. In IEEE/ACM ToN, 2009.
[49]
Eduardo E P. Pujol, Will Scott, Eric Wustrow, and J Alex Halderman. Initial measurements of the Cuban street network. In ACM IMC, 2017.
[50]
John P Rula and Fabian E Bustamante. Behind the curtain: Cellular DNS and content replica selection. In ACM IMC, 2014.
[51]
John P Rula, Fabián E Bustamante, and Moritz Steiner. Cell spotting: studying the role of cellular networks in the Internet. In ACM IMC, 2017.
[52]
Mario A Sánchez, John S Otto, Zachary S Bischof, David R Choffnes, Fabián E Bustamante, Balachander Krishnamurthy, and Walter Willinger. Dasu: Pushing experiments to the Internet's edge. In USENIX NSDI, 2013.
[53]
Mario A Sanchez, Fabian E Bustamante, Balachander Krishnamurthy, Walter Willinger, Georgios Smaragdakis, and Jeffrey Erman. Interdomain traffic estimation for the outsider. In ACM IMC, 2014.
[54]
Quirin Scheitle, Oliver Hohlfeld, Julien Gamba, Jonas Jelten, Torsten Zimmermann, Stephen D Strowes, and Narseo Vallina-Rodriguez. A long way to the top: Significance, structure, and stability of Internet top lists. In ACM IMC, 2018.
[55]
Neil Spring, Ratul Mahajan, David Wetherall, and Thomas Anderson. Measuring ISP topologies with Rocketfuel. In IEEE/ACM ToN, 2004.
[56]
Florian Streibelt, Jan Böttger, Nikolaos Chatzis, Georgios Smaragdakis, and Anja Feldmann. Exploring EDNS-Client-Subnet adopters in your free time. In ACM IMC, 2013.
[57]
Ao-Jan Su, David R Choffnes, Aleksandar Kuzmanovic, and Fabián E Bustamante. Drafting behind Akamai (travelocity-based detouring). In ACM SIGCOMM, 2006.
[58]
Yi Sun, Junchen Jiang, Vyas Sekar, Hui Zhang, Fuyuan Lin, and Nanshu Wang. Using video-based measurements to generate a real-time network traffic map. In ACM HotNets, 2014.
[59]
Matthew Thomas. Chromium's impact on root DNS traffic, 2020. URL https://blog.apnic.net/2020/08/21/chromiums-impact-on-root-dns-traffic.
[60]
Sipat Triukose, Zhihua Wen, and Michael Rabinovich. Measuring a commercial content delivery network. In ACM WWW, 2011.
[61]
Narseo Vallina-Rodriguez, Srikanth Sundaresan, Christian Kreibich, Nicholas Weaver, and Vern Paxson. Beyond the radio: Illuminating the higher layers of mobile networks. In ACM MobiSys, 2015.
[62]
Kevin Vermeulen, Justin P Rohrer, Robert Beverly, Olivier Fourmaux, and Timur Friedman. Diamond-Miner: Comprehensive discovery of the Internet's topology diamonds. In USENIX NSDI, 2020.
[63]
Florian Wohlfart, Nikolaos Chatzis, Caglar Dabanoglu, Georg Carle, and Walter Willinger. Leveraging interconnections for performance: the serving infrastructure of a large CDN. In ACM SIGCOMM, 2018.
[64]
Bahador Yeganeh, Ramakrishnan Durairajan, Reza Rejaie, and Walter Willinger. How cloud traffic goes hiding: A study of Amazon's peering fabric. In ACM IMC, 2019.
[65]
Kyriakos Zarifis, Tobias Flach, Srikanth Nori, David Choffnes, Ramesh Govindan, Ethan Katz-Bassett, Z Morley Mao, and Matt Welsh. Diagnosing path inflation of mobile client traffic. In PAM, 2014.
Index Terms
- Towards a traffic map of the Internet: Connecting the dots between popular services and users
Index terms have been assigned to the content through auto-classification.
Recommendations
Addressing Unwanted Traffic on the Internet: A Community Response
Unwanted traffic is a serious problem on today's network. For too long, the balance between the value of the transactions occurring on the network and the security of the infrastructure itself has tipped in the wrong direction. In this article, the ...
An Adaptive Road Traffic Regulation with Simulation and Internet of Things
SIGSIM-PADS '16: Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete SimulationTraffic congestion is a growing concern in most cities across the world. It is primarily caused by a sudden increase in the number of vehicles in a relatively small number of roads and intersections, while other roads have the capacity to accommodate ...
Comments
Information & Contributors
Information
Published In
November 2021
246 pages
ISBN:9781450390873
DOI:10.1145/3484266
Copyright © 2021 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 November 2021
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- National Science Foundation
- BMBF
- European Research Council
- Spanish national project
Conference
HotNets '21
Sponsor:
HotNets '21: The 20th ACM Workshop on Hot Topics in Networks
November 10 - 12, 2021
Virtual Event, United Kingdom
Acceptance Rates
Overall Acceptance Rate 110 of 460 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 1,175Total Downloads
- Downloads (Last 12 months)323
- Downloads (Last 6 weeks)31
Reflects downloads up to 15 Oct 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in