research-article

Selecting representative IP addresses for internet topology studies

Authors:

John HeidemannAuthors Info & Claims

IMC '10: Proceedings of the 10th ACM SIGCOMM conference on Internet measurement

Pages 411 - 423

https://doi.org/10.1145/1879141.1879195

Published: 01 November 2010 Publication History

Abstract

An Internet hitlist is a set of addresses that cover and can represent the the Internet as a whole. Hitlists have long been used in studies of Internet topology, reachability, and performance, serving as the destinations of traceroute or performance probes. Most early topology studies used manually generated lists of prominent addresses, but evolution and growth of the Internet make human maintenance untenable. Random selection scales to today's address space, but most random addresses fail to respond. In this paper we present what we believe is the first automatic generation of hitlists informed censuses of Internet addresses. We formalize the desirable characteristics of a hitlist: responsiveness, each representative responds to pings; completeness, they cover all the allocated IPv4 address space; and stability, list evolution is minimized when possible. We quantify the accuracy of our automatic hitlists, showing that only one-third of the Internet allows informed selection of representatives. Of informed representatives, 50--60% are likely to respond three months later, and we show that causes for non-responses are likely due to dynamic addressing (so no stable representative exists) or firewalls. In spite of these limitations, we show that the use of informed hitlists can add 1.7 million edge links (a 5% growth) to traceroute-based Internet topology studies Our hitlists are available free-of-charge and are in use by several other research projects.

References

[1]

Réka Albert, Hawoong Jeong, and Albert-László Barabási. Error and attack tolerance in complex networks. Nature, 406:378--382, July 27 2000.

[2]

Adam Bender, Rob Sherwood, and Neil Spring. Fixing Ally's growing pains with velocity modeling. In Proceedings of the 8th ACM Internet Measurement Conference, pages 337--342, Vouliagmeni, Greece, October 2008. ACM.

Digital Library

[3]

Randy Bush, James Hiebert, Olaf Maennel, Matthew Roughan, and Steve Uhlig. Testing the reachability of (new) address space. In Proceedings of the ACM Workshop on Internet Nework Management, pages 236--241, Kyoto, Japan, August 2007. ACM.

Digital Library

[4]

Randy Bush, Olaf Maennel, Matthew Roughan, and Steve Uhlig. Internet optometry: assessing the broken glasses in internet reachability. In Proceedings of the ACM Internet Measurement Conference, pages 242--253. ACM, November 2009.

Digital Library

[5]

Xue Cai and John Heidemann. Understanding address usage in the visible internet. Technical Report ISI-TR-2009--656, USC/Information Sciences Institute, February 2009.

[6]

CAIDA. The internet topology data kit 2010-01. http://www.caida.org/data/active/ internet-topology-data-kit/, January 2010.

[7]

CAIDA. The ipv4 routed /24 topology dataset 2009--12. http://www.caida.org/data/active/ipv4_ routed_24_topology_dataset.xml, January 2010.

[8]

Kimberly Claffy, Young Hyun, Ken Keys, Marina Fomenkov, and Dmitri Krioukov. Internet mapping: from art to science. In Proceedings of the IEEE Cybersecurity Applications and Technologies Conference for Homeland Security (CATCH), pages 205--211, Alexandria, VA, USA, March 2009. IEEE.

Digital Library

[9]

David D. Clark, Craig Partridge, J. Christopher Ramming, and John T. Wroclawski. A knowledge plane for the Internet. In Proceedings of the ACM SIGCOMM Conference, pages 3--10, Karlsruhe, Germany, August 2003. ACM.

Digital Library

[10]

Eric Cronin, Sugih Jamin, Cheng Jin, Anthony R. Kurc, Danny Raz, and Yuval Shavitt. Constrained mirror placement on the Internet. IEEE Journal of Selected Areas in Communication, 20(7):1369--1383, September 2002.

Digital Library

[11]

Doug Cutting. Scalable computing with Hadoop. http://wiki.apache.org/lucene-hadoop-data/ attachments/HadoopPresentations/attachments/ yahoo-sds.pdf, May 2006. Lecture note.

[12]

Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified data processing on large clusters. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation, pages 137--150, San Francisco, California, USA, December 2004. USENIX.

Digital Library

[13]

Alex Dekhtyar and Jane Huffman Hayes. Good benchmarks are hard to find: Toward the benchmark for information retrieval applications in software engineering. In Proceedings of the 22nd International Conference on Software Maintenance, Philadelphia, Pennsylvania, USA, September 2006. ACM.

[14]

John Dilley, Bruce Maggs, Jay Parikh, Harald Prokop, Ramesh Sitaraman, and Bill Weihl. Globally distributed content delivery. IEEE Internet Computing, 6(5):50--58, September 2002.

Digital Library

[15]

Paul Francis, Sugih Jamin, Cheng Jin, Yixin Jin, Danny Raz, Yuval Shavitt, and Lixia Zhang. IDMaps: A global internet host distance estimation service. ACM/IEEE Transactions on Networking, 9(5):525--540, October 2001.

Digital Library

[16]

V. Fuller, T. Li, J. Yu, and K. Varadhan. Classless inter-domain routing (CIDR): an address assignment and aggregation strategy. RFC 1519, Internet Request For Comments, September 1993.

Digital Library

[17]

Ramesh Govindan and Hongsuda Tangmunarunkit. Heuristics for Internet map discovery. In Proceedings of the IEEE Infocom, pages 1371--1380, Tel Aviv, Israel, March 2000. IEEE.

[18]

John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos, Genevieve Bartlett, and Joseph Bannister. Census and survey of the visible Internet. In Proceedings of the ACM Internet Measurement Conference, pages 169--182, Vouliagmeni, Greece, October 2008. ACM.

Digital Library

[19]

Bradley Huffaker, Marina Fomenkov, David Moore, and kc claffy. Macroscopic analyses of the infrastructure: measurement and visualization of internet connectivity and performance. http://www. caida.org/outreach/papers/pam2001/skitter.xml, November 2001.

[20]

Bradley Huffaker, Marina Fomenkov, Daniel J. Plummer, David Moore, and k claffy. Distance metrics in the internet. In Proceedings of the IEEE International Telecommunications Symposium. IEEE, 2002.

[21]

Ken Keys. IP alias resolution techniques. Technical report, CAIDA, 2008.

[22]

Harsha V. Madhyastha, Tomas Isdal, Michael Piatek, Colin Dixon, Thomas Anderson, Arvind Krishnamurthy, and Arun Venkataramani. iPlane: An information plane for distributed services. In Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, pages 367--380, Seattle, WA, USA, November 2006. USENIX.

Digital Library

[23]

Harsha V. Madhyastha, Ethan Katz-Bassett, Thomas Anderson, Arvind Krishnamurthy, and Arun Venkataramani. iPlane Nano: Path prediction for peer-to-peer applications. In Proceedings of the 6th USENIX Symposium on Network Systems Design and Implementation, Boston, MA, USA, April 2009. USENIX.

Digital Library

[24]

Eric Pfanner. Broadband speeds surge in many countries. New York Times, page B8, Oct. 1 2009.

[25]

Rob Sherwood, Adam Bender, and Neil Spring. DisCarte: A disjunctive Internet cartographer. In Proceedings of the ACM SIGCOMM Conference, pages 303--315, Seatle, Washington, USA, August 2008. ACM.

Digital Library

[26]

Neil Spring, Ratul Mahajan, and David Wetherall. Measuring ISP topologies with Rocketfuel. In Proceedings of the ACM SIGCOMM Conference, pages 133--145, Pittsburgh, Pennsylvania, USA, August 2002. ACM.

Digital Library

[27]

Matthew Sullivan and Luis Munoz. Suggested generic DNS naming schemes for large networks and unassigned hosts. Work in progress (Internet draft draft-msullivan-dnsop-generic-naming-schemes-00.txt, April 2006.

[28]

The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. Technical report, Department of Health, Education, and Welfare, April 1979.

[29]

USC/LANDER Project. Internet IPv4 address space census. PREDICT ID USC-LANDER/internet_ address_survey_it11w-20060307. Retrieval information for this and other censuses is at http://www.isi.edu/ant/traces/, March 2006.

[30]

D.G. Waddington, F. Chang, R. Viswanathan, and B. Yao. Topology discovery for public IPv6 networks. ACM Computer Communication Review, 33(3):59--68, July 2003.

Digital Library

[31]

Feng Wang, Zhuoqing Morley Mao, Jia Wang, Lixin Gao, and Randy Bush. A measurement study on the impact of routing events on end-to-end Internet path performance. In Proceedings of the ACM SIGCOMM Conference, Pisa, Italy, August 2006. ACM.

Digital Library

[32]

Rich Wolski. Dynamically forecasting network performance using the network weather service. Journal of Cluster Computing, 1:119--132, January 1998. Also released as UCSD technical report TR-CS96--494.

Digital Library

[33]

Edward Wyatt. Despite ruling, F.C.C. says it will move forward on expanding broadband. New York Times, page B3, April 15 2010.

[34]

Yinglian Xie, Fang Yu, Kannan Achan, Eliot Gillum, Moises Goldszmidt, and Ted Wobber. How dynamic are IP addresses? In Proceedings of the ACM SIGCOMM Conference, pages 301--312, Kyoto, Japan, August 2007. ACM.

Digital Library

Cited By

Degen BJonker Mvan Rijswijk-Deij RSommese R(2024)An Empirical Characterization of Anycast Convergence TimeProceedings of the 2024 Applied Networking Research Workshop10.1145/3673422.3674890(23-30)Online publication date: 23-Jul-2024
https://dl.acm.org/doi/10.1145/3673422.3674890
Barreto RBertholdo LMarcos PVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Poster: Traffic Engineering Security ImplicationsProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3689672(771-772)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3689672
Salamatian LVermeulen KCunha IGiotsas VKatz-Bassett EVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)metAScritic: Reframing AS-Level Topology Discovery as a Recommendation SystemProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688429(337-364)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688429
Show More Cited By

Index Terms

Selecting representative IP addresses for internet topology studies
1. Networks

Recommendations

Towards capturing representative AS-level Internet topologies

Recent studies on AS-level Internet connectivity have attracted considerable attention. These studies exclusively relied on BGP data from the Oregon route-views [University of Oregon Route Views Project, http:// www.routeviews.org] to derive some ...
Predicting Zero-day Malicious IP Addresses
SafeConfig '17: Proceedings of the 2017 Workshop on Automated Decision Making for Active Cyber Defense

Blacklisting IP addresses is an important part of enterprise security today. Malware infections and Advanced Persistent Threats can be detected when blacklisted IP addresses are contacted. It can also thwart phishing attacks by blocking suspicious ...
Internet topology discovery
DataTraffic Monitoring and Analysis

Since the nineties, the Internet has seen an impressive growth, in terms of users, intermediate systems (such as routers), autonomous systems, or applications. In parallel to this growth, the research community has been looking for obtaining and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IMC '10: Proceedings of the 10th ACM SIGCOMM conference on Internet measurement

November 2010

496 pages

ISBN:9781450304832

DOI:10.1145/1879141

Program Chair:
Mark Allman
International Computer Science Institute

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

USENIX Assoc: USENIX Assoc

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IMC '10

Sponsor:

IMC '10: Internet Measurement Conference

November 1 - 30, 2010

Melbourne, Australia

Acceptance Rates

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
333
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Degen BJonker Mvan Rijswijk-Deij RSommese R(2024)An Empirical Characterization of Anycast Convergence TimeProceedings of the 2024 Applied Networking Research Workshop10.1145/3673422.3674890(23-30)Online publication date: 23-Jul-2024
https://dl.acm.org/doi/10.1145/3673422.3674890
Barreto RBertholdo LMarcos PVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Poster: Traffic Engineering Security ImplicationsProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3689672(771-772)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3689672
Salamatian LVermeulen KCunha IGiotsas VKatz-Bassett EVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)metAScritic: Reframing AS-Level Topology Discovery as a Recommendation SystemProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688429(337-364)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688429
Baltra GSong XHeidemann J(2024)Ebb and Flow: Implications of ISP Address DynamicsPassive and Active Measurement10.1007/978-3-031-56252-5_7(132-149)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56252-5_7
Sethuraman MBischof ZDainotti A(2024)Towards Improving Outage Detection with Multiple Probing ProtocolsPassive and Active Measurement10.1007/978-3-031-56249-5_8(189-205)Online publication date: 20-Mar-2024
https://doi.org/10.1007/978-3-031-56249-5_8
Darwich ORimlinger HDreyfus MGouel MVermeulen KMontpetit MLeivadeas AUhlig SJaved M(2023)Replication: Towards a Publicly Available Internet Scale IP Geolocation DatasetProceedings of the 2023 ACM on Internet Measurement Conference10.1145/3618257.3624801(1-15)Online publication date: 24-Oct-2023
https://dl.acm.org/doi/10.1145/3618257.3624801
Anderson SBell TEgan PWeinshenker NBarford P(2023)Measuring the Impacts of Power Outages on Internet Hosts in the United StatesCritical Infrastructure Protection XVII10.1007/978-3-031-49585-4_4(62-90)Online publication date: 29-Dec-2023
https://doi.org/10.1007/978-3-031-49585-4_4
Alaraj ABock KLevin DWustrow E(2023)A Global Measurement of Routing Loops on the InternetPassive and Active Measurement10.1007/978-3-031-28486-1_16(373-399)Online publication date: 21-Mar-2023
https://dl.acm.org/doi/10.1007/978-3-031-28486-1_16
Sethuraman MBischof ZDainotti ABarakat CPelsser CBenson TChoffnes D(2022)Analysis of IPv4 address space utilization with ANT ISI dataset and censysProceedings of the 22nd ACM Internet Measurement Conference10.1145/3517745.3563018(744-745)Online publication date: 25-Oct-2022
https://dl.acm.org/doi/10.1145/3517745.3563018
Del Fiore JPersico VMerindol PPelsser CPescape A(2021)The Art of Detecting Forwarding DetoursIEEE Transactions on Network and Service Management10.1109/TNSM.2021.306215118:3(3619-3632)Online publication date: Sep-2021
https://doi.org/10.1109/TNSM.2021.3062151
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten