research-article

Learning to Route

Authors:

Asaf Valadarsky,

Michael Schapira,

Aviv TamarAuthors Info & Claims

HotNets '17: Proceedings of the 16th ACM Workshop on Hot Topics in Networks

Pages 185 - 191

https://doi.org/10.1145/3152434.3152441

Published: 30 November 2017 Publication History

Abstract

Recently, much attention has been devoted to the question of whether/when traditional network protocol design, which relies on the application of algorithmic insights by human experts, can be replaced by a data-driven (i.e., machine learning) approach. We explore this question in the context of the arguably most fundamental networking task: routing. Can ideas and techniques from machine learning (ML) be leveraged to automatically generate "good" routing configurations? We focus on the classical setting of intradomain traffic engineering. We observe that this context poses significant challenges for data-driven protocol design. Our preliminary results regarding the power of data-driven routing suggest that applying ML (specifically, deep reinforcement learning) to this context yields high performance and is a promising direction for further research. We outline a research agenda for ML-guided routing.

Supplementary Material

MP4 File (valadarsky.mp4)

Download
736.79 MB

References

[1]

DeepMind AI Reduces Google Data Centre Cooling Bill by 40%. https://goo.gl/QTdU2T.

[2]

IBM ILOG CPLEX Optimizer. http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/index.html.

[3]

M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat. Hedera: Dynamic flow scheduling for data center networks. In NSDI, 2010.

Digital Library

[4]

M. Alizadeh, A. Greenberg, D. A. Maltz, J. Padhye, P. Patel, B. Prabhakar, S. Sengupta, and M. Sridharan. Data Center TCP (DCTCP). ACM SIGCOMM, 2010.

Digital Library

[5]

M. Allalouf and Y. Shavitt. Maximum flow routing with weighted max-min fairness. In Quality of Service in the Emerging Networking Panorama.

[6]

D. Andersen, H. Balakrishnan, F. Kaashoek, and R. Morris. Resilient overlay networks. 2001.

Digital Library

[7]

Y. Azar, E. Cohen, A. Fiat, H. Kaplan, and H. Racke. Optimal oblivious routing in polynomial time. STOC, 2003.

Digital Library

[8]

D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. arXiv, 2014.

[9]

C. M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, 2006.

Digital Library

[10]

M. Chiesa, G. Rétvári, and M. Schapira. Lying your way to better traffic engineering. CoNEXT, 2016.

Digital Library

[11]

T. W. S. Chow and C. T. Leung. Nonlinear autoregressive integrated neural network model for short-term load forecasting. IEE Proceedings - Generation, Transmission and Distribution, 1996.

[12]

M. Dong, Q. Li, D. Zarchy, P. B. Godfrey, and M. Schapira. PCC: Re-architecting congestion control for consistent high performance. NSDI, 2015.

Digital Library

[13]

Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel. Benchmarking deep reinforcement learning for continuous control. ICML, 2016.

Digital Library

[14]

P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M. Kermarrec. The many faces of publish/subscribe. ACM Comput. Surv., 2003.

Digital Library

[15]

B. Fortz, J. Rexford, and M. Thorup. Traffic engineering with traditional ip routing protocols. Comm. Mag., 2002.

Digital Library

[16]

B. Fortz and M. Thorup. Increasing internet capacity using local search. Computational Optimization and Applications, 2004.

Digital Library

[17]

B. Fortz and M. Thorup. Optimizing ospf/is-is weights in a changing world. IEEE J.Sel. A. Commun., 2006.

Digital Library

[18]

M. Ghodbadi, R. Mahajan, A. Phanishayee, H. Rastegarfar, P.-A. Blanche, M. Glick, D. Kilper, J. Kulkarni, G. Ranade, and N. Devanur. ProjecToR: Agile Reconfigurable Datacenter Interconnect. SIGCOMM, 2016.

Digital Library

[19]

D. Halperin, S. Kandula, J. Padhye, P. Bahl, and D. Wetherall. Augmenting data center networks with multi-gigabit wireless links. SIGCOMM, 2011.

Digital Library

[20]

N. Hamedazimi, Z. Qazi, H. Gupta, V. Sekar, S. R. Das, J. P. Longtin, H. Shah, and A. Tanwer. Firefly: A reconfigurable wireless data center fabric using free-space optics. SIGCOMM, 2014.

Digital Library

[21]

K. Janocha and W. M. Czarnecki. On loss functions for deep neural networks in classification. CoRR, 2017.

[22]

W. S. Jewell. Multi-commodity Network Solutions. 1966.

[23]

J. Jiang, V. Sekar, I. Stoica, and H. Zhang. Unleashing the potential of data-driven networking. COMSNETS, 2017.

[24]

S. Kandula, D. Katabi, B. Davie, and A. Charny. Walking the tightrope: Responsive yet stable traffic engineering. 2005.

[25]

S. Kassing, A. Valadarsky, G. Shahaf, M. Schapira, and A. Singla. Augmenting data center networks with multi-gigabit wireless links. SIGCOMM, 2011.

Digital Library

[26]

S. Knight, H. Nguyen, N. Falkner, R. Bowden, and M. Roughan. The internet topology zoo. IEEE Journal on Selected Areas in Communications, 2011.

[27]

J. Kober, J. A. Bagnell, and J. Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 2013.

Digital Library

[28]

A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.

Digital Library

[29]

P. Kumar, Y. Yuan, C. Yu, N. Foster, R. D. Kleinberg, and R. Soulé. Kulfi: Robust traffic engineering using semi-oblivious routing. CoRR, 2016.

[30]

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.

[31]

M. Majer, C. Bobda, A. Ahmadinia, and J. Teich. Packet routing in dynamically changing networks on chip. In Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 3 - Volume 04, IPDPS '05, pages 154.2--, Washington, DC, USA, 2005. IEEE Computer Society.

Digital Library

[32]

H. Mao, M. Alizadeh, I. Menache, and S. Kandula. Resource management with deep reinforcement learning. HotNets, 2016.

Digital Library

[33]

H. Mao, R. Netravali, and M. Alizadeh. Neural adaptive bitrate streaming with pensive. SIGCOMM, 2017.

Digital Library

[34]

A. Medina, N. Taft, K. Salamatian, S. Bhattacharyya, and C. Diot. Traffic matrix estimation: Existing techniques and new directions. 2002.

Digital Library

[35]

N. Michael and A. Tang. Halo: Hop-by-hop adaptive link-state optimal routing. IEEE/ACM Transactions on Networking, 2015.

Digital Library

[36]

V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu. Asynchronous methods for deep reinforcement learning. ICML, 2016.

Digital Library

[37]

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. A. Riedmiller. Playing atari with deep reinforcement learning. CoRR, 2013.

[38]

V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 2015.

[39]

M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of Machine Learning. The MIT Press, 2012.

Digital Library

[40]

M. Roughan, A. Greenberg, C. Kalmanek, M. Rumsewicz, J. Yates, and Y. Zhang. Experience in measuring backbone traffic variability: Models, metrics, measurements and meaning. IMW, 2002.

Digital Library

[41]

J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz. Trust region policy optimization. ICML, 2015.

Digital Library

[42]

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms. ArXiv e-prints, 2017.

[43]

F. Shahrokhi and D. W. Matula. The maximum concurrent flow problem. J. ACM, 1990.

Digital Library

[44]

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 2016.

[45]

R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. MIT press Cambridge, 1998.

Digital Library

[46]

H. Wang and B. Li. Lube: Mitigating bottlenecks in wide area data analytics. HotCloud, 2017.

Digital Library

[47]

C. J. Watkins and P. Dayan. Q-learning. Machine Learning, 1992.

[48]

K. Winstein and H. Balakrishnan. Tcp ex machina: Computer-generated congestion control. SIGCOMM, 2013.

Digital Library

[49]

Y. Wu, E. Mansimov, S. Liao, R. B. Grosse, and J. Ba. Scalable trust-region method for deep reinforcement learning using kronecker-factored approximation. ArXiv e-prints, 2017.

[50]

D. Xu, M. Chiang, and J. Rexford. Link-state routing with hop-by-hop forwarding can achieve optimal traffic engineering. INFOCOM, 2008.

[51]

X. Zhou, Z. Zhang, Y. Zhu, Y. Li, S. Kumar, A. Vahdat, B. Y. Zhao, and H. Zheng. Mirror mirror on the ceiling: Flexible wireless links for data centers. SIGCOMM, 2012.

Digital Library

Cited By

Liu XZhao SCui YWang XSekar VYu MSeneviratne AVeitch D(2024)FIGRET: Fine-Grained Robustness-Enhanced Traffic EngineeringProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672258(117-135)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672258
AlQiam AYao YWang ZAhuja SZhang YRao SRibeiro BTawarmalani MSekar VYu MSeneviratne AVeitch D(2024)Transferable Neural WAN TE for Changing TopologiesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672237(86-102)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672237
Barzegar SRuiz MVelasco L(2024)Autonomous Flow Routing for Near Real-Time Quality of Service AssuranceIEEE Transactions on Network and Service Management10.1109/TNSM.2023.333920121:2(2504-2514)Online publication date: Apr-2024
https://doi.org/10.1109/TNSM.2023.3339201
Show More Cited By

Index Terms

Learning to Route
1. Computing methodologies
  1. Machine learning
2. Networks
  1. Network protocols
    1. Network layer protocols
      1. Routing protocols

Index terms have been assigned to the content through auto-classification.

Recommendations

Optimal Route Reflection Topology Design
LANC '18: Proceedings of the 10th Latin America Networking Conference

Autonomous Systems (ASes) exchange routing information about networks they can reach in the Internet, and the most widely extended way to connect them is by means of Border Gateway Protocol (BGP) sessions. ASes set up external BGP (eBGP) sessions ...
Designing optimal iBGP route-reflection topologies
NETWORKING'08: Proceedings of the 7th international IFIP-TC6 networking conference on AdHoc and sensor networks, wireless networks, next generation internet

The Border Gateway Protocol (BGP) is used today by all Autonomous Systems (AS) in the Internet. Inside each AS, iBGP sessions distribute the external routes among the routers. In large ASs, relying on a full-mesh of iBGP sessions between routers is not ...
Generating adaptive route instructions using hierarchical reinforcement learning
SC'10: Proceedings of the 7th international conference on Spatial cognition

We present a learning approach for efficiently inducing adaptive behaviour of route instructions. For such a purpose we propose a two-stage approach to learn a hierarchy of wayfinding strategies using hierarchical reinforcement learning. Whilst the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HotNets '17: Proceedings of the 16th ACM Workshop on Hot Topics in Networks

November 2017

206 pages

ISBN:9781450355698

DOI:10.1145/3152434

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Israeli Centers for Research Excellence

Conference

HotNets-XVI

Sponsor:

SIGCOMM

HotNets-XVI: The 16th ACM Workshop on Hot Topics in Networks

November 30 - December 1, 2017

CA, Palo Alto, USA

Acceptance Rates

HotNets '17 Paper Acceptance Rate 28 of 124 submissions, 23%;

Overall Acceptance Rate 110 of 460 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

136
Total Citations
View Citations
1,985
Total Downloads

Downloads (Last 12 months)144
Downloads (Last 6 weeks)18

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu XZhao SCui YWang XSekar VYu MSeneviratne AVeitch D(2024)FIGRET: Fine-Grained Robustness-Enhanced Traffic EngineeringProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672258(117-135)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672258
AlQiam AYao YWang ZAhuja SZhang YRao SRibeiro BTawarmalani MSekar VYu MSeneviratne AVeitch D(2024)Transferable Neural WAN TE for Changing TopologiesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672237(86-102)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672237
Barzegar SRuiz MVelasco L(2024)Autonomous Flow Routing for Near Real-Time Quality of Service AssuranceIEEE Transactions on Network and Service Management10.1109/TNSM.2023.333920121:2(2504-2514)Online publication date: Apr-2024
https://doi.org/10.1109/TNSM.2023.3339201
Di Cicco NIbrahimi MTroia STornatore M(2024)DeepLS: Local Search for Network Optimization Based on Lightweight Deep Reinforcement LearningIEEE Transactions on Network and Service Management10.1109/TNSM.2023.328743321:1(108-119)Online publication date: Feb-2024
https://doi.org/10.1109/TNSM.2023.3287433
Zhang JZeng CZhang HHu SChen K(2024)LiteFlow: Toward High-Performance Adaptive Neural Networks for Kernel DatapathIEEE/ACM Transactions on Networking10.1109/TNET.2023.329315232:1(627-642)Online publication date: Feb-2024
https://doi.org/10.1109/TNET.2023.3293152
He QWang YWang XXu WLi FYang KMa L(2024)Routing Optimization With Deep Reinforcement Learning in Knowledge Defined NetworkingIEEE Transactions on Mobile Computing10.1109/TMC.2023.323544623:2(1444-1455)Online publication date: Feb-2024
https://doi.org/10.1109/TMC.2023.3235446
Diao XGu HWei WJiang GLi B(2024)Deep Reinforcement Learning Based Dynamic Flowlet Switching for DCNIEEE Transactions on Cloud Computing10.1109/TCC.2024.338213212:2(580-593)Online publication date: Apr-2024
https://doi.org/10.1109/TCC.2024.3382132
Nougnanke BLoye JBaffier JFerlin SBruyere MLabit Y(2024)gPerfIsol: GNN-Based Rate-Limits Allocation for Performance Isolation in Multi-Tenant Cloud2024 27th Conference on Innovation in Clouds, Internet and Networks (ICIN)10.1109/ICIN60470.2024.10494419(194-201)Online publication date: 11-Mar-2024
https://doi.org/10.1109/ICIN60470.2024.10494419
Gu CSong XNg BXiang QGuo ZLi G(2024)An ML-Accelerated Framework for Large-Scale Constrained Traffic Engineering2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00014(47-58)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCS60910.2024.00014
Abrol AMohan PTruong-Huu T(2024)A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-Gen NetworksICC 2024 - IEEE International Conference on Communications10.1109/ICC51166.2024.10622726(465-471)Online publication date: 9-Jun-2024
https://doi.org/10.1109/ICC51166.2024.10622726
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents