Abstract
As Internet usage increases, new, smarter, networking methods are required to enhance or maintain Quality of Service (QoS). One method, Software-Defined Networking (SDN) offers many advantages by separating the Forwarding and Control Planes. However, heuristic routing algorithms employed by SDN, such as Shortest Path, are not always suited for QoS-based pathfinding. This paper introduces a new Q-Routing algorithm that separates training and pathfinding, utilising two network metrics - latency and bandwidth - instead of latency alone. Two versions of this algorithm are employed, a static and a dynamic version where additional re-training is undertaken to allow Q-Routing to adapt to changing network environments. Both are tested on different size mesh topologies. The results show that static and dynamic Q-Routing are faster at pathfinding compared to K-Shortest Path and on average, find equally good routes.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cisco: VNI Complete Forecast Highlights (2021). https://www.cisco.com/c/m/en_us/solutions/service-provider/vni-forecast-highlights.html
Teh, Y.-C.: Lockdown Leads to Surge in TV Screen Time and Streaming (2020). https://www.ofcom.org.uk/about-ofcom/latest/features-and-news/lockdown-leads-to-surge-in-tv-screen-time-and-streaming
Bellman, R.: On a Routing Problem. Q. Appl. Maths. 16, 87–90 (1958)
Ford, L.: Network flow theory. Technical report. Santa Monica, USA (1956)
Dijkstra, E.: A note on two problems in connections with graphs. Numer. Math. (1959)
Watkins, C.J.C.H.: Learning from delayed rewards. Ph.D. thesis, Cambridge, UK (1989). http://www.cs.rhul.ac.uk/~chrisw/new_thesis.pdf
While, R.: Artificial intelligence: reinforcement learning (2019). http://teaching.csse.uwa.edu.au/courses/CITS4211/Lectures/wk7.pdf
Huang, Y., Tan, T., Wang, N., Chen, Y., Li, Y.: Resource allocation For D2D communications with a novel distributed Q-learning algorithm in heterogeneous networks. In: International Conference on Machine Learning and Cybernetics (ICMLC), pp. 533–537. IEEE (2018). https://doi.org/10.1109/ICMLC.2018.8526955
Wen, S., Chen, J., Li, Z., Rad, A.B.: Fuzzy Q-learning obstacle avoidance algorithm of humanoid robot in unknown environment. In: 37th Chinese Control Conference (CCC), pp. 5186–5190. IEEE (2018)
Boyan, J., Littman, M.: Packet routing in dynamically changing networks: a reinforcement learning approach. In: Proceedings of 6th International Conference on Neural Information Processing Systems, Denver, Colorado, USA, pp. 671–678 (1993)
Kavalerov, A., Shilova, Y., Likhacheva, Y.: Adaptive Q-routing with random echo and route memory. In: 20th Conference of Open Innovations Association (FRUCT), pp. 138–145. IEEE (2018)
Kumar, S., Miikkulainen, R.: Dual reinforcement Q-routing: an on-line adaptive routing algorithm. In: Smart Engineering Systems: Neural Networks, Fuzzy Logic, Data Mining, and Evolutionary Programming, USA (1997). http://nn.cs.utexas.edu/?kumar:annie97
Chavula, J., Densmore, M., Suleman, H.: Using SDN and reinforcement learning for traffic engineering in UbuntuNet alliance. In: International Conference on Advances in Computing and Communication Engineering (ICACCE), pp. 349–355. IEEE, South Africa (2016)
Jalil, S., Rehmani, M., Chalup, H.: DQR: deep Q-routing in software defined networks. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2020)
Kreutz, D., Ramos, F., Veríssimo, P., Rothenberg, C.: Software-defined networking: a comprehensive survey. Proc. IEEE 103, 14–76 (2015)
Du, J., Huang, X., Wu, F., Leng, S.: Reinforcement learning empowered QoS-aware adaptive Q-routing in ad-hoc networks. In: 2020 International Wireless Communications and Mobile Computing (IWCMC), pp. 551–556. IEEE (2020)
Harewood-Gill, D., Martin, T., Nejabati, R.: The performance of Q-learning within SDN controlled static and dynamic mesh networks. In: 2020 6th IEEE Conference on Network Softwarization (NetSoft), pp. 185–189 (2020)
Ryu: Component-Based Software Defined Networking Framework (2021). https://ryu-sdn.org/
Grotto Networking: SDN Fun (2019). https://www.grotto-networking.com/SDNfun.html
Hong, C., et al.: Achieving high utilization with software-driven WAN. In: ACM SIGCOMM Computer Communication Review, pp. 15–26. IEEE (2013)
Acknowledgements
We thank the Engineering and Physical Sciences Research Council Centre for Doctoral Training in Communications (EP/I028153/1 and EP/L016656/1) and Roke Manor for financial assistance and support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Harewood-Gill, D., Martin, T., Nejabati, R. (2022). Q-Routing Using Multiple QoS Metrics in SDN. In: Jansen, T., Jensen, R., Mac Parthaláin, N., Lin, CM. (eds) Advances in Computational Intelligence Systems. UKCI 2021. Advances in Intelligent Systems and Computing, vol 1409. Springer, Cham. https://doi.org/10.1007/978-3-030-87094-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-87094-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87093-5
Online ISBN: 978-3-030-87094-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)