TTQR: A Traffic- and Thermal-Aware Q-Routing for 3D Network-on-Chip
Abstract
:1. Introduction
2. Related Work
2.1. Reactive Techniques
2.2. Proactive Techniques
3. Preliminaries
3.1. Q-Learning
3.2. Q-Routing
4. Traffic- and Thermal-Aware Q-Routing Algorithm (TTQR)
4.1. Routing Function
4.2. Selection Function
4.2.1. Q1-Table for Optimizing Latency
4.2.2. Q2-Table for Optimizing Temperature
4.3. Summary of TTQR
Algorithm 1: Sequential of the TTQR routing algorithm. |
|
5. Simulation Results and Discussion
5.1. Simulation Setup
5.2. Analysis of Network Performance
5.3. Analysis of Statistical Traffic Load Distribution (STLD)
5.4. Analysis of Temperature Distribution
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Zheng, H.; Wang, K.; Louri, A. Adapt-noc: A flexible network-on-chip design for heterogeneous manycore architectures. In Proceedings of the 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), Seoul, Korea, 27 February–3 March 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 723–735. [Google Scholar]
- Wang, Z.; Chen, X.; Li, C.; Guo, Y.; Liao, M.; Liu, Z. Load-balanced link distribution in mesh-based many-core systems. In Proceedings of the 21st IEEE International Conference on High Performance Computing and Communications, 17th IEEE International Conference on Smart City and 5th IEEE International Conference on Data Science and Systems, HPCC/SmartCity/DSS 2019, Zhangjiajie, China, 10–12 August 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1028–1034. [Google Scholar]
- Momeni, M.; Pozveh, A.J. An adaptive approximation method for traffic reduction in network on chip. In Proceedings of the 2020 6th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS), Mashhad, Iran, 23–12 December 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–5. [Google Scholar]
- Kim, D.; Yoo, S.; Lee, S. A network congestion-aware memory controller. In Proceedings of the 2010 Fourth ACM/IEEE International Symposium on Networks-on-Chip, Grenoble, France, 3–6 May 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 257–264. [Google Scholar]
- Gratz, P.; Grot, B.; Keckler, S.W. Regional congestion awareness for load balance in networks-on-chip. In Proceedings of the 2008 IEEE 14th International Symposium on High Performance Computer Architecture, Salt Lake City, UT, USA, 16–20 February 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 203–214. [Google Scholar]
- Badr, H.G.; Podar, S. An optimal shortest-path routing policy for network computers with regular mesh-connected topologies. IEEE Trans. Comput. 1989, 38, 1362–1371. [Google Scholar] [CrossRef]
- Feng, W.C.; Shin, K.G. Impact of selection functions on routing algorithm performance in multicomputer networks. In Proceedings of the 11th International Conference on Supercomputing, Vienna, Austria, 7–11 July 1997; pp. 132–139. [Google Scholar]
- Kim, J.; Park, D.; Theocharides, T.; Vijaykrishnan, N.; Das, C.R. A low latency router supporting adaptivity for on-chip interconnects. In Proceedings of the 42nd Design Automation Conference, Anaheim, CA, USA, 13–17 June; IEEE: Piscataway, NJ, USA, 2005; pp. 559–564. [Google Scholar]
- Ebrahimi, M.; Daneshtalab, M.; Liljeberg, P.; Plosila, J.; Tenhunen, H. Agent-based on-chip network using efficient selection method. In Proceedings of the 2011 IEEE/IFIP 19th International Conference on VLSI and System-on-Chip, Hong Kong, China, 3–5 October 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 284–289. [Google Scholar]
- Chen, K.C.; Lin, S.Y.; Hung, H.S.; Wu, A.Y.A. Topology-aware adaptive routing for nonstationary irregular mesh in throttled 3D NoC systems. IEEE Trans. Parallel Distrib. Syst. 2012, 24, 2109–2120. [Google Scholar] [CrossRef]
- Tedesco, L.P.; Rosa, T.; Clermidy, F.; Calazans, N.; Moraes, F.G. Implementation and evaluation of a congestion aware routing algorithm for networks-on-chip. In Proceedings of the 23rd Symposium on Integrated Circuits and System Design, São Paulo, Brazil, 6–9 September 2010; pp. 91–96. [Google Scholar]
- Farahnakian, F.; Ebrahimi, M.; Daneshtalab, M.; Liljeberg, P.; Plosila, J. Q-learning based congestion-aware routing algorithm for on-chip network. In Proceedings of the 2011 IEEE 2nd International Conference on Networked Embedded Systems for Enterprise Applications, Perth, Australia, 8–9 December 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 1–7. [Google Scholar]
- Liu, Y.; Guo, R.; Xu, C.; Weng, X.; Yang, Y. A Q-learning based fault-tolerant and congestion-aware adaptive routing algorithm for networks-on-chip. IEEE Embed. Syst. Lett. 2022. [Google Scholar] [CrossRef]
- Chao, C.H.; Chen, K.C.; Yin, T.C.; Lin, S.Y.; Wu, A.Y. Transport-layer-assisted routing for runtime thermal management of 3D NoC systems. ACM Trans. Embed. Comput. Syst. 2013, 13, 1–22. [Google Scholar] [CrossRef] [Green Version]
- Chen, K.C.; Kuo, C.C.; Hung, H.S.; Wu, A.Y.A. Traffic-and thermal-aware adaptive beltway routing for three dimensional network-on-chip systems. In Proceedings of the 2013 IEEE International Symposium on Circuits and Systems (ISCAS), Beijing, China, 19–23 May 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1660–1663. [Google Scholar]
- Chao, C.H.; Jheng, K.Y.; Wang, H.Y.; Wu, J.C.; Wu, A.Y. Traffic-and thermal-aware run-time thermal management scheme for 3D NoC systems. In Proceedings of the 2010 Fourth ACM/IEEE International Symposium on Networks-on-Chip, Hamburg, Germany, 24–25 September 2020; IEEE: Piscataway, NJ, USA, 2010; pp. 223–230. [Google Scholar]
- Wang, J.; Gu, H.; Wang, K.; Yang, Y.; Wang, K. DRTL: A heat-balanced deadlock-free routing algorithm for 3D topology network-on-chip. Microprocess. Microsystems 2016, 45, 95–104. [Google Scholar] [CrossRef] [Green Version]
- Dash, R.; Majumdar, A.; Pangracious, V.; Turuk, A.K.; Risco-Martin, J.L. ATAR: An adaptive thermal-aware routing algorithm for 3-D network-on-chip systems. IEEE Trans. Components, Packag. Manuf. Technol. 2018, 8, 2122–2129. [Google Scholar] [CrossRef]
- Lee, S.C.; Han, T.H. Q-function-based traffic-and thermal-aware adaptive routing for 3D network-on-chip. Electronics 2020, 9, 392. [Google Scholar] [CrossRef] [Green Version]
- Taheri, E.; Mohammadi, K.; Patooghy, A. ON–OFF: A reactive routing algorithm for dynamic thermal management in 3D NoCs. IET Comput. Digit. Tech. 2019, 13, 11–19. [Google Scholar] [CrossRef]
- Kuo, C.C.; Chen, K.C.; Chang, E.J.; Wu, A.Y. Proactive thermal-budget-based beltway routing algorithm for thermal-aware 3D NoC systems. In Proceedings of the 2013 International Symposium on System on Chip (SoC), Tampere, Finland, 22–24 October 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1–4. [Google Scholar]
- Lee, Y.S.; Hsin, H.K.; Chen, K.C.; Chang, E.J.; Wu, A.Y.A. Thermal-aware dynamic buffer allocation for proactive routing algorithm on 3D network-on-chip systems. In Proceedings of the 2014 International Symposium on VLSI Design, Automation and Test, Hsinchu, Taiwan, 28–30 April 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 1–4. [Google Scholar]
- Sivakumar, P.; Pandiaraj, K.; JeyaPrakash, K. Optimization of thermal aware multilevel routing for 3D IC. Analog. Integr. Circuits Signal Process. 2020, 103, 131–142. [Google Scholar] [CrossRef]
- Cao, K.; Zhou, J.; Wei, T.; Chen, M.; Hu, S.; Li, K. A survey of optimization techniques for thermal-aware 3D processors. J. Syst. Archit. 2019, 97, 397–415. [Google Scholar] [CrossRef]
- Li, W.; He, C.; Fu, H.; Zheng, J.; Dong, R.; Xia, M.; Yu, L.; Luk, W. A real-time tree crown detection approach for large-scale remote sensing images on FPGAs. Remote. Sens. 2019, 11, 1025. [Google Scholar] [CrossRef] [Green Version]
- Zhang, X.; Liu, R.; Ren, J.; Gui, Q. Adaptive fractional image enhancement algorithm based on rough set and particle swarm optimization. Fractal Fract. 2022, 6, 100. [Google Scholar] [CrossRef]
- Chawra, V.K.; Gupta, G.P. Optimization of the wake-up scheduling using a hybrid of memetic and tabu search algorithms for 3D-wireless sensor networks. Int. J. Softw. Sci. Comput. Intell. 2022, 14, 1–18. [Google Scholar] [CrossRef]
- Al-Ayyoub, M.; AlZu’bi, S.; Jararweh, Y.; Shehab, M.A.; Gupta, B.B. Accelerating 3D medical volume segmentation using GPUs. Multimed. Tools Appl. 2018, 77, 4939–4958. [Google Scholar] [CrossRef]
- Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
- Watkins, C.J.; Dayan, P. Q-learning. Mach. Learn. 1992, 8, 279–292. [Google Scholar] [CrossRef]
- Zhang, J.X.; Yang, G.H. Low-complexity tracking control of strict-feedback systems with unknown control directions. IEEE Trans. Autom. Control. 2019, 64, 5175–5182. [Google Scholar] [CrossRef]
- Zhang, X.; Dai, L. Image enhancement based on rough set and fractional Order differentiator. Fractal Fract. 2022, 6, 214. [Google Scholar] [CrossRef]
- Shahabinejad, N.; Beitollahi, H. Q-thermal: A Q-learning-based thermal-aware routing algorithm for 3-D network on-chips. IEEE Trans. Components Packag. Manuf. Technol. 2020, 10, 1482–1490. [Google Scholar] [CrossRef]
- Jheng, K.Y.; Chao, C.H.; Wang, H.Y.; Wu, A.Y. Traffic-thermal mutual-coupling co-simulation platform for three-dimensional network-on-chip. In Proceedings of the 2010 International Symposium on VLSI Design, Automation and Test, Hsin Chu, Taiwan, 26–29 April 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 135–138. [Google Scholar]
State-Space | Action-Space | Q-Value | Goal | ||||
---|---|---|---|---|---|---|---|
s | a | d | |||||
Current-Router | Next-Router | Output-Port | Latency | Destina tion-Router | |||
Node-4 | 1 | 3 | South | West | Node-0 | ||
Node-4 | 1 | - | South | - | - | Node-1 | |
Node-4 | 1 | 5 | South | East | Node-2 | ||
Node-4 | 3 | - | West | - | - | Node-3 | |
Node-4 | 4 | - | Local | - | - | - | Node-4 |
Node-4 | 5 | - | East | - | - | Node-5 | |
Node-4 | 3 | 7 | West | North | Node-6 | ||
Node-4 | 7 | - | North | - | - | Node-7 | |
Node-4 | 5 | 7 | East | North | Node-8 |
Action | Q1-Value |
---|---|
Output-Port | |
North | A |
East | B |
South | C |
West | D |
Action | Q2-Value |
---|---|
Output-Port | |
North | |
East | |
South | |
West |
Parameter | Value |
---|---|
Packet size | 8 flits |
Buffer size | 16 flits |
Simulation time | cycles |
Warm-up time | cycles |
Mesh size | |
Traffic pattern | random, shuffle, bit-reversal |
Ambient temperature | 25 °C |
Routing algorithm | TAAR, TTQR |
Random | Shuffle | Bit-Reversal | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
TAAR | Q1 | Q2 | TTQR | TAAR | Q1 | Q2 | TTQR | TAAR | Q1 | Q2 | TTQR | |
Mean(°C) | 96.309 | 96.416 | 96.564 | 96.500 | 89.665 | 89.934 | 89.885 | 89.926 | 92.610 | 94.556 | 94.550 | 94.337 |
S.D.(°C) | 6.043 | 6.449 | 5.421 | 5.828 | 5.535 | 5.552 | 5.495 | 5.667 | 6.616 | 7.136 | 7.869 | 7.375 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, H.; Chen, X.; Zhao, Y.; Li, C.; Lu, J. TTQR: A Traffic- and Thermal-Aware Q-Routing for 3D Network-on-Chip. Sensors 2022, 22, 8721. https://doi.org/10.3390/s22228721
Liu H, Chen X, Zhao Y, Li C, Lu J. TTQR: A Traffic- and Thermal-Aware Q-Routing for 3D Network-on-Chip. Sensors. 2022; 22(22):8721. https://doi.org/10.3390/s22228721
Chicago/Turabian StyleLiu, Hanyan, Xiaowen Chen, Yunping Zhao, Chen Li, and Jianzhuang Lu. 2022. "TTQR: A Traffic- and Thermal-Aware Q-Routing for 3D Network-on-Chip" Sensors 22, no. 22: 8721. https://doi.org/10.3390/s22228721