Abstract

The memory over optical network (MONet) system is a disaggregated data center architecture where serial (HMC)/parallel (DDR4) memory resources can be accessed over optically switched interconnects within and between racks. An FPGA/ASIC-based custom hardware IP (ReMAT) supports heterogeneous memory pools, accommodates optical-to-electrical conversion for remote access, performs the required serial/parallel conversion, and hosts the necessary local memory controller. An optically interconnected HMC-based (serial I/O type) memory card is accessed by a memory controller embedded in the compute card, simplifying the hardware near the memory modules. This substantially reduces overheads on latency, cost, power consumption, and space. We characterize CPU–memory performance by experimentally demonstrating the impact of distance, number of switching hops, transceivers, channel bonding, and bit rate per transceiver on the bit error rate, power consumption, additional latency, sustained remote memory bandwidth/throughput (using industry standard benchmark STREAMS), and cloud workload performance (such as operations per second, average added latency, and retired instructions per second memcached with YCSB cloud workloads). MONet pushes the CPU–memory operational limit from a few centimeters to tens of meters, yet applications can experience as low as 10% performance penalty (at 36 m) compared to a direct-attached equivalent. Using the proposed parallel topology, a system can support up to 100,000 disaggregated cards.

© 2021 Optical Society of America

Optically Disaggregated Data Centers With Minimal Remote Memory Latency: Technologies, Architectures, and Resource Allocation [Invited]

Georgios Zervas, Hui Yuan, Arsalan Saljoghei, Qianqiao Chen, and Vaibhawa Mishra
J. Opt. Commun. Netw. 10(2) A270-A285 (2018)

DACON: a reconfigurable application-centric optical network for disaggregated data center infrastructures [Invited]

Xiaotao Guo, Xuwei Xue, Fulong Yan, Bitao Pan, Georgios Exarchakos, and Nicola Calabretta
J. Opt. Commun. Netw. 14(1) A69-A80 (2022)

In Compute/Memory Dynamic Packet/Circuit Switch Placement for Optically Disaggregated Data Centers

Adaranijo Peters, George Oikonomou, and Georgios Zervas
J. Opt. Commun. Netw. 10(7) B164-B178 (2018)

RDON: a rack-scale disaggregated data center network based on a distributed fast optical switch

Xiaotao Guo, Fulong Yan, Jingyan Wang, George Exarchakos, Yuanyuan Peng, Xuwei Xue, Bitao Pan, and Nicola Calabretta
J. Opt. Commun. Netw. 12(8) 251-263 (2020)

Optics enabled networks and architectures for data center cost and power efficiency [Invited]

Marc Taubenblatt, Pavlos Maniotis, and Asser Tantawi
J. Opt. Commun. Netw. 14(1) A41-A49 (2022)