PolarDB Serverless: A Cloud Native Database for Disaggregated Data Centers

The trend in the DBMS market is to migrate to the cloud for elasticity, high availability, and lower costs. The traditional, monolithic database architecture is difficult to meet these requirements. With the development of high-speed network and new memory technologies, disaggregated data center has become a reality: it decouples various components from monolithic servers into separated resource pools (e.g., compute, memory, and storage) and connects them through a high-speed network. The next generation cloud native databases are designed based on disaggregated data centers, disassembling the monolithic database architecture into independently extensible and composable components. This facilitates the construction of stateful services (e.g., memory pool and storagepool) as multi-tenant systems to achieve better resource utilization, elasticity, availability, and scalability.In this paper, we describe the novel architecture of PolarDB Serverless, which follows the disaggregation design paradigm: the CPU resource on compute nodes is decoupled from remote memory pool and storage pool, and each resource pool grows or shrinks independently, providing elasticity at multiple levels. Database processes on multiple compute nodes share cached pages in memory and persistent data in storage to amortize the cost. With this new architecture, different resources no longer share the same failure domain. The single point of failure in each component is handled independently, improving system reliability significantly. We design our system to mitigate the inherent penalty brought by resource disaggregation, and introduce optimizations such as one-sided RDMAverbs, optimistic locking, page materialization offloading, prefetching, and table scan pushdown. Compared to the architecture that uses local resources, PolarDB Serverless achieves better dynamic resource provisioning capabilities and 5.3 times faster failure recovery speed, while achieving comparable performance.