Userland CO-PAGER: boosting data-intensive applications with non-volatile memory, userspace paging

With the emergence of low-latency non-volatile memory (NVM) storage, the software overhead, incurred by the operating system, becomes more prominent. The Linux (monolithic) kernel, incorporates a complex I/O subsystem design, using redundant memory copies and expensive user/kernel context switches to perform I/O. Memory-mapped I/O, which internally uses demand paging, has recently become popular when paired with low-latency storage. It improves I/O performance by mapping the data DMA transfers directly to userspace memory and removing the additional data copy between user/kernel space. However, for data-intensive applications, when there is insufficient physical memory, frequent page faults can still trigger expensive mode switches and I/O operations. To tackle this problem, we propose CO-PAGER, which is a lightweight userspace memory service. CO-PAGER consists of a minimal kernel module and a userspace component. The userspace component handles (redirected) page faults, performs memory management and I/O operations and accesses NVM storage directly. The kernel module is used to update memory mapping between user and kernel space. In this way CO-PAGER can bypass the deep kernel I/O stacks and provide a flexible/customizable and efficient memory paging service in userspace. We provide a general programming interface to use the CO-PAGER service. In our experiments, we also demonstrate how the CO-PAGER approach can be applied to a MapReduce framework and improves performance for data-intensive applications.