Towards Latency-Aware Linux Scheduling for Serverless Workloads

A key principle in the design of the Linux kernel's Completely Fair Scheduler (CFS) is fairness: all running tasks receive a minimum time slice during every scheduling period, ensuring that none starve. However, this may lead to a significant number of context switches when a server is overloaded with a large number of colocated tasks, which may cause significant degradation in server performance. Unfortunately, this situation is exactly what we found when hosting serverless-style workloads which typically consist of a large number of short-lived, CPU-bound functions sharing resources. We propose modifying the Linux CFS to mitigate this problem by giving priority to the long tail of least loaded functions. These are the functions which are mostly idle and only run occasionally for a short while after being triggered unexpectedly. The large number of such functions in serverless environments means that prioritising them helps drain contended CPU run queues, reducing the total overhead due to context switching, thereby improving the performance not only of the prioritised functions but other functions as well. We implement this policy in the Linux kernel scheduler and demonstrate how it integrates well with Knative, an open source Kubernetes-based serverless framework. Given contention scenarios synthesised from real-world traces, our modified CFS can introduce a 5--30% increase in attainment of latency targets.