An Empirical Analysis on Memcached's Replacement Policies

The performance of large-scale web services heavily relies on the hit ratio of the key-value caches. One core component of a high-performance key-value cache is the replacement policy. A right replacement policy can help the caching system achieve a better hit ratio with no extra space cost, thereby improving the system’s throughput and end-to-end latency. Memcached and Redis are two widely used in-memory key-value caching software in many production systems. Both Memcached and Redis are simple to use and capable of ensuring end-to-end latency requirements for latency critical services. Memcached and Redis use different policies for cache replacement. In contrast, Memcached uses LRU or a variant of segmented-LRU (SegLRU) for replacement, while Redis uses KLRU, a random sampling-based LRU policy which evicts the LRU object from K randomly selected samples. This naturally leads to the question: “how does one compare to the other in actual production usage?” To answer the question, we implement the KLRU policy on Memcached. We evaluate the effectiveness of these three policies using both synthetic and actual production workloads. Our empirical analysis shows that, both SegLRU and KLRU outperform LRU in scalability for write-intensive workloads. However, despite the fact that SegLRU and KLRU are considerably different in terms of their heuristic and implementation, they yield very similar cache hit ratios, throughput, and scalability, with the random sampling-based LRU slightly winning over write-heavy workloads. KLRU also shows advantages in its simplicity in data structures and flexibility in adjusting the sampling size K to adapt to different workloads.