Chucky: A Succinct Cuckoo Filter for LSM-Tree

Modern persistent key-value stores rely on an LSM-tree in storage (SSD) to optimize writes and Bloom filters in memory (DRAM) to optimize reads. In this work, we show that the Bloom filters are creeping as a performance bottleneck. First, the costs of probing and constructing them grow with data size. Second, their costs are becoming more pronounced as storage access on modern SSDs is becoming less expensive relative to memory access. Recently, a new slew of data structures has emerged as an alternative to Bloom filters. They work by storing a fingerprint for every data entry within a compact hash table. We call them Fingerprint Filters (FFs). In this paper, we show how to scale an LSM-tree's memory bandwidth by replacing its Bloom filters with an FF that's augmented with every entry's location within the LSM-tree. However, we show that this new design does not scale out-of-the-box in terms of its false positive rate (FPR). The culprit is the auxiliary location information, which grows superlinarly with the data size, thus taking away bits from the fingerprints. By harnessing information theory and compression techniques, we show how to scale the size of this location information to keep the FPR small as the data grows. In this way, we show how to achieve the best of both worlds: scalable memory and storage bandwidth at the same time.