TrieKV: Managing Values After KV Separation to Optimize Scan Performance in LSM-Tree
Pages 402 - 416
Abstract
Persistent key-value(KV) stores are mainly designed based on the Log-Structured Merge-tree(LSM-tree) for high write performance, yet the LSM-tree suffers from the inherently high I/O amplification which influences the read and write performance when KV stores grow in size. KV separation mitigates I/O amplification by storing only keys in the LSM-tree while values are in separated storage. However, the KV separation breaks the key sequence of values, which influences their range query performance. We propose TrieKV make the most of the hard-disk drives(HDD)’s sequential read performance advantages to improve range query performance. TrieKV uses a dynamic prefix index and a collaborative KV data merging and sorting mechanism to manage values after KV separation. Compared with the typical KV separation storage system WiscKey, TrieKV achieves range query performance under HDD. Meanwhile, TrieKV also performs better than WiscKey in all six YCSB workloads.
References
[1]
Pebblesdb: Building key-value stores using fragmented log-structured merge trees. In: The 26th Symposium, pp. 497–514 (2017)
[2]
Triad: Creating synergies between memory, disk and log in log structured key-value stores. In: Proceedings of the 2017 USENIX Conference on Usenix Annual Technical Conference (2017)
[3]
Chan, H.H.W., Li, Y., Lee, P.P.C., Xu, Y.: HashKV: enabling efficient updates in KV storage via hashing. In: Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference, p. 14 (2018)
[4]
Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM symposium on Cloud computing - SoCC 2010, Indianapolis, Indiana, USA, p. 143. ACM Press (2010)
[5]
Dai, Y., et al.: USENIX Assoc: From WiscKey to Bourbon: A Learned Index for Log-Structured Merge Trees, pp. 155–171 (2020)
[6]
Dayan, N., Athanassoulis, M., Idreos, S.: Optimal bloom filters and adaptive merging for LSM-trees. ACM Trans. Database Syst. 43(4) (2018).
[7]
Facebook: Rocksdb documentation. https://rocksdb.org.cn/doc.html
[8]
Google: Leveldb documentation. https://github.com/google/leveldb/blob/master/doc/index.md
[9]
Hu, X.Y., Eleftheriou, E., Haas, R., Iliadis, I., Pletka, R.: Write amplification analysis in flash-based solid state drives. In: Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference on - SYSTOR 2009, Haifa, Israel, p. 1. ACM Press (2009)
[10]
Huang, C., Hu, H., Wei, X., Qian, W., Zhou, A.: Partition pruning for range query on distributed log-structured merge-tree. Front. Comput. Sci. 14(3) (2020).
[11]
Im, J., Bae, J., Chung, C., Arvind, Lee, S.: Design of LSM-tree-based Key-value SSDs with Bounded Tails. ACM Trans. Storage 17(2) (2021).
[12]
Li, C., Chen, H., Ruan, C., Ma, X., Xu, Y.: Leveraging NVME SSDS for building a fast, cost-effective, LSM-tree-based KV store. ACM Trans. Storage 17(4) (2021).
[13]
Li, Y., Tian, C., Guo, F., Li, C., Xu, Y., USENIX Assoc: ElasticBF: elastic bloom filter with hotness awareness for boosting read performance in large key-value stores, pp. 739–752 (2019)
[14]
Li, Y., et al.: Differentiated Key-Value storage management for balanced I/O performance. In: 2021 USENIX Annual Technical Conference (USENIX ATC 21), pp. 673–687. USENIX Association (2021). https://www.usenix.org/conference/atc21/presentation/li-yongkun
[15]
Lu, K., Zhao, N., Wan, J., Fei, C., Zhao, W., Deng, T.: TridentKV: a read-optimized LSM-tree based KV Store via adaptive indexing and space-efficient partitioning. IEEE Trans. Parallel Distrib. Syst. 33(8), 1953–1966 (2022)., https://ieeexplore.ieee.org/document/9563237/
[16]
Lu L, Pillai TS, Gopalakrishnan H, Arpaci-Dusseau AC, and Arpaci-Dusseau RH Wisckey: separating keys from values in SSD-conscious storage ACM Trans. Storage (TOS) 2017 13 1 5
[17]
Lu Z, Cao Q, Mei F, Jiang H, and Li J A novel multi-stage forest-based key-value store for holistic performance improvement IEEE Trans. Parallel Distrib. Syst. 2020 31 4 856-870
[18]
Luo, S., et al.: A robust space-time optimized range filter for key-value stores, pp. 2071–2086 (2020).
[19]
Ouaknine, K., Agra, O., Guz, Z.: Optimization of RocksDB for Redis on Flash. In: Proceedings of the International Conference on Compute and Data Analysis - ICCDA 2017, Lakeland, FL, USA, pp. 155–161. ACM Press (2017)
[20]
O’Neil P, Cheng E, Gawlick D, and O’Neil E The log-structured merge-tree (LSM-tree) Acta Informatica 1996 33 4 351-385
[21]
Pan FF, Yue YL, and Xiong J dcompaction: speeding up compaction of the LSM-tree via delayed compaction J. Comput. Sci. Technol. 2017 32 1 41-54
[22]
Pugh, W.: Skip lists: a probabilistic alternative to balanced trees. In: Workshop on Algorithms & Data Structures (1990)
[23]
Ren, K., Zheng, Q., Arulraj, J., Gibson, G.: Slimdb: a space-efficient key-value storage engine for semi-sorted data. Proc. VLDB Endow. 10(13), 2037–2048 (2017).
[24]
Sun, X., Yu, J., Zhou, Z., Xue, C.J.: FPGA-based compaction engine for accelerating LSM-tree key-value stores. In: 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp. 1261–1272 (2020).
[25]
Yao, T., et al.: Matrixkv: reducing write stalls and write amplification in LSM-tree based KV stores with matrix container in NVM. In: The 2020 USENIX Annual Technical Conference, pp. 17–31 (2020)
[26]
Wang H, Yue Y, He S, and Wang W Zhang F, Zhai J, Snir M, Jin H, Kasahara H, and Valero M KT-store: a key-order and write-order hybrid key-value store with high write and range-query performance Network and Parallel Computing 2018 Cham Springer 64-76
[27]
Wang, Y., Jin, P., Wan, S.: HotKey-LSM: a hotness-aware LSM-tree for big data storage, pp. 5849–5851 (2020).
[28]
Wu, F., Yang, M., Zhang, B., Du, D., USENIX Assoc: AC-key: adaptive caching for LSM-based key-value stores, pp. 603–615 (2020)
[29]
Yao, T., Wan, J., Huang, P., He, X., Wu, F., Xie, C.: Building efficient key-value stores via a lightweight compaction tree. ACM Trans. Storage 13(4) (2017).
[30]
Yue Y, Wang W, Li Y, and He B Building an efficient put-intensive key-value store with skip-tree IEEE Trans. Parallel Distrib. Syst. 2017 23 961-973
[31]
Zhang, B., Du, D.H.C.: NVLSM: a persistent memory key-value store using log-structured merge tree with accumulative compaction. ACM Trans. Storage 17(3) (2021).
[32]
Zhang, H., et al.: Succinct range filters. ACM Trans. Database Syst. 45(2) (2020).
[33]
Zhang, Q., Li, Y., Lee, P.P.C., Xu, Y., Cui, Q., Tang, L.: UniKV: toward high-performance and scalable KV storage in mixed workloads via unified indexing. In: 2020 IEEE 36th International Conference on Data Engineering (ICDE), Dallas, TX, USA, pp. 313–324. IEEE (2020)
Recommendations
LSM-tree managed storage for large-scale key-value store
SoCC '17: Proceedings of the 2017 Symposium on Cloud ComputingKey-value stores are increasingly adopting LSM-trees as their enabling data structure in the backend storage, and persisting their clustered data through a file system. A file system is expected to not only provide file/directory abstraction to organize ...
MTDB: an LSM-tree-based key-value store using a multi-tree structure to improve read performance
AbstractTraditional LSM-tree-based key-value storage systems face significant read amplification issues due to the multi-level structure of LSM-tree, the unordered SSTable files in Level 0, and the lack of an in-memory index structure for key-value pairs. ...
FlatLSM: Write-Optimized LSM-Tree for PM-Based KV Stores
The Log-Structured Merge Tree (LSM-Tree) is widely used in key-value (KV) stores because of its excwrite performance. But LSM-Tree-based KV stores still have the overhead of write-ahead log and write stall caused by slow L0 flush and L0-L1 compaction. New ...
Comments
Information & Contributors
Information
Published In
Oct 2023
539 pages
ISBN:978-981-97-2386-7
DOI:10.1007/978-981-97-2387-4
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 28 April 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in