MDB-KCP: persistence framework of in-memory database with CRIU-based container checkpoint in Kubernetes
Authors: 
Jeongmin Lee, Hyeongbin Kang, Hyeon-jin Yu, Ji-Hyun Na, Jungbin Kim, Jae-hyuck Shin, Seo-Young NohAuthors Info & Claims
Jeongmin Lee, Hyeongbin Kang, Hyeon-jin Yu, Ji-Hyun Na, Jungbin Kim, Jae-hyuck Shin, Seo-Young NohAuthors Info & ClaimsAbstract
As the demand for container technology and platforms increases due to the efficiency of IT resources, various workloads are being containerized. Although there are efforts to integrate various workloads into Kubernetes, the most widely used container platform today, the nature of containers makes it challenging to support persistence for memory-centric workloads like in-memory databases. In this paper, we discuss the drawbacks of one of the persistence support methods used for in-memory databases in a Kubernetes environment, namely, the data snapshot. To address these issues, we propose a compromise solution of using container checkpoints. Through this approach, we can perform checkpointing without incurring additional memory usage due to CoW, which is a problem in fork-based data snapshots during snapshot creation. Additionally, container checkpointing induces up to 7.1 times less downtime compared to the main process-based data snapshot. Furthermore, during database recovery, it is possible to achieve up to 11.3 times faster recovery compared to the data snapshot method.
References
[1]
Beloglazov A, Buyya R, Lee YC, and Zomaya A A taxonomy and survey of energy-efficient data centers and cloud computing systems Adv Comput 2011 82 47-111
[2]
Pelletingeas C Performance evaluation of virtualization with cloud computing (Doctoral dissertation) 2010
[3]
Li Z, Kihl M, Lu Q, Andersson JA (2017) Performance overhead comparison between hypervisor and container based virtualization. In: 2017 IEEE 31st International Conference on advanced information networking and applications (AINA). Taipei, IEEE, p 955–962
[4]
CNCF (2020) CNCF SURVEY 2020. https://www.cncf.io/wp-content/uploads/2020/11/CNCF_Survey_Report_2020.pdf. Accessed 14 Feb 2024
[5]
Rodriguez MA and Buyya R Container-based cluster orchestration systems: a taxonomy and future directions Softw Pract Exp 2019 49 5 698-719
[6]
Bergman K, Borkar S, Campbell D, Carlson W, Dally W, Denneau M et al (2008) Exascale computing study: technology challenges in achieving exascale systems. In: Defense Advanced Research Projects Agency Information Processing Techniques Office (DARPA IPTO), Tech. Rep, 15, 181
[7]
Abourezq M, Idrissi A (2016) Database-as-a-service for big data: an overview. Int J Adv Comput Sci Appl 7(1):157–177
[8]
CRIU. Checkpoint/Restore in Userspace. https://criu.org/Main_Page. Accessed 10 Dec 2023
[9]
cri-o. Lightweight container runtime for Kubernetetes. https://cri-o.io. Accessed 10 Feb 2024
[10]
Zhang H, Chen G, Ooi BC, Tan KL, and Zhang M In-memory big data management and processing: a survey IEEE Trans Knowl Data Eng 2015 27 7 1920-1948
[11]
Magalhaes A, Monteiro JM, and Brayner A Main memory database recovery: a survey ACM Comput Surv (CSUR) 2021 54 2 1-36
[12]
Bao X, Liu L, Xiao N, Lu Y, Cao W (2016) Persistence and recovery for in-memory NoSQL services: a measurement study. In: 2016 IEEE International Conference on Web Services (ICWS). San Francisco, IEEE, p 530–537
[13]
Redis. Redis persistence, how Redis writes data to disk. https://redis.io/docs/management/persistence. Accessed 3 Jan 2024
[14]
Park J, Lee Y, Yeom HY, Son Y (2020) Memory efficient fork-based checkpointing mechanism for in-memory database systems. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing. p 420–427
[15]
EDUCBA (2023) Redis persistence. https://www.educba.com/redis-persistence. Accessed 10 Jan 2024
[16]
Shrestha R High availability and performance of database in the cloud-traditional master-slave replication versus modern cluster-based solutions 2017
[17]
CRIU. Checkpoint/Restore. https://criu.org/Checkpoint/Restore. Accessed 10 Dec 2023
[18]
Li L, Wang G, Wu G, Yuan Y (2018) Consistent snapshot algorithms for in-memory database systems: experiments and analysis. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE). Paris, IEEE, p 1284–1287
[19]
Lee K, Kim H, and Yeom HY Validity tracking based log management for in-memory databases IEEE Access 2021 9 111493-111504
[20]
Bhardwaj A, Gupta U, Budhiraja I, Chaudhary R (2023) Container-based migration technique for fog computing architecture. In: 2023 international conference for advancement in technology (ICONAT). Goa, India, IEEE, p 1–6
[21]
Tran MN, Vu XT, and Kim Y Proactive stateful fault-tolerant system for kubernetes containerized services IEEE Access 2022 10 102181-102194
[22]
Kubernetes (2023) Installing kubeadm. https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm. Accessed 10 Dec 2023
[23]
Cooper BF, Silberstein A, Tam E, Ramakrishnan R, Sears R (2010) Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM symposium on Cloud computing. p 143–154
[24]
docker docs. docker checkpoint. https://docs.docker.com/engine/reference/commandline/checkpoint. Accessed 8 Jan 2024
[25]
Reber A (2022) Forensic container checkpointing in Kubernetes. https://kubernetes.io/blog/2022/12/05/forensic-container-checkpointing-alpha. Accessed 11 Nov 2023
[26]
MDB-KCP. Available: https://github.com/CBNU-DCLab/MDB-KCP. Accessed 20 May 2024
Index Terms
- MDB-KCP: persistence framework of in-memory database with CRIU-based container checkpoint in Kubernetes
Index terms have been assigned to the content through auto-classification.
Recommendations
State machine replication in containers managed by Kubernetes
Computer virtualization brought fast resource provisioning to data centers and the deployment of pay-per-use cost models. The system virtualization provided by containers like Docker has improved this flexibility of resource provisioning. Applications ...
Container stowage pre-planning: using search to generate solutions, a case study
Container-ships are vessels possessing an internal structure that facilitates the handling of containerised cargo. At each port along the vessel's journey, containers destined for those ports are unloaded and additional containers destined for ...
Research on the Integrated Management of Container Terminal Based on Radio Frequency Identification Technology
ICMTMA '11: Proceedings of the 2011 Third International Conference on Measuring Technology and Mechatronics Automation - Volume 03With the rapid development of port logistics, it sets a higher request to the management level of the container terminal. In view of the terminal issues of the tedious process of manually collecting container and truck information with higher error rate,...
Comments
Information & Contributors
Information
Published In
© The Author(s) 2024.
Publisher
Hindawi Limited
London, United Kingdom
Publication History
Published: 25 July 2024
Accepted: 09 July 2024
Received: 02 April 2024
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Jan 2025