research-article

Lightweight Kubernetes Distributions: A Performance Comparison of MicroK8s, k3s, k0s, and Microshift

Authors:

Heiko Koziolek,

Nafise EskandaniAuthors Info & Claims

ICPE '23: Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering

Pages 17 - 29

https://doi.org/10.1145/3578244.3583737

Published: 15 April 2023 Publication History

Abstract

With containers becoming a prevalent method of software deployment, there is an increasing interest to use container orchestration frameworks not only in data centers, but also on resource-constrained hardware, such as Internet-of-Things devices, Edge gateways, or developer workstations. Consequently, software vendors have released several lightweight Kubernetes (K8s) distributions for container orchestration in the last few years, but it remains difficult for software developers to select an appropriate solution. Existing studies on lightweight K8s distribution performance tested only small workloads, showed inconclusive results, and did not cover recently released distributions. The contribution of this paper is a comparison of MicroK8s, k3s, k0s, and MicroShift, investigating their minimal resource usage as well as control plane and data plane performance in stress scenarios. While k3s and k0s showed by a small amount the highest control plane throughput and MicroShift showed the highest data plane throughput, usability, security, and maintainability are additional factors that drive the decision for an appropriate distribution.

References

[1]

Mohab Aly, Foutse Khomh, and Soumaya Yacout. 2018. Kubernetes or open-shift? which technology best suits eclipse hono iot deployments. In 2018 IEEE 11th Conference on Service-Oriented Computing and Applications (SOCA). IEEE, 113--120.

[2]

Marco Barletta, Marcello Cinque, Luigi De Simone, and Raffaele Della Corte. 2022. Introducing k4.0s: a model for mixed-criticality container orchestration in industry 4.0. arXiv preprint arXiv:2205.14188.

[3]

Victor R Basili. 1994. Goal question metric paradigm. Encyclopedia of software engineering, 528--532.

[4]

David Bernstein. 2014. Containers and cloud: from lxc to docker to kubernetes. IEEE cloud computing, 1, 3, 81--84.

[5]

Sebastian Böhm and Guido Wirtz. 2021. Profiling lightweight container platforms: microk8s and k3s in comparison to kubernetes. In ZEUS, 65--73.

[6]

Brendan Burns, Joe Beda, Kelsey Hightower, and Lachlan Evenson. 2022. Kubernetes: up and running. " O'Reilly Media, Inc.".

[7]

Simon Eismann, Cor-Paul Bezemer, Weiyi Shang, Duan Okanovic, and André van Hoorn. 2020. Microservices: a performance tester's dream or nightmare? In Proceedings of the ACM/SPEC International Conference on Performance Engineering, 138--149.

Digital Library

[8]

Halim Fathoni, Chao-Tung Yang, Chih-Hung Chang, and Chin-Yin Huang. 2019. Performance comparison of lightweight kubernetes in edge devices. In International Symposium on Pervasive Systems, Algorithms and Networks. Springer, 304--309.

[9]

Arnaldo Pereira Ferreira and Richard Sinnott. 2019. A performance evaluation of containers running on managed kubernetes services. In 2019 IEEE Inter- national Conference on Cloud Computing Technology and Science (CloudCom). IEEE, 199--208.

[10]

Tom Goethals, Filip De Turck, and Bruno Volckaert. 2019. Fledge: kubernetes compatible container orchestration on low-resource edge devices. In International Conference on Internet of Vehicles. Springer, 174--189.

[11]

Robert Heinrich, André Van Hoorn, Holger Knoche, Fei Li, Lucy Ellen Lwakatare, Claus Pahl, Stefan Schulte, and Johannes Wettinger. 2017. Performance engineering for microservices: research challenges and directions. In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, 223--226.

Digital Library

[12]

Anshul Jindal, Vladimir Podolskiy, and Michael Gerndt. 2019. Performance modeling for cloud microservice applications. In Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering, 25--32.

Digital Library

[13]

Paridhika Kayal. 2020. Kubernetes in fog computing: feasibility demonstration, limitations and improvement scope. In 2020 IEEE 6th World Forum on Internet of Things (WF-IoT). IEEE, 1--6.

[14]

Jeongchul Kim and Kyungyong Lee. 2019. Functionbench: a suite of workloads for serverless cloud function service. In 2019 IEEE 12th International Conference on Cloud Computing (CLOUD). IEEE, 502--504.

[15]

Antti Kivimaki. 2021. Evaluation of Lightweight Kubernetes Distributions in Edge Computing Context. Master's thesis. Tampere University.

[16]

Vojdan Kjorveziroski and Sonja Filiposka. 2022. Kubernetes distributions for the edge: serverless performance evaluation. The Journal of Supercomputing, 1--28.

Digital Library

[17]

Ritik Kumar and Munesh Chandra Trivedi. 2021. Networking analysis and performance comparison of kubernetes cni plugins. In Advances in Computer, Communication and Computational Sciences. Springer, 99--109.

[18]

Victor Medel, Rafael Tolosana-Calasanz, José Ángel Bañares, Unai Arronategui, and Omer F Rana. 2018. Characterising resource management performance in kubernetes. Computers & Electrical Engineering, 68, 286--297.

[19]

Sergii Telenyk, Oleksii Sopov, Eduard Zharikov, and Grzegorz Nowakowski. 2021. A comparison of kubernetes and kubernetes-compatible platforms. In 2021 11th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS). Vol. 1. IEEE, 313--317.

Digital Library

[20]

László Toka, Gergely Dobreff, Balázs Fodor, and Balázs Sonkoly. 2021. Machine learning-based scaling management for kubernetes edge clusters. IEEE Transactions on Network and Service Management, 18, 1, 958--972.

Digital Library

[21]

Olaf Zimmermann. 2017. Microservices tenets. Computer Science-Research and Development, 32, 3, 301--310.

Digital Library

Cited By

Kumari STulshyan VTewari H(2024)Cyber Security on the Edge: Efficient Enabling of Machine Learning on IoT DevicesInformation10.3390/info1503012615:3(126)Online publication date: 23-Feb-2024
https://doi.org/10.3390/info15030126
Sarda KNamrud ZLitoiu MShwartz LWatts Id'Amorim M(2024)Leveraging Large Language Models for the Auto-remediation of Microservice Applications: An Experimental StudyCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663855(358-369)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663855
Barletta MCinque MDe Simone LCorte R(2024)Criticality-aware Monitoring and Orchestration for Containerized Industry 4.0 EnvironmentsACM Transactions on Embedded Computing Systems10.1145/360456723:1(1-28)Online publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1145/3604567
Show More Cited By

Index Terms

Lightweight Kubernetes Distributions: A Performance Comparison of MicroK8s, k3s, k0s, and Microshift
1. Software and its engineering
  1. Software notations and tools
    1. Development frameworks and environments
      1. Software as a service orchestration system

Recommendations

Kubernetes Scheduling: Taxonomy, Ongoing Issues and Challenges
Continuous integration enables the development of microservices-based applications using container virtualization technology. Container orchestration systems such as Kubernetes, which has become the de facto standard, simplify the deployment of container-...
Performance Engineering Practices for Modern Industrial Applications at ABB Research
ICPE '23 Companion: Companion of the 2023 ACM/SPEC International Conference on Performance Engineering

ABB is developing a vast range of software services for process automation applications used in chemical production facilities, power plants, and container ships. High responsiveness and resource efficiency is important in this domain both for real-time ...
KOSMOS: Vertical and Horizontal Resource Autoscaling for Kubernetes
Service-Oriented Computing
Abstract
Cloud applications are increasingly executed onto lightweight containers that can be efficiently managed to cope with highly varying and unpredictable workloads. Kubernetes, the most popular container orchestrator, provides means to automatically ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPE '23: Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering

April 2023

244 pages

ISBN:9798400700682

DOI:10.1145/3578244

General Chairs:
Marco Vieira
University of Coimbra, Portugal
,
Valeria Cardellini
University of Rome Tor Vergata, Italy
,
Program Chairs:
Antinisca Di Marco
University of L'Aquila, Italy
,
Petr Tuma
Charles University, Czechia

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICPE '23

Sponsor:

ICPE '23: ACM/SPEC International Conference on Performance Engineering

April 15 - 19, 2023

Coimbra, Portugal

Acceptance Rates

ICPE '23 Paper Acceptance Rate 15 of 46 submissions, 33%;

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
658
Total Downloads

Downloads (Last 12 months)278
Downloads (Last 6 weeks)28

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kumari STulshyan VTewari H(2024)Cyber Security on the Edge: Efficient Enabling of Machine Learning on IoT DevicesInformation10.3390/info1503012615:3(126)Online publication date: 23-Feb-2024
https://doi.org/10.3390/info15030126
Sarda KNamrud ZLitoiu MShwartz LWatts Id'Amorim M(2024)Leveraging Large Language Models for the Auto-remediation of Microservice Applications: An Experimental StudyCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663855(358-369)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663855
Barletta MCinque MDe Simone LCorte R(2024)Criticality-aware Monitoring and Orchestration for Containerized Industry 4.0 EnvironmentsACM Transactions on Embedded Computing Systems10.1145/360456723:1(1-28)Online publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1145/3604567
Akbar MEsposito MHyrynsalmi SKumar KLcnarduzzi VLi XMehraj AMikkonen TMoreschini SMäkitalo NOivo MPaavonen AParveen RSmolander KSu RSystä KTaibi DYang NZhang ZZohaib M(2024)6GSoft: Software for Edge-to-Cloud Continuum2024 50th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA64295.2024.00082(499-506)Online publication date: 28-Aug-2024
https://doi.org/10.1109/SEAA64295.2024.00082
Ascensão PNeto LVelasquez KAbreu D(2024)Assessing Kubernetes Distributions: A Comparative Study2024 IEEE 22nd Mediterranean Electrotechnical Conference (MELECON)10.1109/MELECON56669.2024.10608706(832-837)Online publication date: 25-Jun-2024
https://doi.org/10.1109/MELECON56669.2024.10608706
Konieczny MMusiał SZieliński S(2024)Hybrid Serverless Processing in Environmental Monitoring - Building Digital Twin2024 IEEE 65th International Scientific Conference on Information Technology and Management Science of Riga Technical University (ITMS)10.1109/ITMS64072.2024.10741932(1-6)Online publication date: 3-Oct-2024
https://doi.org/10.1109/ITMS64072.2024.10741932
Koukis GSkaperas SKapetanidou IMamatas LTsaoussidis V(2024)Performance Evaluation of Kubernetes Networking Approaches across Constraint Edge Environments2024 IEEE Symposium on Computers and Communications (ISCC)10.1109/ISCC61673.2024.10733726(1-6)Online publication date: 26-Jun-2024
https://doi.org/10.1109/ISCC61673.2024.10733726
Koukis GSkaperas SKapetanidou IMamatas LTsaoussidis V(2024)Evaluating CNI Plugins Features & Tradeoffs for Edge Cloud Applications2024 IEEE Symposium on Computers and Communications (ISCC)10.1109/ISCC61673.2024.10733657(1-6)Online publication date: 26-Jun-2024
https://doi.org/10.1109/ISCC61673.2024.10733657
Koukis GSkaperas SKapetanidou ITsaoussidis VMamatas L(2024)An Open-Source Experimentation Framework for the Edge Cloud ContinuumIEEE INFOCOM 2024 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)10.1109/INFOCOMWKSHPS61880.2024.10620846(01-07)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOMWKSHPS61880.2024.10620846
Barletta MDe Simone LCorte RDi Martino C(2024)Failover Timing Analysis in Orchestrating Container-based Critical Applications2024 19th European Dependable Computing Conference (EDCC)10.1109/EDCC61798.2024.00026(81-84)Online publication date: 8-Apr-2024
https://doi.org/10.1109/EDCC61798.2024.00026
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten