research-article

Open access

Charliecloud: unprivileged containers for user-defined software stacks in HPC

Authors:

Reid Priedhorsky,

Tim RandlesAuthors Info & Claims

SC '17: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 36, Pages 1 - 10

https://doi.org/10.1145/3126908.3126925

Published: 12 November 2017 Publication History

Abstract

Supercomputing centers are seeing increasing demand for user-defined software stacks (UDSS), instead of or in addition to the stack provided by the center. These UDSS support user needs such as complex dependencies or build requirements, externally required configurations, portability, and consistency. The challenge for centers is to provide these services in a usable manner while minimizing the risks: security, support burden, missing functionality, and performance. We present Charliecloud, which uses the Linux user and mount namespaces to run industry-standard Docker containers with no privileged operations or daemons on center resources. Our simple approach avoids most security risks while maintaining access to the performance and functionality already on offer, doing so in just 800 lines of code. Charliecloud promises to bring an industry-standard UDSS user workflow to existing, minimally altered HPC resources.

References

[1]

Amazon Web Services, Inc. 2015. An introduction to high performance computing on AWS. White paper. Amazon Web Services, Inc. https://d0.awsstatic.com/whitepapers/Intro_to_HPC_on_AWS.pdf

[2]

Evan Andersen. 2016. How Nvidia breaks Chrome incognito. (Jan. 2016). https://charliehorse55.wordpress.com/2016/01/09/how-nvidia-breaks-chrome-incognito/

[3]

Diego Calleja. 2013. Linux 3.8. (April 2013). http://kernelnewbies.org/Linux_3.8

[4]

Hao Chen, David Wagner, and Drew Dean. 2002. Setuid Demystified. In USENIX Security Symposium. http://crypto.stanford.edu/cs155/papers/setuid-usenix02.pdf

Digital Library

[5]

CoreOS Inc. 2017. rkt 1.25.0 documentation. (2017). https://coreos.com/rkt/docs/1.25.0/

[6]

Docker, Inc. 2016. Dockerfile reference. Documentation. Docker, Inc. https://docs.docker.com/engine/reference/builder/

[7]

Docker Inc. 2017. Docker Docs. Documentation. Docker, Inc. https://docs.docker.com

[8]

John L. Furlani and Peter W. Osel. 1996. Abstract yourself with modules. In USENIX System Administration Conference. http://modules.sourceforge.net/docs/absmod.pdf

Digital Library

[9]

Tyler Hicks. 2017. CVE-2017-7184: kernel: Local privilege escalation in XFRM framework. (March 2017). http://seclists.org/oss-sec/2017/q1/689

[10]

Solomon Hykes. 2015. Introducing runC: A lightweight universal container runtime. (June 2015). https://blog.docker.com/2015/06/runc

[11]

Intel Corporation 2016. Intel® MPI benchmarks: User guide and methodology description. Documentation. Intel Corporation. https://software.intel.com/sites/default/files/managed/66/e8/IMB_Users_Guide.pdf

[12]

Keith R. Jackson, Lavanya Ramakrishnan, Krishna Muriki, Shane Canon, Shreyas Cholia, John Shalf, Harvey J. Wasserman, and Nicholas J. Wright. 2010. Performance analysis of high performance computing applications on the Amazon Web Services cloud. In IEEE CloudCom.

Digital Library

[13]

Doug Jacobsen and Shane Canon. 2015. Contain this, unleashing Docker for HPC. (May 2015). http://www.nersc.gov/assets/Uploads/nersc-brownbag-docker-jacobsen-canon.pdf

[14]

Douglas M. Jacobsen and Richard Shane Canon. 2015. Contain this, unleashing Docker for HPC. In Cray User Group. http://www.nersc.gov/assets/Uploads/cug2015udi.pdf

[15]

Venkateswararao Jujjuri, Eric Van Hensbergen, Anthony Liguori, and Badari Pulavarty. 2010. VirtFS---a virtualization aware file system pass-through. In Ottawa Linux Symposium (OLS). https://www.kernel.org/doc/ols/2010/ols2010-pages-109-120.pdf

[16]

Michael Kerrisk. 2013. Namespaces in operation, part 1: Namespaces overview. Linux Weekly News (Jan. 2013). https://lwn.net/Articles/531114/

[17]

Michael Kerrisk. 2013. Namespaces in operation, part 5: User namespaces. Linux Weekly News (Feb. 2013). https://lwn.net/Articles/532593/

[18]

Michael Kerrisk et al. 2015. pid_namespaces(7). Man page. http://man7.org/linux/man-pages/man7/pid_namespaces.7.html

[19]

Michael Kerrisk et al. 2016. chroot(2). Man page. http://man7.org/linux/man-pages/man2/chroot.2.html

[20]

Michael Kerrisk et al. 2016. clone(2). Man page. http://man7.org/linux/man-pages/man2/clone.2.html

[21]

Michael Kerrisk et al. 2016. namespaces(7). Man page. http://man7.org/linux/man-pages/man7/namespaces.7.html

[22]

Michael Kerrisk et al. 2016. setns(2). Man page. http://man7.org/linux/man-pages/man2/setns.2.html

[23]

Michael Kerrisk et al. 2016. unshare(2). Man page. http://man7.org/linux/man-pages/man2/unshare.2.html

[24]

Michael Kerrisk et al. 2016. user_namespaces(7). Man page. http://man7.org/linux/man-pages/man7/user_namespaces.7.html

[25]

Gregory M. Kurtzer. 2016. Singularity. (July 2016). http://singularity.lbl.gov/

[26]

Ning Liu, Jason Cope, Philip Carns, Christopher Carothers, Robert Ross, Gary Grider, Adam Crume, and Carlos Maltzahn. 2012. On the role of burst buffers in leadership-class storage systems. In Mass Storage Systems and Technologies (MSST).

[27]

Scott Lowe. 2009. What is SR-IOV? (Dec. 2009). http://blog.scottlowe.org/2009/12/02/what-is-sr-iov/

[28]

Doug McIlroy, E. N. Pinson, and B. A. Tague. 1978. UNIX time-sharing system: Foreword. Bell System Technical Journal 67, 6 (1978).

[29]

Open Container Initiative 2016. About. Mission statement. Open Container Initiative. https://www.opencontainers.org/about

[30]

Larry Pezzaglia. 2012. CHOS in production. (April 2012). https://www.nersc.gov/assets/pubs_presos/chos.pdf

[31]

Red Hat Inc. 2016. CVE-2016-10208. (Nov. 2016). https://access.redhat.com/security/cve/cve-2016-10208

[32]

Reventlov. 2015. Using the docker command to root the host (totally not a security issue). (April 2015). http://reventlov.com/advisories/using-the-docker-command-to-root-the-host

[33]

Rami Rosen. 2016. Namespaces and cgroups, the basis of Linux containers. (Feb. 2016). http://www.netdevconf.org/1.1/proceedings/slides/rosen-namespaces-cgroups-lxc.pdf

[34]

Cristian Ruiz, Emmanuel Jeanvoine, and Lucas Nussbaum. 2015. Performance evaluation of containers for HPC. In Euro-Par 2015: Parallel Processing Workshops.

[35]

Jerome H. Saltzer. 1974. Protection and the control of information sharing in Multics. CACM 17, 7 (July 1974).

Digital Library

[36]

Simes. 2002. How to break out of a chroot() jail. (May 2002). https://web.archive.org/web/20160209154009/http://www.bpfh.net/simes/computing/chroot-break.html

[37]

Robert Swiecki. 2016. NsJail. (Dec. 2016). https://google.github.io/nsjail/

[38]

systemd contributors. 2017. systemd-nspawn. Man page. https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html

[39]

Wikipedia editors. 2016. Virtualization. (Feb. 2016). https://en.wikipedia.org/w/index.php?title=Virtualization&oldid=704408822

[40]

Miguel G. Xavier, Marcelo V. Neves, Fabio D. Rossi, Tiago C. Ferreto, Timoteo Lange, and Cesar A. F. De Rose. 2013. Performance evaluation of container-based virtualization for high performance computing environments. In Euromicro Parallel, Distributed, and Network-Based Processing.

Digital Library

Cited By

Sun RSiller K(2024)HPC Container Management at the University of VirginiaPractice and Experience in Advanced Research Computing 2024: Human Powered Computing10.1145/3626203.3670568(1-4)Online publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1145/3626203.3670568
Pelofske E(2024)Analysis of a Programmable Quantum Annealer as a Random Number GeneratorIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.336405419(3636-3643)Online publication date: 8-Feb-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3364054
Prout AReuther AHoule MJones MMichaleas PAnderson LArcand WBergeron BBestor DBonn ABurrill DByun CGadepally VHubbell MJananthan HLuszczek PMilechin LMorales GMullen JRosa AYee CKepner J(2024)HPC with Enhanced User SeparationSC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SCW63240.2024.00221(1765-1772)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SCW63240.2024.00221
Show More Cited By

Index Terms

Charliecloud: unprivileged containers for user-defined software stacks in HPC

Recommendations

New Directions for Container Debloating
FEAST '17: Proceedings of the 2017 Workshop on Forming an Ecosystem Around Software Transformation

Application containers, such as Docker containers, are light-weight virtualization environments that "contain" applications together with their resources and configuration information. While they are becoming increasingly popular as a method for agile ...
Integrating Containers into Workflows: A Case Study Using Makeflow, Work Queue, and Docker
VTDC '15: Proceedings of the 8th International Workshop on Virtualization Technologies in Distributed Computing

Workflows are a widely used abstraction for representing large scientific applications and executing them on distributed systems such as clusters, clouds, and grids. However, workflow systems have been largely silent on the question of precisely what ...
Survey of adaptive containerization architectures for HPC
SC-W '23: Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis

Containers offer an array of advantages that benefit research reproducibility and portability. As container tools mature, container security improves, and high-performance computing (HPC) and cloud system tools converge, supercomputing centers are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '17: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

November 2017

801 pages

ISBN:9781450351140

DOI:10.1145/3126908

General Chair:
Bernd Mohr
Jülich Supercomputing Center, Jülich, Germany
,
Program Chair:
Padma Raghavan
Vanderbilt University, Nashville, TN

Copyright © 2017 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGHPC: ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

U.S. Department of Energy

Conference

SC '17

Sponsor:

SIGHPC

SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2017

Colorado, Denver

Acceptance Rates

SC '17 Paper Acceptance Rate 61 of 327 submissions, 19%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

90
Total Citations
View Citations
2,147
Total Downloads

Downloads (Last 12 months)346
Downloads (Last 6 weeks)43

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sun RSiller K(2024)HPC Container Management at the University of VirginiaPractice and Experience in Advanced Research Computing 2024: Human Powered Computing10.1145/3626203.3670568(1-4)Online publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1145/3626203.3670568
Pelofske E(2024)Analysis of a Programmable Quantum Annealer as a Random Number GeneratorIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.336405419(3636-3643)Online publication date: 8-Feb-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3364054
Prout AReuther AHoule MJones MMichaleas PAnderson LArcand WBergeron BBestor DBonn ABurrill DByun CGadepally VHubbell MJananthan HLuszczek PMilechin LMorales GMullen JRosa AYee CKepner J(2024)HPC with Enhanced User SeparationSC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SCW63240.2024.00221(1765-1772)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SCW63240.2024.00221
Priedhorsky RJennings MPhinney M(2024)Zero-consistency root emulation for unprivileged container image buildSC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SCW63240.2024.00023(126-136)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SCW63240.2024.00023
Kristiansson JWikfeldt T(2024)Developing AI Applications for the HPC-Cloud Continuum with ColonyOS2024 23rd International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC62236.2024.10705401(1-8)Online publication date: 8-Jul-2024
https://doi.org/10.1109/ISPDC62236.2024.10705401
Baresi LQuattrocchi GRasi N(2024)A qualitative and quantitative analysis of container enginesJournal of Systems and Software10.1016/j.jss.2024.111965210:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.111965
Medeiros DSchieffer GWahlgren JPeng I(2024)Understanding Layered Portability from HPC to Cloud in Containerized EnvironmentsHigh Performance Computing. ISC High Performance 2024 International Workshops10.1007/978-3-031-73716-9_31(439-452)Online publication date: 14-Dec-2024
https://doi.org/10.1007/978-3-031-73716-9_31
Dennis JBaker ADobbins BBell MSun JKim YCha T(2023)Enabling efficient execution of a variational data assimilation applicationInternational Journal of High Performance Computing Applications10.1177/1094342022111980137:2(101-114)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1177/10943420221119801
Mujkanovic NDurillo JHammer NMüller T(2023)Survey of adaptive containerization architectures for HPCProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624588(165-176)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624588
Priedhorsky ROgas JDavis IV CHounshel ZLee AStormer BGoff R(2023)Charliecloud’s layer-free, Git-based container build cacheProceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624585(135-146)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624585
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten