CloudNative Days Tokyo 2020での、lazypullに関する発表資料です。https://event.cloudnativedays.jp/cndt2020/talks/16
Stargz Snapshotterのリポジトリ:
https://github.com/containerd/stargz-snapshotter
4. Copyright(c)2020 NTT Corp. All Rights Reserved
Pull % %
p t4 H
e [ . u 6
0
0
l
1 e ] 0 0
. 0
0 a
.
r . 6
6 4
2 0 2
5. Copyright(c)2020 NTT Corp. All Rights Reserved
Pull (CRI)
API
kubectl apply
pull
kubelet
Pod
CRI
pull OCI
Docker containerd
runc gVisor Kata Containers
(CRI)
(OCI)
6. Copyright(c)2020 NTT Corp. All Rights Reserved
containerd
l CNCF graduated
l GKE[1] AWS Fargate[2] AKS(preview)[3] IKS[4] 6
l CRI CRI Kubernetes
l
l Kata Containers AWS Firecracker
l CRI
https://github.com/containerd/containerd
[1] https://cloud.google.com/kubernetes-engine/docs/concepts/using-containerd ; [2] https://aws.amazon.com/jp/blogs/containers/under-the-hood-fargate-data-plane/ ; [3] https://azure.microsoft.com/ja-
jp/updates/azure-kubernetes-service-aks-support-for-containerd-runtime-is-in-preview/ ; [4] https://www.ibm.com/cloud/blog/ibm-cloud-kubernetes-service-supports-containerd
8. Copyright(c)2020 NTT Corp. All Rights Reserved
containerd Stargz Snapshotter lazypull
l containerd 8
l Google CRFS 8 stargz
lazypull
l stargz eStargz 8
containerd Stargz Snapshotter
https://github.com/containerd/stargz-snapshotter
Stargz
Snapshotter
Kubelet
OCI
8
lazypull
stargz
pull
Slacker[1],CernVM-FS[2],Filegrain[3],Microsoft Teleportation[4],Google CRFS[5]
[1] https://www.usenix.org/conference/fast16/technical-sessions/presentation/harter [2] https://github.com/containerd/containerd/issues/2943 [3] https://github.com/AkihiroSuda/filegrain, [4]
https://stevelasker.blog/2019/10/29/azure-container-registry-teleportation/ [5] https://github.com/google/crfs
9. Copyright(c)2020 NTT Corp. All Rights Reserved
OCI stargz eStargz
Stargz
Snapshotter
stargz eStargz
lazypull
lazypull
pull run
(ctr-remote )
(DockerHub )
l stargz Google CRFS https://github.com/google/crfs 9 lazypull
l eStargz
l 9 OCI
• lazypull
• lazypull stargz/eStargz
squash
10. Copyright(c)2020 NTT Corp. All Rights Reserved
l
l 0 11 1 0
l https://github.com/moby/buildkit/pull/1402
CI
l Go stargz 1 CRFS 0Go
0
l https://github.com/golang/go/issues/30829
Serverless FaaS cold start
l https://github.com/knative/serving/issues/5913
11. Copyright(c)2020 NTT Corp. All Rights Reserved
l 1 1
• Dockerfile1RUN 1 1
l 1 1
l 1
l Docker[1] OCI[2] 1
[1] https://docs.docker.com/registry/spec/manifest-v2-2/ ; [2] https://github.com/opencontainers/image-spec/tree/v1.0.1
12. Copyright(c)2020 NTT Corp. All Rights Reserved
Pull API
GET /v2/<image>/blobs/GET /v2/<image>/manifests/
l API 1
l API API
l Docker[1] OCI[2] 2
l 2 optional Range Request 2 1
• 2
[1] https://docs.docker.com/registry/spec/api/ ; [2] https://github.com/opencontainers/distribution-spec/tree/v1.0.0-rc0
13. Copyright(c)2020 NTT Corp. All Rights Reserved
( )pull
rootfs
GET /v2/<image>/blobs/
GET /v2/<image>/manifests/
14. Copyright(c)2020 NTT Corp. All Rights Reserved
containerd remote snapshotter (1.4 )
l
l
l pull
Remote
Snapshotter
15. Copyright(c)2020 NTT Corp. All Rights Reserved
Stargz Snapshotter lazypull
Stargz
Snapshotter
rootfs FUSE
l
l stargz/eStargz pull
Lazy
pull
GET /v2/<image>/manifests/
GET /v2/<image>/blobs/
stargz
eStargz
16. Copyright(c)2020 NTT Corp. All Rights Reserved
tar
bin/bash
bin/ls
etc/passw
d
etc/group
usr/bin/apt
layer = tar (+ )
GET /v2/<image>/blobs/
l 1
6 1
l gzip
17. Copyright(c)2020 NTT Corp. All Rights Reserved
stargz
tar.gz stargz
bin/bash
bin/ls
etc/passwd
etc/group
usr/bin/apt
TOCEntries
HTTP Range Request
pull
bin/bash
bin/ls
etc/passwd
etc/group
usr/bin/apt
gzip
7
1
l Google CRFS https://github.com/google/crfs lazypull
l Seekable tar.gz
• Range Request 1 7
• tar.gz7 7
17
7
18. Copyright(c)2020 NTT Corp. All Rights Reserved
bin/bash
usr/bin/apt
etc/passwd
etc/group
bin/ls
GET /v2/<image-name>/blobs/
GET
Range: bytes=1048-2047
19. Copyright(c)2020 NTT Corp. All Rights Reserved
eStargz prefetch
stargz eStargz
bin/ls
usr/bin/apt
entrypoint.sh
bin/bash
bin/ls
usr/bin/apt
entrypoint.sh
bin/bash Range Request
landmark file
TOCEntriesTOCEntries
l eStargz stargz
l 1 9 NW
20. Copyright(c)2020 NTT Corp. All Rights Reserved
prefetch eStargz
bin/bash
entrypoint.sh
GET /v2/<image-name>/blobs/
GET
Range: bytes=0-1047
21. Copyright(c)2020 NTT Corp. All Rights Reserved
prefetch
l Docker 21 stargz eStargz
l eStargz
l
21
l 2 2
eStargzproc
ctr-remote
22. Copyright(c)2020 NTT Corp. All Rights Reserved
l Hello Bench [Harter, et al. 2016]2 2 2
• 2DockerHub
• ”print hello world”2
• ”up and running”
l : EC2 Oregon (m5.2xlarge, Ubuntu 20.04)
l : Docker Hub (docker.io)
l Stargz Snapshotter : b53e8fe8d37751753bc623b037729b6a6d9c1122
l pull create run 95 100 2
[Harter et al. 2016] Tyler Harter, Brandon Salmon, Rose Liu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau. "Slacker: Fast Distribution with Lazy
Docker Containers". 14th USENIX Conference on File and Storage Technologies (FAST ’16). February 22–25, 2016, Santa Clara, CA, USA
NTT
pull create run
print hello world
up and running
23. Copyright(c)2020 NTT Corp. All Rights Reserved
NTT
0 5 10 15 20 25
estargz
stargz
legacy
python:3.7 (print “hello”)
pull create run [sec]
Prefetch
24. Copyright(c)2020 NTT Corp. All Rights Reserved
NTT
0 5 10 15 20 25 30
estargz
stargz
legacy
gcc:9.2.0 (printf(“hello”); )
pull create run [sec]
25. Copyright(c)2020 NTT Corp. All Rights Reserved
0 5 10 15 20 25
estargz
stargz
legacy
glassfish:4.1-jdk8 ( “Running GlassFish” )
pull create run
NTT
[sec]
26. Copyright(c)2020 NTT Corp. All Rights Reserved
early stage
l NW CDN lazy pull
l READ 2
l 2
https://github.com/containerd/stargz-snapshotter
l CRI 6 containerd lazypull
l Remote snapshotter > 1.4
pull
l Stargz Snapshotter stargz eStargz
lazypull
l stargz: Google CRFS https://github.com/google/crfs lazypull
l eStargz: stargz prefetch