Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

DevOps Workflow: A Tutorial on Linux Containers

•
•

In this deck from the Stanford HPC Conference, Christian Kniep from Docker, Inc. gives a tutorial on linux containers. "This tutorial provides a detailed overview of the components needed to run containerized applications and explores how distributed HPC applications can be tackled. We’ll explain the concept of Linux Containers and describe the bits and pieces participants will explore following step-by-step examples. The workshop will introduce the predominant forms of orchestration in the industry; what problems they solve and how to approach the problem. Attendees will explore the benefits and drawbacks of orchestrators first hand with their own small exemplary stack deployments. Finally the workshop will introduce how HPC and Big Data workloads can be tackled on-top of these service-oriented clusters." Watch the video: https://youtu.be/LJinZpCTyk0 Learn more: http://www.docker.com/ and http://hpcadvisorycouncil.com Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1 of 19
Download to read offline
DevOps Workflow From 0 to kube in 60 min Christian Kniep, v2018-02-20 Technical Account Manager, Docker Inc.http://qnib.org/devops-workflow
Motivation Iteration barriers
ClusterLaptop Works on my Laptop! Why is DevOps not enough. ProdDevelopment Dev Artifact: code Env: interactive Ops Artifact: pkg/img Env: automated DevOps Release/Integration Release Artifact: pkg/img Env: automated DevRelOps Prod Ops Artifact: pkg/img Env: automated Development Dev Artifact: code Env: interactive DevRel*Ops Prod Ops Artifact: pkg/img Env: automated Development Dev Artifact: code Env: interactive Q&A Performance Release/Integration Release Artifact: pkg/img Env: automated Experiment Experiment Experiment
● Ship the image using a (private) Docker Registry ● Orchestrate the Image: docker-compose.yml Build, Ship & Run Where ‘Run’ == ‘Orchestrate’ ● Build the image: Dockerfile + context -> Docker Image For the developer the orchestration should be transparent and out of scope. Artifacts are: 1. Dockerfile + context (git repository) 2. docker-compose.yml (git repository)
Application Containerization Portability, Reproducibility As a developer I want to work on a webapp for DockerCon EU, which represents a simple three tier stack. ● web: JS app I am working on ● words: API serving business logic ● db: Database backend web api db api
Prerequisite for Workshop http://qnib.org/devops-workflow Docker Desktop - Docker4Mac / Docker4Win edge-channel (provides kubernetes) - DockerCE for Linux Git Repository git clone https://github.com/ChristianKniep/k8s-wordsmith-demo.git cd k8s-wordsmith-demo.git git checkout update/container_to_kube docker-compose pull cd swarmprom ; docker-compose pull
#1 Building the Image(s) First Step as a Developer
docker-compose build $ export TAG=$(date +%s) $ docker-compose build WARNING: Some services (api) use the 'deploy' key, which will be ignored. Building web Step 1/12 : FROM golang:1.9.1-alpine3.6 as builder *snip* Successfully tagged qnib/k8s-wordsmith-web:1518955537 Building api *snip* Successfully tagged qnib/k8s-wordsmith-api:1518955537 Building db *snip* Successfully tagged qnib/k8s-wordsmith-db:1518955537
#2 Simple Container One-shot Orchestration
docker-compose build $ docker-compose up WARNING: Some services (api) use the 'deploy' key, which will be ignored. Creating network "k8swordsmithdemo_default" with the default driver Creating k8swordsmithdemo_web_1 ... done Creating k8swordsmithdemo_api_1 ... done Creating k8swordsmithdemo_db_1 ... done Attaching to k8swordsmithdemo_db_1, k8swordsmithdemo_api_1, k8swordsmithdemo_web_1 db_1 | The files belonging to this database system will be owned by user "postgres". db_1 | This user must also own the server process.
#3 Deploying Stack in SWARM Simple yet Powerful Orchestration
SWARM at a Glance Easy to understand and reason about
$ cd swarmprom $ docker stack deploy -c docker-compose.yml prom Creating network prom_net Creating config prom_service_rules Creating config prom_node_name Creating config prom_caddy_config Creating config prom_dockerd_config Creating config prom_node_rules Creating config prom_task_rules Creating service prom_caddy Creating service prom_dockerd-exporter Creating service prom_cadvisor Creating service prom_grafana Creating service prom_node-exporter Creating service prom_prometheus Deploy Auxiliary Stack to Monitor SWARM Fork of: https://github.com/stefanprodan/swarmprom
Docker stack deploy Using the Docker SWARM API $ docker stack deploy -c docker-compose.yml words Ignoring unsupported options: build Creating network words_default Creating service words_web Creating service words_db Creating service words_api $ docker service ls --format="{{.Name}}t{{.Replicas}}t{{.Image}}t{{.Ports}}" words_api 5/5 qnib/k8s-wordsmith-api:1518955537 words_db 1/1 qnib/k8s-wordsmith-db:1518955537 words_web 1/1 qnib/k8s-wordsmith-web:1518955537 *:8080->80/tcp $ $ docker stack deploy -c docker-compose.yml words #2>dev/null Updating service words_web (id: fv7z1lryqmysxhzobdjrbgecl) image qnib/k8s-wordsmith-web:1518955537 could not be accessed on a registry to record its digest. Each node will access qnib/k8s-wordsmith-web:1518955537 independently, possibly leading to different nodes running different versions of the image.
#4 Deploying Stack in Kubernetes Using CustomResourceDefinitions
Docker stack deploy #2 Using the docker-CRD within Kube $ docker stack rm words Removing service words_api Removing service words_db Removing service words_web Removing network words_default $ export DOCKER_ORCHESTRATOR=kubernetes $ docker stack deploy -c docker-compose.yml words Stack words was created Waiting for the stack to be stable and running... - Service api has one container running - Service db has one container running - Service web has one container running Stack words is stable and running $
#5 Deploying Stack in Kubernetes Using Kubernetes Object Files
Kubectl deploy #2 Using the kubectl CLI $ export DOCKER_ORCHESTRATOR=kubernetes $ kubectl apply -f . deployment "kube-api" created service "kube-api" created deployment "kube-db" created service "kube-db" created deployment "kube-web" created service "kube-web" created $
THANK YOU :) christian.kniep@docker.com

More Related Content

DevOps Workflow: A Tutorial on Linux Containers

  • 1. DevOps Workflow From 0 to kube in 60 min Christian Kniep, v2018-02-20 Technical Account Manager, Docker Inc.http://qnib.org/devops-workflow
  • 3. ClusterLaptop Works on my Laptop! Why is DevOps not enough. ProdDevelopment Dev Artifact: code Env: interactive Ops Artifact: pkg/img Env: automated DevOps Release/Integration Release Artifact: pkg/img Env: automated DevRelOps Prod Ops Artifact: pkg/img Env: automated Development Dev Artifact: code Env: interactive DevRel*Ops Prod Ops Artifact: pkg/img Env: automated Development Dev Artifact: code Env: interactive Q&A Performance Release/Integration Release Artifact: pkg/img Env: automated Experiment Experiment Experiment
  • 4. ● Ship the image using a (private) Docker Registry ● Orchestrate the Image: docker-compose.yml Build, Ship & Run Where ‘Run’ == ‘Orchestrate’ ● Build the image: Dockerfile + context -> Docker Image For the developer the orchestration should be transparent and out of scope. Artifacts are: 1. Dockerfile + context (git repository) 2. docker-compose.yml (git repository)
  • 5. Application Containerization Portability, Reproducibility As a developer I want to work on a webapp for DockerCon EU, which represents a simple three tier stack. ● web: JS app I am working on ● words: API serving business logic ● db: Database backend web api db api
  • 6. Prerequisite for Workshop http://qnib.org/devops-workflow Docker Desktop - Docker4Mac / Docker4Win edge-channel (provides kubernetes) - DockerCE for Linux Git Repository git clone https://github.com/ChristianKniep/k8s-wordsmith-demo.git cd k8s-wordsmith-demo.git git checkout update/container_to_kube docker-compose pull cd swarmprom ; docker-compose pull
  • 7. #1 Building the Image(s) First Step as a Developer
  • 8. docker-compose build $ export TAG=$(date +%s) $ docker-compose build WARNING: Some services (api) use the 'deploy' key, which will be ignored. Building web Step 1/12 : FROM golang:1.9.1-alpine3.6 as builder *snip* Successfully tagged qnib/k8s-wordsmith-web:1518955537 Building api *snip* Successfully tagged qnib/k8s-wordsmith-api:1518955537 Building db *snip* Successfully tagged qnib/k8s-wordsmith-db:1518955537
  • 10. docker-compose build $ docker-compose up WARNING: Some services (api) use the 'deploy' key, which will be ignored. Creating network "k8swordsmithdemo_default" with the default driver Creating k8swordsmithdemo_web_1 ... done Creating k8swordsmithdemo_api_1 ... done Creating k8swordsmithdemo_db_1 ... done Attaching to k8swordsmithdemo_db_1, k8swordsmithdemo_api_1, k8swordsmithdemo_web_1 db_1 | The files belonging to this database system will be owned by user "postgres". db_1 | This user must also own the server process.
  • 11. #3 Deploying Stack in SWARM Simple yet Powerful Orchestration
  • 12. SWARM at a Glance Easy to understand and reason about
  • 13. $ cd swarmprom $ docker stack deploy -c docker-compose.yml prom Creating network prom_net Creating config prom_service_rules Creating config prom_node_name Creating config prom_caddy_config Creating config prom_dockerd_config Creating config prom_node_rules Creating config prom_task_rules Creating service prom_caddy Creating service prom_dockerd-exporter Creating service prom_cadvisor Creating service prom_grafana Creating service prom_node-exporter Creating service prom_prometheus Deploy Auxiliary Stack to Monitor SWARM Fork of: https://github.com/stefanprodan/swarmprom
  • 14. Docker stack deploy Using the Docker SWARM API $ docker stack deploy -c docker-compose.yml words Ignoring unsupported options: build Creating network words_default Creating service words_web Creating service words_db Creating service words_api $ docker service ls --format="{{.Name}}t{{.Replicas}}t{{.Image}}t{{.Ports}}" words_api 5/5 qnib/k8s-wordsmith-api:1518955537 words_db 1/1 qnib/k8s-wordsmith-db:1518955537 words_web 1/1 qnib/k8s-wordsmith-web:1518955537 *:8080->80/tcp $ $ docker stack deploy -c docker-compose.yml words #2>dev/null Updating service words_web (id: fv7z1lryqmysxhzobdjrbgecl) image qnib/k8s-wordsmith-web:1518955537 could not be accessed on a registry to record its digest. Each node will access qnib/k8s-wordsmith-web:1518955537 independently, possibly leading to different nodes running different versions of the image.
  • 15. #4 Deploying Stack in Kubernetes Using CustomResourceDefinitions
  • 16. Docker stack deploy #2 Using the docker-CRD within Kube $ docker stack rm words Removing service words_api Removing service words_db Removing service words_web Removing network words_default $ export DOCKER_ORCHESTRATOR=kubernetes $ docker stack deploy -c docker-compose.yml words Stack words was created Waiting for the stack to be stable and running... - Service api has one container running - Service db has one container running - Service web has one container running Stack words is stable and running $
  • 17. #5 Deploying Stack in Kubernetes Using Kubernetes Object Files
  • 18. Kubectl deploy #2 Using the kubectl CLI $ export DOCKER_ORCHESTRATOR=kubernetes $ kubectl apply -f . deployment "kube-api" created service "kube-api" created deployment "kube-db" created service "kube-db" created deployment "kube-web" created service "kube-web" created $