Introduction to Docker and deployment and Azure

Docker
September 2014—Docker 1.2

@jpetazzo
● Wrote dotCloud PAAS deployment tools
– EC2, LXC, Puppet, Python, Shell, ØMQ...
● Docker contributor
– Security, Networking...
● Runs all kinds of crazy things in Docker
– Docker-in-Docker, VPN-in-Docker,
KVM-in-Docker, Xorg-in-Docker...

Raise your hand if you have ...
● Tried Docker (online tutorial)

● Tried the real Docker (e.g. deployed remote VM)

● Installed Docker locally (e.g. with boot2docker)

● Written a Dockerfile (and built it!)

● An image on Docker Hub (pushed or autobuilt)

● An image on Docker Hub (pushed or autobuilt)
● Deployed Docker images for dev/QA/test/prod...

Agenda
● What is Docker and Why it matters
● What are containers
● The Docker ecosystem (Engine, Hub, etc.)
● Deployment options and first steps
● What's new in the latest Docker?

What
is Docker
Why
it matters

Deploy everything
● Webapps
● Backends
● SQL, NoSQL
● Big data
● Message queues
● … and more

Deploy almost everywhere
● Linux servers
● VMs or bare metal
● Any distro
● Kernel 3.8+ (or RHEL 2.6.32)
Currently: focus on x86_64.
(But people reported success on arm.)

Deploy reliably & consistently

Deploy reliably & consistently
● If it works locally, it will work on the server
● With exactly the same behavior
● Regardless of versions
● Regardless of distros
● Regardless of dependencies

Deploy efficiently
● Containers are lightweight
– Typical laptop runs 10-100 containers easily
– Typical server can run 100-1000 containers
● Containers can run at native speeds
– Lies, damn lies, and other benchmarks:
http://qiita.com/syoyo/items/bea48de8d7c6d8c73435
http://www.slideshare.net/BodenRussell/kvm-and-docker-lxc-benchmarking-with-openstack

Infiniband throughput and latency:
no difference at all

Booting 15 OpenStack VMs:
KVM vs Docker

Memory speed:
Bare Metal vs Docker vs KVM

Docker Engine
+ Docker Hub
= Docker Platform

The Docker
Engine runs
containers.

OK, but
what is a
container?

High level approach:
it's a lightweight VM
● Own process space
● Own network interface
● Can run stuff as root
● Can have its own /sbin/init
(different from the host)
« Machine Container »

Low level approach:
it's chroot on steroids
● Can also not have its own /sbin/init
● Container = isolated process(es)
● Share kernel with host
● No device emulation (neither HVM nor PV)
« Application Container »

Alright, I get this.
Containers = nimble Vms.
Let's just tell the CFO,
and get back to work!

What happens when
something becomes
10-100x cheaper?

Random example:
testing
● Project X has 100 unit tests
● Each test needs a pristine SQL database

Random example:
testing
● Plan A: spin up 1 database, clean after each use
– If we don't clean correctly, random tests will fail
– Cleaning correctly can be expensive (e.g. reload DB)

Random example:
testing
● Plan B: spin up 100 databases
– … in parallel: needs too much resources
– … one after the other: takes too long

Random example:
testing
● Plan C: spin up 100 databases in containers
– fast, efficient (no overhead, copy-on-write)
– easy to implement without virtualization black belt

Containers
make testing
(and many other things)
way easier

Problem: shipping goods
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?

Solution:
the intermodal shipping container

Problem: shipping code
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?
? ? ? ? ? ?

Solution:
the Linux container

Separation of concerns:
Dave the Developer
● Inside my container:
– my code
– my libraries
– my package manager
– my app
– my data

Separation of concerns:
Oscar the Ops guy
● Outside the container:
– logging
– remote access
– network configuration
– monitoring

Docker: the cast
● Docker Engine
● Docker Hub
● Docker, the community
● Docker Inc, the company

Docker Engine
● Open Source engine to commoditize LXC
● Uses copy-on-write for quick provisioning
● Written in Go, runs as a daemon, comes with a CLI
● Everything exposed through a REST API
● Allows to build images in standard, reproducible way
● Allows to share images through registries
● Defines standard format for containers
(stack of layers; 1 layer = tarball+metadata)

… Open Source?
● Nothing up the sleeve, everything on the table
– Public GitHub repository: https://github.com/dotcloud/docker
– Bug reports: GitHub issue tracker
– Mailing lists: docker-user, docker-dev (Google groups)
– IRC channels: #docker, #docker-dev (Freenode)
– New features: GitHub pull requests (see CONTRIBUTING.md)
– Docker Governance Advisory Board (elected by contributors)

Docker Hub
Collection of services to make Docker more useful.
● Public registry
(push/pull your images for free)
● Private registry
(push/pull secret images for $)
● Automated builds
(link github/bitbucket repo; trigger build on commit)
● More to come!

Docker, the community
● >500 contributors
● ~20 core maintainers
● >8,000 Dockerized projects on GitHub
● >30,000 repositories on Docker Hub
● >250 meetups in >90 cities in >30 countries
● >750,000 downloads of boot2docker

Docker Inc, the company
● Headcount: ~50
● Led by Open Source veteran Ben Golub
(GlusterFS)
● Revenue:
– t-shirts and stickers featuring the cool blue whale
– SAAS delivered through Docker Hub
– Support & Training

One-time setup
● On your dev env (Linux, OS X, Windows)
– boot2docker (25 MB VM image)
– Natively (if you run Linux)
● On your servers (Linux)
– Packages (Ubuntu, Debian, Fedora, Gentoo, Arch...)
– Single binary install (Golang FTW!)
– Easy provisioning on Azure, Rackspace, Digital Ocean...
– Special distros: CoreOS, Project Atomic

Azure deployment
VMNAME=jpetazzo
IMAGE=b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04-LTS-amd64-server-20140724-en-us-
30GB
USER=jpetazzo
PASSWORD=1234abcdABCD@
LOCATION="West US"
azure vm docker create $VMNAME
$IMAGE $USER $PASSWORD -l "$LOCATION"
export DOCKER_HOST=tcp://$VMNAME.cloudapp.net:4243
docker --tls version
azure vm endpoint create $VMNAME 80

Authoring images
with a Dockerfile

FROM ubuntu:14.04
RUN apt-get update
RUN apt-get install -y nginx
RUN echo 'Hi, I am in your container!'
>/usr/share/nginx/html/index.html
CMD nginx -g "daemon off;"
EXPOSE 80
docker build -t jpetazzo/staticweb .
docker run -P jpetazzo/staticweb

FROM ubuntu:12.04
RUN apt-get -y update
RUN apt-get install -y g++
RUN apt-get install -y erlang-dev erlang-base-hipe ...
RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...
RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz
| tar -C /tmp -zxf-
RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]nport = 8101nbind_address = 0.0.0.0"
> /usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101
CMD ["/usr/local/bin/couchdb"]
docker build -t jpetazzo/couchdb .

FROM debian:jessie
RUN apt-get -y update
RUN apt-get install -y python-pip
RUN mkdir /src
WORKDIR /src
ADD requirements.txt /src
RUN pip install -r requirements.txt
ADD . /src
RUN python setup.py install

Do you even
Chef?
Puppet?
Ansible?
Salt?

Summary
With Docker, I can:
● put my software in containers
● run those containers anywhere
● write recipes to automatically build containers

Advanced concepts
● naming
– give a unique name to your containers
● links
– connect containers together
● volumes
– separate code and data
– share data between containers

Recent features: 0.10
● TLS support for API access
● Configurable DNS search
● BTRFS is no longer experimental
● Integration with systemd cgroups
● Use proxy environment variables (for registry)

● SELinux integration
(works better with CentOS)
● DNS integration for links
(access linked containers by hostname)
● « docker run --net »
– use host networking for high speed
– share network of another container

● docker pause/unpause
● more importantly: 1.0 release candidate :-)

Docker 1.1
● .dockerignore
(don't upload your .git anymore!)
● docker logs --tail
– further logging improvements on the way
(truncate)

Docker 1.2
● New cool options for docker run
--restart=always/no/on-failure
--cap-add=NETADMIN
--cap-drop=CHOWN
--device=/dev/kvm:/dev/kvm

Coming soon
(maybe)
● logging improvements
● device mapper tuning
● image squashing
● ARM support
● use secrets in builds
● volume management
● hairpin nat
● IPV6 support
● seccomp + native
● user namespaces

Thank you! Questions?
http://docker.com/
@docker
@jpetazzo

Introduction to Docker and deployment and Azure

More Related Content

Introduction to Docker and deployment and Azure