Container Orchestration using kubernetes

Classification: Public
Kubernetes

Cloud Native Computing Foundation

every cloud supports kubernetes
https://www.sinax.be/en/aws/
https://www.westconcomstor.com/za/en/vendors/wc-vendors/microsoft-azure-EN-UK.html
https://www.g2crowd.com/products/google-kubernetes-engine-gke/details
Kubernetes support on cloud

Kubernetes Market Adoption Survey

What does “Kubernetes” mean?
Greek for “pilot” or “helmsman of a ship”

A Container Orchestration System.
● A project that was spun out of Google as an open source container
orchestration platform (~2 billion containers/week).
● Built from the lessons learned in the experiences of developing and running
Google’s Borg and Omega.
● Designed from the ground-up as a loosely coupled collection of
components centred around deploying, maintaining and scaling workloads.
What is Kubernetes?

What does Kubernetes do?
● Known as the linux kernel of distributed systems.
● Abstracts away the underlying hardware of the nodes and provides a uniform
interface for workloads to be both deployed and consume the shared pool of
resources.
● Works as an engine for resolving state by converging the actual and the
desired state of the system.

Kubernetes is self-healing
Kubernetes will ALWAYS try and steer the cluster to its desired state.
● User: “I want 3 healthy instances of Redis to always be running.”
● Kubernetes: “Okay, I’ll ensure there are always 3 instances up and running.”
● Kubernetes: “Oh look, one has died. I’m going to attempt to spin up a new one.”

What can Kubernetes really do?
● Autoscale Workloads
● Blue/Green Deployments
● Fire off Jobs and scheduled CronJobs
● Manage Stateless and Stateful Applications
● Built-in Service Discovery
● ~Easily integrate and support 3rd party apps~

Kubernetes in a nutshell

Service Model : Pets vs Cattle

Kubernetes Architecture

Masters – Acts as the primary control plane for Kubernetes.
• Masters are responsible at a minimum for running the API Server, scheduler and
cluster controller.
• They also manage storing cluster state, cloud-provider specificcomponents and other
cluster essential services.
Nodes -Are the ‘workers’ of a Kubernetes cluster.
• Theyrun a minimal agent that manages the node itself, and execute workloads as
designated by the master.
Architecture Overview

kube-apiserver – the heart of the cluster
● Provides a forward facing REST interface into the Kubernetes control
plane and datastore.
● All clients and other applications interact with Kubernetes strictly
through the API Server.
● Acts as the gatekeeper to the cluster by handling authentication and
authorization, request validation and admission control in addition to
being the front-end to the backing datastore.
Kubectl
Kubectl is the official Kubernetes command line interface tool. It is
used to communicate with the API.

etcd – the key-value datastore
● etcd acts as the cluster datastore.
● A standalone incubating CNCF project https://www.cncf.io/projects/
● Purpose in relation to Kubernetes is to provide a strong, consistent
and highly available key-value store for persisting all cluster state.

kube-controller-manager
● Serves as the primary daemon that manages all core components’
reconciliation loops.
● Handles a lot of the business logic of Kubernetes.
● Monitors the cluster state via the API Server and steers the cluster towards
the desired state.

kube-scheduler
Evaluates workload requirements and attempts to place it on a matching
resource.
● The default scheduler uses the “binpacking” mode.
● Workload Requirements can include: general hardware requirements,
affinity/anti-affinity, labels, and other various custom resource
requirements.
● Is swappable, you can create your own scheduler

Node Components

kubelet
Acts as the node agent responsible for managing pod lifecycle on its host.
Kubelet understands YAML container manifests that it can read from several
sources:
● File path
● HTTP Endpoint
● Etcd watch acting on any changes
● HTTP Server mode accepting container manifests over a simple API.

kube-proxy
● Manages the network rules for Services on each node.
● Performs connection forwarding or load balancing for
Kubernetes Services.
● Available Proxy Modes:
○ ipvs (default if supported)
○ iptables (default fallback)
○ userspace (legacy)

Container Runtime – the executor
● A container runtime is a CRI (Container Runtime Interface) compatible
application that executes and manages containers.
○ Docker (default, built into the kubelet atm) ○ containerd
○ cri-o
○ rkt
○ Kata Containers (formerly clear and hyper)
○ Virtlet (VM CRI compatible runtime)

Additional Services
Kube-dns - Provides cluster wide DNS Services. Services are resolvable to
<service>.<namespace>.svc.cluster.local.
Heapster - Metrics Collector for kubernetes cluster, used by some
resources such as the Horizontal Pod Autoscaler. (required for
kubedashboard metrics)
Kube-dashboard -A general purpose web based UI for kubernetes.

Kube-dashboard

Kubernetes concepts

Pod: The basic building block in Kubernetes. The name is the same idea as a pod of
whales, one of more containers– or better said, a running process on the cluster —
running on a given system which can be scaled up or down.
Service: A consistent network access point for a pod. What this means is that
regardless of the underlying containers’ state – restarts, etc, a service will always
provide access to the given pod/application.
ReplicaSet: Ensures a given number of pods are running in the cluster at a given time.
Deployment: Suited for more ephemeral applications like web applications, anything
that doesn’t have persistent qualities. It Provides declarative updates for Pods and
ReplicaSets by describing a desired state the deployment in turn acts on.

Stateful Set: Suited for stateful applications like databases or even monitoring systems like
Prometheus. Similar to a deployment in that it manages the deployment and scaling of a set of pods,
but differs in that it ensures ordinal (things happen in a given sequence and are numbered) and
uniqueness of pods. Overall, it ensures that all resources in the stateful set have a sticky identity
across restarts.
Labels: When creating Kubernetes objects, one can give any number of labels to each object which
are discoverable across the system. For instance, one could create a database with the label
“customer_database” and another application pod could reference it by simply using that name
“customer_database” in the selector value of the manifest file.
Namespaces: Virtual clusters. Provides a means to separate applications in the cluster.

Mini-kube : Install and configure
Refer git repo for instructions.

Networking
To be updated from this slide onwards

Networking -FundamentalRules
1) All Pods can communicate with all other Pods without NAT
2) All nodes can communicate with all Pods (and vice-versa) without
NAT.
3) The IP that a Pod sees itself as is the same IP that others see it as.

Networking –Fundamentals Applied
• Containers in a pod exist within the same network namespaceand share an IP;
allowing for intrapod communication over localhost.
• Pods are givena cluster unique IP for the duration of its lifecycle,but the pods themselves
are fundamentally ephemeral.
• Services are givena persistent cluster unique IP that spans the Pods lifecycle.
• External Connectivity is generally handed by an integrated cloudprovider or other
external entity (load balancer)

Networking -CNI
Networking within Kubernetes is plumbed via the Container Network
Interface (CNI), an interface between a container runtime and a
network implementation plugin.
Compatible CNI Network Plugins:
● Calico
● Cillium
● Contiv
● Contrail
● Flannel
● GCE
● kube-router
● Multus
● OpenVSwitch
● OVN
● Romana
● Weave

Kubernetes
Concepts

Kubernetes Concepts-Core
Cluster - A collection of hosts that aggregate their available resources including cpu, ram,
disk, and their devices into a usable pool.
Master - The master(s) represent a collection of components that make up the control plane
of Kubernetes. These components are responsible for all cluster decisions including both
scheduling and responding to cluster events.
Node - A single host, physical or virtual capable of running pods. A node is managed by
the master(s), and at a minimum runs both kubelet and kube-proxy to be considered part
of the cluster.
Namespace - A logical cluster or environment. Primary method of dividing a cluster
or scoping access.

Concepts -Core(cont.)
Label - Key-value pairs that are used to identify, describe and group together related sets
of objects. Labels have a strict syntax and available character set. *
Annotation - Key-value pairs that contain non-identifying information or metadata.
Annotations do not have the the syntax limitations as labels and can contain structured
or unstructured data.
Selector - Selectors use labels to filter or select objects. Both equality-based (=, ==, !=)
or simple key-value matching selectors are supported.
* https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set

Labels:
app: nginx
tier: frontned
Annotations
description: “nginxfrontend”
Selector:
app:
nginx
tier: frontend
Labels, andAnnotations, and
Selectors

Set-basedselectors
Valid Operators:
● In
● NotIn
● Exists
● DoesNotExist
Supported Objects with set-based
selectors:
● Job
● Deployment
● ReplicaSet
● DaemonSet
● PersistentVolumeClaims

Concepts - Workloads
Pod - A pod is the smallest unit of work or management resource within Kubernetes. It
is comprised of one or more containers that share their storage, network, and context
(namespace, cgroupsetc).
ReplicationController - Method of managing pod replicas and their lifecycle.
Their scheduling, scaling, anddeletion.
ReplicaSet - Next Generation ReplicationController. Supports set-based selectors.
Deployment - A declarative method of managing stateless Pods and ReplicaSets.
Provides rollback functionality in addition to more granular update control mechanisms.

Deployment
Contains configuration
of how updates or
‘deployments’ should be
managed in addition to
the pod template used to
generate the
ReplicaSet.
ReplicaSet
Generated ReplicaSet
from Deploymentspec.

Concepts -Workloads (cont.)
StatefulSet - A controller tailored to managing Pods that must persist or maintain state.
Pod identity including hostname, network, and storage will be persisted.
DaemonSet - Ensures that all nodes matching certain criteria will run an instance of
a supplied Pod. Ideal for cluster wide services such as log forwarding, or health
monitoring.

StatefulSet
● Attaches to ‘headeless service’ (notshown)
nginx.
● Pods given unique ordinal names using the
pattern
<statefulset name>-<ordinal index>.
● Creates independent persistent volumes based
on the ‘volumeClaimTemplates’.

DaemonSet
● Bypasses default scheduler
● Schedules a single instance on every host
while adhering to tolerances and taints.

Concepts -Workloads (cont.)
Job - The job controller ensures one or more pods are executed and successfully terminates.
It will do this until it satisfies the completion and/or parallelism condition.
CronJob - An extension of the Job Controller, it provides a method of executing jobs on
a cron-like schedule.

Jobs
● Number of pod executions can be controlled
via spec.completions
● Jobs can be parallelized using
spec.parallelism
● Jobs and Pods are NOT
automatically cleaned up after a job
has completed.

CronJob
● Adds cron schedule to job template

Concepts - Network
Service - Services provide a method of exposing and consuming L4 Pod network
accessible resources. They use label selectors to map groups of pods and ports to a cluster-
unique virtual IP.
Ingress - An ingress controller is the primary method of exposing a cluster service
(usually http) to the outside world. These are load balancers or routers that usually offer
SSL termination, name-based virtual hosting etc.

Service
● Acts as the unified method of accessing replicated pods.
● Four major Service Types:
○ CluterIP - Exposes service on a strictly cluster-internal IP (default)
○ NodePort - Service is exposed on each node’s IP on a
statically defined port.
○ LoadBalancer - Works in combination with a cloud provider to
expose a service outside the cluster on a static external IP.
○ ExternalName - used to references endpoints OUTSIDE the
cluster by providing a static internally referenced DNS name.

IngressController
● Deployed as a pod to one or more
hosts
● Ingress controllers are an
external controller with multiple
options.
○ Nginx
○ HAproxy
○ Contour
○ Traefik
● Specific features and controller
specific configuration is passed
through annotations.

Concepts - Storage
Volume - Storage that is tied to the Pod Lifecycle, consumable by one or
more containers within the pod.
PersistentVolume - A PersistentVolume (PV) represents a storage resource. PVs
are commonly linked to a backing storage resource, NFS, GCEPersistentDisk, RBD etc.
and are provisioned ahead of time. Their lifecycle is handled independently from a pod.
PersistentVolumeClaim - A PersistentVolumeClaim (PVC) is a request for storage
that satisfies a set of requirements instead of mapping to a storage resource directly.
Commonly used with dynamically provisioned storage.
StorageClass - Storage classes are an abstraction on top of an external storage
resource. These will include a provisioner, provisioner configuration parameters as well
as a PV reclaimPolicy.

Volumes

PersistentVolumes
● PVs are a cluster-wide resource
● Not directly consumable by a Pod
● PV Parameters:
○ Capacity
○ accessModes
■ ReadOnlyMany (ROX)
■ ReadWriteOnce (RWO)
■ ReadWriteMany (RWX)
○ persistentVolumeReclaimPolic
y
■ Retain
■ Recycle
■ Delete
○ StorageClass

Persistent VolumeClaims
● PVCs are scoped to namespaces
● Supports accessModes likePVs
● Uses resource request model similar to Pods
● Claims will consume storage from matching
PVs or StorageClasses based on
storageClass and selectors.

StorageClasses
● Uses an external system defined by
the provisioner to dynamically
consume and allocate storage.
● Storage ClassFields
○ Provisioner
○ Parameters
○ reclaimPolicy

Concepts -Configuration
• ConfigMap - Externalized data stored within kubernetes that can
be referenced as a commandline argument, environment variable,
or injected as a file into a volume mount. Ideal for separating
containerized application from configuration.
• Secret - Functionally identical to ConfigMaps, but stored encoded as
base64, and encrypted at rest (if configured).

ConfigMaps andSecrets
● Can be used in Pod Config:
○ Injected as a file
○ Passed as an environment variable
○ Used as a container command (requires passing as env
var)

Concepts -Auth and Identity(RBAC)
[Cluster]Role - Roles contain rules that act as a set of permissions that apply verbs like
“get”, “list”, “watch” etc over resources that are scoped to apiGroups. Roles are scoped to
namespaces, and ClusterRoles are applied cluster-wide.
[Cluster]RoleBinding - Grant the permissions as defined in a [Cluster]Role to one or
more “subjects” which can be a user, group, or service account.
ServiceAccount- ServiceAccounts provide a consumable identity for pods or
external services that interact with the cluster directly and are scoped to namespaces.

[Cluster]Role
● Permissions translate to url
path. With “” defaulting to core
group.
● Resources act as items the
role should be granted
access to.
● Verbs are the actions the role
can perform on the referenced
resources.

[Cluster]RoleBinding
● Can reference multiple subjects
● Subjects can be of kind:
○ User
○ Group
○ ServiceAccount
● roleRef targets a single role
only.

Behind
TheScenes

Deployment From
Beginning toEnd

Kubectl
1)Kubectl performs client side
validation on manifest (linting).
2)Manifest is prepared and serialized
creating a JSON payload.

APIserver RequestLoop
3)Kubectl authenticates to apiserver via x509, jwt,
http auth proxy, other plugins, or http-basic auth.
4)Authorization iterates over availableAuthZ
sources: Node,ABAC, RBAC, or webhook.
5)AdmissionControl checks resource quotas,
other security related checks etc.
6) Request is stored in etcd.
7) Initializers are given opportunity to mutate request before the object is published.
8) Request is published on apiserver.

DeploymentController
9)Deployment Controller is notified of the new
Deployment via callback.
10)Deployment Controller evaluates cluster state and
reconciles the desired vs current state and forms a
request for the new ReplicaSet.
11)apiserver request loop evaluates Deployment
Controller request.
12) ReplicaSet ispublished.

ReplicaSetController
13)ReplicaSet Controller is notified of the new ReplicaSet
via callback.
14)ReplicaSet Controller evaluates cluster state and
reconciles the desired vs current state and forms a
request for the desired amount of pods.
15)apiserver request loop evaluates
ReplicaSet Controller request.
16) Pods published, and enter ‘Pending’ phase.

Scheduler
17)Scheduler monitors published pods with no
‘NodeName’ assigned.
18)Applies scheduling rules and filters to find a
suitable node to host the Pod.
19)Scheduler creates a binding of Pod to Node
and POSTs to apiserver.
20) apiserver request loop evaluates POST
request.
21)Pod status is updated with node binding and sets
status to‘PodScheduled’.

Kubelet -PodSync
22)The kubelet daemon on every node polls the apiserver filtering
for pods matching its own ‘NodeName’; checking its current state
with the desired state published through the apiserver.
23)Kubelet will then move through a series of internal processes to
prepare the pod environment. This includes pulling secrets,
provisioning storage, applying AppArmor profiles and other
various scaffolding. During this period, it will asynchronously be
POST’ing the ‘PodStatus’ to the apiserver through the standard
apiserver request loop.

Pause andPlumbing
24)Kubelet then provisions a ‘pause’ container via the
CRI (Container Runtime Interface). The pause
container acts as the parent container for the Pod.
25)The network is plumbed to the Pod via the CNI
(Container Network Interface), creating a veth pair
attached to the pause container and to a container
bridge (cbr0).
26)IPAM handled by the CNI plugin assigns an IP to
the pause container.

Kublet -CreateContainers
24) Kubelet pulls the container Images.
25) Kubelet first creates and starts any init containers.
26)Once the optional init containers complete, the
primary pod containers are started.

Pod Status
27)If there are any liveless/readiness probes, these are executed before
the PodStatus isupdated.
28)If all complete successfully, PodStatus is set to ready and the
container has started successfully.
The Pod is
Deployed!

Questions?

Container Orchestration using kubernetes

More Related Content

Similar to Container Orchestration using kubernetes

Similar to Container Orchestration using kubernetes (20)

More from Puneet Kumar Bhatia (MBA, ITIL V3 Certified)

More from Puneet Kumar Bhatia (MBA, ITIL V3 Certified) (17)

Recently uploaded

Recently uploaded (20)

Container Orchestration using kubernetes