Kubernetes Java Operator

© 2018 Software AG. All rights reserved.
Anthony Dahanne
@anthonydahanne
blog.dahanne.net
December 3rd 2018
A KUBERNETES OPERATOR
WRITTEN IN JAVA ?

SOME CONTEXT : YOUR PRESENTER !
„Anthony Dahanne, Software Engineer
@Terracotta, a Software AG company
„Montréal JUG leader
„Working on Terracotta cloud deployments
(Docker, Kubernetes, AWS, etc.)
„Also working on Management and Monitoring for
Terracotta products

“The Cloud”
n
…
SOME CONTEXT : YOUR PRESENTER’S MISSION
PostgreSQL
Webapp with
Ehcache3 Clustered
Terracotta Server Terracotta Server
„Ehcache3 : Open Source caching library for Java „Terracotta Server : OSS clustered backend for Ehcache3

AGENDA
• It all started with Docker
• Then came Kubernetes
• But Helm too !
• Extending Kubernetes in Java ???
• What’s next ?

DOCKERFILE !
FROM openjdk:8-jdk-alpine
LABEL maintainers="Anthony Dahanne <anthony.dahanne@softwareag.com>"
RUN wget -q http://repo1.maven.org/maven2/3.6.1/ehcache-clustered-3.6.1-kit.tgz
&& tar xvzf ehcache-clustered-3.6.1-kit.tgz -C /terracotta
COPY conf/tc-config-active-passive.xml /terracotta/server/conf/tc-config-active-passive.xml
COPY entrypoint.sh /terracotta/entrypoint.sh
# all below commands will now be relative to this path
WORKDIR /terracotta/server
# the tsa port (used by the clients to connect to the cluster)
EXPOSE 9410
# the group port (used to sync the passives with the active)
EXPOSE 9430
ENV OFFHEAP_RESOURCE1_NAME "offheap-1"
ENV OFFHEAP_RESOURCE1_UNIT "MB"
ENV OFFHEAP_RESOURCE1_SIZE "512"
# before starting the terracotta server, we update the tc-config.xml configuration file
ENTRYPOINT /terracotta/entrypoint.sh

ORCHESTRATION IN 2016 / 2017 ? DOCKERSWARM !
version: '3'
networks:
terracotta-net:
driver: overlay
services:
client:
image: terracotta/sample-ehcache-client:5.5.1
environment:
- TERRACOTTA_SERVER_URL=terracotta-1:9410,terracotta-2:9410
networks:
- terracotta-net
terracotta-1:
hostname: terracotta-1
image: terracotta/terracotta-server-oss:5.5.1
environment:
- TC_SERVER1=terracotta-1
- TC_SERVER2=terracotta-2
networks:
- terracotta-net

THEN CAME KUBERNETES
apiVersion: v1
kind: ConfigMap
metadata:
name: tc-config
data:
tc-config.xml: |
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tc-config xmlns="http://www.terracotta.org/config">
<plugins>
<config>
<ohr:offheap-resources xmlns:ohr="http://www.terracotta.org/config/offheap-resource">
<ohr:resource name="offheap-1" unit="MB">512</ohr:resource>
<ohr:resource name="offheap-2" unit="MB">128</ohr:resource>
</ohr:offheap-resources>
</config>
</plugins>
<servers>
<server host="terracotta-0.terracotta" name="terracotta-0" bind="0.0.0.0">
<logs>stdout:</logs>
<tsa-port bind="0.0.0.0">9410</tsa-port>
<tsa-group-port bind="0.0.0.0">9430</tsa-group-port>
</server>
<server host="terracotta-1.terracotta" name="terracotta-1" bind="0.0.0.0">
<logs>/stdout:</logs>
<tsa-port bind="0.0.0.0">9410</tsa-port>
<tsa-group-port bind="0.0.0.0">9430</tsa-group-port>
</server>
<client-reconnect-window>120</client-reconnect-window>
</servers>
</tc-config>
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: caching-client
name: caching-client
spec:
selector:
matchLabels:
app: caching-client
replicas: 1
template:
metadata:
labels:
app: caching-client
spec:
containers:
- image: terracotta/sample-ehcache-client:5.5.1
name: caching-client
env:
- name: TERRACOTTA_SERVER_URL
value: "terracotta-0.terracotta:9410,terracotta-1.terracotta:9410"
---
apiVersion: v1
kind: Service
metadata:
name: terracotta
labels:
app: terracotta
annotations:
# see https://github.com/kubernetes/kubernetes/issues/39363 , to have dns entries available immediately
service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
ports:
- name: terracotta-port
port: 9410
- name: sync-port
port: 9430
clusterIP: None
selector:
app: terracotta
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: terracotta
spec:
selector:
matchLabels:
app: terracotta
serviceName: terracotta
replicas: 2
template:
metadata:
labels:
app: terracotta
spec:
initContainers:
- name: init-terracotta
# be careful with busybox versions : https://github.com/docker-library/busybox/issues/48
image: busybox:1.28
# check service name resolution works fine; if it can't resolve the service, a split brain could occur
command: ['sh', '-c', 'until nslookup terracotta; do echo "waiting for terracotta to resolve"; sleep 2; done;']
containers:
- name: terracotta
command: ["bin/start-tc-server.sh"]
args: ["-f", "/config/tc-config.xml", "-n", "$(POD_NAME)"]
imagePullPolicy: Always
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
ports:
- containerPort: 9410
name: terracotta-port
- containerPort: 9430
name: sync-port
volumeMounts:
- name: config-volume
mountPath: /config
volumes:
- name: config-volume
configMap:
name: tc-config
Very verbose
No templatization

SED OR ITS KUBERNETES BROTHER, KUSTOMIZE
kustomize targets kubernetes; it understands and can patch kubernetes style API objects. It's like
make, in that what it does is declared in a ﬁle, and it's like sed, in that it emits editted text.
resources:
- kubernetes-complete-deployment.yaml
imageTags:
- name: terracotta/terracotta-server-oss
newTag: 5.4.3
kustomization.yaml
containers:
- name: terracotta
my-deployment.yaml
kustomize build | kubectl apply -f -
containers:
- name: terracotta
??????????????
???????????
?????????????
??????????????

KUBERNETES PACKAGING : HELM
• Helm is installed on the client, Tiller is the server side
• With Helm you deploy / create Charts that are run as Releases
• In a Chart, you package your Kubernetes manifests, and your dependencies
• A very notable feature is the “templatization“ of your Kubernetes manifests
APT / YUM FOR KUBERNETES
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ template "terracotta.fullname" . }}
labels:
app: {{ template "terracotta.name" . }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
app: {{ template "terracotta.name" . }}
serviceName: {{ template "terracotta.fullname" . }}
spec:
{{- if .Values.nodeSelector }}
nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
{{- end }}
containers:
- name: {{ template "terracotta.fullname" . }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}”

QUICK BACK TO BASICS : KUBERNETES ARCHITECTURE
By Khtan66 - CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=53571935

WHAT IS AN OPERATOR ?
• An Operator is a method of packaging, deploying and managing a Kubernetes
application.
• To be able to make the most of Kubernetes, you need a set of cohesive APIs to
extend in order to service and manage your applications that run on Kubernetes.
You can think of Operators as the runtime that manages this type of application
on Kubernetes.
• Operators are purpose-built to run a Kubernetes application, with operational
knowledge baked in. They will be smarter and more tailored than generic tools.
from : https://coreos.com/operators/

KUBERNETES OPERATOR IN JAVA
• Operators (or controllers) provide better user experience for deploying and
managing complex applications like databases (PostgreSQL, Terracotta server, etc.)
• They can create and manage their own Custom Resource Definitions (CRDs)
- or provide a CLI or UI via their own REST endpoints
USING FABRIC8 OR KUBERNETES JAVA SDK
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-client</artifactId>
<version>4.0.3</version>
</dependency>
<dependency>
<groupId>io.kubernetes</groupId>
<artifactId>client-java</artifactId>
<version>3.0.0-beta2</version>
</dependency>
Service tmcService = new ServiceBuilder()
.withNewMetadata()
.withName("tmc")
.endMetadata()
.withNewSpec()
.addNewPort()
.withName("tmc-port")
.withPort(9480)
.endPort()
.withType("LoadBalancer")
.addToSelector("app", "tmc")
.endSpec()
.build();

TERRACOTTA OPERATOR ARCHITECTURE
Kubernetes Cluster
Terracotta Operator
REST API
CLI / Web UI
K8S API Server
Java SDK
REST calls
tc-configs
operator config
Terracotta
ServerTerracotta
Server
ConfigMaps
Services,
StatefulSets
A PRIVILEGED POD THAT LISTENS TO THE USER

TERRACOTTA OPERATOR ARCHITECTURE
Kubernetes Cluster
Terracotta Operator
kubectl apply
K8S API Server
Java SDK
Watch
tc-configs
operator config
Terracotta
ServerTerracotta
Server
ConfigMaps
Services,
StatefulSets
A PRIVILEGED POD THAT LISTENS TO THE API SERVER

WHAT’S NEXT ?
• Kubectl plugins : to integrate your operator into kubectl
• Service catalog : to allow your users to just “require” your software
LINKS AND OTHER REFERENCES
• https://github.com/Terracotta-OSS/docker
• https://github.com/helm/charts/tree/master/stable/terracotta
• https://github.com/Terracotta-OSS/terracotta-operator
• The fullstack demo app (and its jib, kubernetes, helm, scaffold files) is on Github

Kubernetes Java Operator

More Related Content

Kubernetes Java Operator