Golang For DevOps And Cloud Engineers

What is Go
• From the of cial Go homepage (https://go.dev/):

• Build fast, reliable, and ef cient software at scale

• Go is an open source programming language supported by Google

• Easy to learn and get started with

• Built-in concurrency and a robust standard library

• Growing ecosystem of partners, communities, and tools

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
 Who am I
• My name is Edward Viaene

• I’m a DevOps & Cloud specialist and Training Instructor

• I started publishing on Udemy in 2015 and have now more than 250,000
students enrolled in one of my DevOps / Cloud courses

• Since 2017 I have been using Go extensively, as it became more popular

in the DevOps / Cloud space

• After years of writing Go code, I feel comfortable now to create this

course, and to teach you all the Go tips & tricks I discovered over the
years
Golang For DevOps and Cloud Engineers - Edward Viaene
 Course Objectives
• To be able to read, understand and write Go code

• To be able to write enterprise ready applications

• To be able to write applications that integrate REST APIs

• To be able to write applications that integrate with a cloud provider

• To be able to write applications that integrate with Kubernetes

• To be able to write applications that integrate with any custom integration

that has a Go SDK available

Golang For DevOps and Cloud Engineers - Edward Viaene

 Course Layout

Integrating with
SDKs, Cloud
Understanding Go by
Providers,
example
Go Concepts Kubernetes by
learning how to use
(http-get application)
external packages
(libraries)

Golang For DevOps and Cloud Engineers - Edward Viaene

 Course Files
• A link to all course les can be found in the next lecture: Source les and
useful information

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
Visual Studio Code Installation
First Go application
 First Go Application
Command line
Helloworld http.Get(…) JSON Parsing
arguments

The ag Custom Error

http.Post(…) Functions
package Handling

JWT Full working

Packaging Testing
Authentication example

Golang For DevOps and Cloud Engineers - Edward Viaene

fl
 http-get client

GET /words
GET /occurrence
test-server
http-get client
(API server)

POST /login

Golang For DevOps and Cloud Engineers - Edward Viaene

 JWT Auth

GET /words

http-get client test-server

{ “page”: “words”, … }

Golang For DevOps and Cloud Engineers - Edward Viaene

 JWT Auth

GET /words

test-server
http-get client
password: xyz
401 Unauthorized

Golang For DevOps and Cloud Engineers - Edward Viaene

 JWT Auth
POST /login
Data: {“password”: “xyz” }

Data: {“token”: “eyJhbGci…” }

test-server
http-get client
password: xyz
GET /words with Authorization
header: Bearer eyJh…

{ “page”: “words”, … }

Golang For DevOps and Cloud Engineers - Edward Viaene

Arrays and Slices
 Arrays and Slices
• Arrays have a xed length whereas slices are dynamic

Array: Slice:

var buffer [7]byte var buffer []byte

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
 Arrays and Slices
• Arrays are the building block of slices

Array: Slice:

var arr1 [7]int = [7]int{7,3,6,0,4,9,10} var arr2 []int = arr1[1:3]

7 3 6 0 4 9 10 3 6

Length: 7 Length: 2
Capacity: 7 Capacity: 6
Element 0: arr1[1]

Golang For DevOps and Cloud Engineers - Edward Viaene

(Cross)-Compiling and cgo
 Building Go Applications
• Go can cross-compile to any supported OS and Architecture

• You need to supply GOOS and GOARCH during “go build”

• “go tool dist list” shows you supported combinations

• When not cross compiling, cgo will be enabled, when cross-compiling it’ll be disabled

• cgo allows you to run C code within Go

• This is relevant even if you’re not using this feature yourself, because standard
Go packages like “net” can use cgo (for example for DNS resolving)

• cgo will link your binary to the current C library available on your operating
system, but it’ll not work on an OS with a different C library

Golang For DevOps and Cloud Engineers - Edward Viaene

 Building Go Applications
Go Binary

CGO_ENABLED=1 go build

glibc/libc/musl
libc system
library
Result is a dynamically linked binary
(External les)

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
 Building Go Applications
Go Binary

Pure go
implementation
of C libraries CGO_ENABLED=0 go build

Result is a statically linked binary

Golang For DevOps and Cloud Engineers - Edward Viaene

 Building Go Applications
• Enabling CGO (when you’re not cross compiling)

• May lead to a binary smaller in size (as C bindings for DNS Resolver,
networking will be in libc/glibc/…)

• You already have the C libraries bundled with your OS, there’s no
need to have them included again in every binary

• Will lead to faster builds

Golang For DevOps and Cloud Engineers - Edward Viaene

 Building Go Applications
• Disabling CGO

• Is necessary when cross-compiling

• Is also necessary if your C library on the destination system is different

(for example you compile on Ubuntu Linux but run on Alpine)

• Ubuntu is using GNU Libc and Alpine musl libc

Golang For DevOps and Cloud Engineers - Edward Viaene

aws-sdk-go
 Amazon Web Services

GET /?Action=DescribeRegions
&AllRegions=true
&AUTHPARAMS
AWS Endpoints
http client ec2.amazonaws.com
s3.amazonaws.com
<DescribeRegionsResponse xmlns="http://ec2.amazonaws.com/doc/2016-11-15/">
<requestId>59dbff89-35bd-4eac-99ed-be587EXAMPLE</requestId>
<regionInfo>
…
</regionInfo>
</DescribeRegionsResponse>

Golang For DevOps and Cloud Engineers - Edward Viaene

 AWS SDK
Open AWS Create IAM Download Con gure
Account User Credentials credentials

AWS CLI Environment

Variables

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
 AWS SDK

Create SSH Find Ubuntu Output instance

Load Con g Launch EC2
KeyPair AMI ID

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
 Kubernetes

REST Calls using x509 certi cates.

Con guration in ~/.kube/con g
Kubernetes API

(minikube, AWS EKS,

Go + k8s.io/client-go
kops, Google
Kubernetes API server responses Kubernetes Engine,
(JSON) Azure Kubernetes, …)

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
fi
SSH in go
 SSH Server
Veri es client using mykey.pub
(authorized_keys)
authentication with mykey.pem

SSH Client (putty,

SSH Server in Go
openssh, ssh in go, …)

SSHv2

Veri es server using

server.pub (known_hosts)

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
Microsoft Azure
 Azure Go SDK
Create Azure Download Azure Credentials con gured
Run az login
Account CLI in $HOME/.azure

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
 Azure Go SDK
Create SSH Retrieve Token Create
Keys from con g ResourceGroup

Create vnet + Create Network

Create public IP
subnet(s) Security Group

Create Network
Create VM
Interface

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
Identity Providers
OpenID Connect
 What is an IdP
• Simply put, an Identity Provider (IdP) manages and maintains identity data
for users

• It’s often used in conjunction with Single Sign On (SSO)

• It gives a user a single login & password (and optional MFA capability)

• It can be used for multiple applications and websites

• While very convenient for the end-user, it’s also more secure

• 2 often used implementations for authentication within an Identity Providers

setup are OIDC and SAML

Golang For DevOps and Cloud Engineers - Edward Viaene

 What is an IdP
• OIDC (OpenID Connect) and SAML (Security Assertion Markup Language) are
authentication mechanisms, they don’t store login/password information
themselves

• You’d still need to validate the login, password, and potentially MFA token with
a separate mechanism

• Users can be in a database, in LDAP, Microsoft Active Directory, or others

• SAML 2.0 was released in 2005, while OpenID Connect (OIDC) was launched
in 2015

• SAML is still used a lot, but OpenID Connect is more lightweight and much
easier to implement
Golang For DevOps and Cloud Engineers - Edward Viaene
 Why implement OIDC
• It’s a great learning experience

• Exposure to a lot of technologies: REST, OAuth, JWT, JWK

• You’re often exposed to an IdP, and it’s worth understanding the inner
workings

• You can build your own IdP authorization server, client, or application

• Understanding how the how ow works will help you when you need
to build one of these components

Golang For DevOps and Cloud Engineers - Edward Viaene

fl
 What is OIDC
• OIDC is short for OpenID Connect

• It’s a simple identity layer on top of the OAuth 2.0 protocol

• OIDC can verify the identity of a user using an Authorization Server

• OIDC uses REST endpoints, so it’s easily implemented

• OIDC uses JSON and JSON Web Tokens (JWT)

Golang For DevOps and Cloud Engineers - Edward Viaene

 OIDC Flows
• Authorization Code Flow

• For web applications that can store a client_secret

• This is the ow we’re going to implement

• Implicit ow

• For frontends / mobile apps that can’t store a client_secret

• Hybrid ow

• Combines above ows

• Immediate access to an ID token

Golang For DevOps and Cloud Engineers - Edward Viaene
fl
fl
fl
fl
 OIDC
3. Authorization Code Request
1. Access Protected Resource
or Login Request (Web) Application

7. Exchange code /authorize

for token endpoint
2. Redirect to
Authorization Server /token
8. Access & ID Tokens endpoint
6. URL with code
Parameter Authorization
User
Server
4. Login prompt

5. Redirect to the application

with “?code=“ query parameter

Golang For DevOps and Cloud Engineers - Edward Viaene

 OIDC
/authorize
endpoint

/authorization?
client_id=1-2-3-4 /token
&redirect_uri=http://localhost:8081/callback
&scope=openid endpoint
&response_type=code
&state=randomstring
User Authorization
Server

/login

Golang For DevOps and Cloud Engineers - Edward Viaene

 OIDC
/authorize
endpoint
Redirect: http://localhost:8081/callback?code=…&state=randomstring

/token
endpoint

User Authorization
Server

(Web) Application
POST /token
grant_type=authorization_code
&client_id=1-2-3-4
&client_secret=secret
&redirect_uri=https://
localhost:8081/callback
&code=…

Golang For DevOps and Cloud Engineers - Edward Viaene

 OIDC
/authorize
endpoint

/token
endpoint

User Authorization
Server

(Web) Application

GET /jwks.json

Golang For DevOps and Cloud Engineers - Edward Viaene

 OIDC Discovery
/authorize
endpoint
GET /.well-known/openid-
configuration
(Web) Application
/token
endpoint

User Authorization
Server
{
"issuer": "http://localhost:8080",
"authorization_endpoint": "http://localhost:8080/authorization",
"token_endpoint": "http://localhost:8080/token",
"userinfo_endpoint": "http://localhost:8080/userinfo",
"jwks_uri": "http://localhost:8080/jwks.json",
"scopes_supported": [
Discovery
],
"oidc" endpoint
"response_types_supported": [
"code"
],
"token_endpoint_auth_methods_supported": [
"none"
]
}

Golang For DevOps and Cloud Engineers - Edward Viaene

 OIDC
/authorize
endpoint

/token
endpoint

User Authorization
Server

/userinfo
(Web) Application
endpoint

GET /userinfo

Authorization: Bearer $ACCESS_TOKEN

Golang For DevOps and Cloud Engineers - Edward Viaene

 Challenge
• If you’d like, you can write the OIDC implementation yourself rst

• I’ll include instructions to step-by-step write the implementation

• The start project contains already the function signatures, and I have
written unit tests for those function to be able to test the validity of your
code

• The start project is located in GitHub under oidc-start (https://github.com/

wardviaene/golang-for-devops-course)

• The solution code is in the folder oidc-demo

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
 Using OIDC
• Now that we have an OIDC compatible authorization server we can start
adding applications that support OIDC

• There are a lot of (SaaS) applications that support OIDC (often next to SAML)

• There’s also companies that can act as an OIDC Provider itself, like Google,
Apple, Facebook (social media logins have OIDC capabilities)

• You could either use their authorization server and trust their token, or
write an integration to validate a successful social login, and issue your
own token with your own server

• Often plug-ins are available to existing tools and software to implement

OIDC
Golang For DevOps and Cloud Engineers - Edward Viaene
 Using OIDC
• In the next lectures, I’ll show the OIDC integration with:

• Jenkins, a popular CI/CD tool

• IAM Federation with OIDC in AWS

• We’ll make AWS trust our IDToken to issue access keys to our users

Golang For DevOps and Cloud Engineers - Edward Viaene

TLS
 TLS
• TLS stands for Transport Layer Security and is used for data encryption

• Web encryption typically uses TLS to encrypt communication between

client and server

• TLS is the successor of SSL (Secure Socket Layer)

• The default port for unencrypted http traf c is 80, the default port for
encrypted (https) traf c is 443

• TLS itself is not an encryption algorithm, but a protocol to negotiate and

agree on a common set of encryption and hashing algorithms (called
the cipher suite)
Golang For DevOps and Cloud Engineers - Edward Viaene
fi
fi
 TLS
• With TLS enabled, the http server offers the client an X.509 certi cate, which can
be validated by the client to ensure the server can be trusted

• The hostname of the server will be included in the server certi cate

• The server certi cate will be signed by a Certi cate Authority (CA)

• If the client can validate the server certi cate, we can trust the server

• To be able to validate the certi cates, we’ll need to always have the certi cates
of the Certi cate Authorities that can sign the certi cates (also called the root
certi cates)

• Browsers typically have this list built-in, and within Go, it’ll also look for those
les in hardcoded system paths to be able to validate certi cates
Golang For DevOps and Cloud Engineers - Edward Viaene
fi
fi
fi
fi
fi
fi
fi
fi
fi
fi
fi
fi
 TLS
• This is all client-server communication where the server offers the client a
certi cate that can be validated

• This is called 1-way TLS

• You can also setup 2-way TLS or mutual TLS (mTLS)

• In this scenario communication can only be established when the client

also has an X.509 certi cate

• This is used often in server-to-server communication, for example to

secure communication between microservices

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
 TLS
• In the following lectures, I’m going to add TLS support to a simple Go http server

• There are multiple strategies to implement TLS:

• Using a self-signed certi cate (we will issue the Certi cate Authority certi cate
ourselves, so only someone who has this speci c CA certi cate will be able to
validate our server certi cate)

• Using a “real” certi cate issued by a company that can sign with a root
certi cate (DigiCert, GeoTrust, RSA, GlobalSign, …)

• Using Let’s Encrypt, a nonpro t Certi cate Authority

• All these approaches can be used for 1-way TLS. For 2-way TLS a self-signed CA is
common, but the other approaches would also work
Golang For DevOps and Cloud Engineers - Edward Viaene
fi
fi
fi
fi
fi
fi
fi
fi
fi
fi
 TLS
Strategies (self-signed, signed by root CA, Let’s Encrypt)
 Self Signed CA
CA Certi cate
Certi cate Certi cate
To validate Server Certi cate Authority Authority
Certi cate Key

Sign Certi cate

TLS

Server Server
Client
Certi cate

Server
Key

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
fi
fi
fi
fi
fi
 Root Signed CA
Sign Certi cate
We don’t have Certi cate based on
access to the Authority Certi cate Signing Request (CSR)
Company
CA Key Unsigned Server
Certi cate (CSR)

Root Certi cates

TLS

Server Server
Client
Certi cate

Server
Key

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
fi
fi
fi
fi
 Let’s encrypt
We don’t have
access to the Let’s Encrypt
Unsigned Server
CA Key Certi cate (CSR)
ACME*
(port 80)
Sign Certi cate
based on
Certi cate Signing Request (CSR)
http://domain.com/.well-known/acme-
challenge/<TOKEN>
Root Certi cates
TLS

Server Server
Client
Certi cate

Server
Key

*Automatic Certi cate Management Environment

Golang For DevOps and Cloud Engineers - Edward Viaene
fi
fi
fi
fi
fi
fi
 Mutual TLS
CA Certi cate Certi cate
CA Certi cate
To validate Server Certi cate Authority To validate Server Certi cate
Certi cate

Sign Certi cate Certi cate Sign Certi cate

Authority
Key

TLS

Client Server Server

Client
Certi cate Certi cate

Client Server
Key Key

Golang For DevOps and Cloud Engineers - Edward Viaene

fi
fi
fi
fi
fi
fi
fi
fi
fi
fi
fi