Introduction to DevOps and the Practical Use Cases at Credit OK
- 1. Presented by Kriangkrai Chaonithi @spicydog 11/11/2019 | KMUTT | Computer Engineer | Software Engineer Introduction to DevOps and the Practical Use Cases at
- 2. Hello! My name is Gap Education ● MS Computer Engineering (KMUTT) ● BS Applied Computer Science (KMUTT) Work Experience ● CTO & co-founder at Credit OK ● Former R&D Manager at Insightera ● Former Android, iOS & PHP Developer at Longdo.COM Fields of Interests ● Software Engineering ● Cloud Architecture & Distributed Computing ● Computer Security ● Machine Learning & NLP https://spicydog.me
- 3. Agenda ● What is DevOps? ○ Why DevOps is so popular? ○ What do DevOps engineers do? ○ DevOps Skill Set ● Where & how we deploy web application? ● DevOps Technologies ○ Version Control Software ○ CI/CD Pipeline ○ Container Technology ○ Container Orchestration ○ Logging & Monitoring ● DevOps at Credit OK ○ Technology Stack ○ Cloud Technology & DevOps Integration ● The modern software development obstacles ● Summary
- 7. Why DevOps is Popular? ● Direct productivity improvement ● Technology advancement ● Open source communities ● Modern technology hype https://unsplash.com/photos/0VwOXfSbBNY
- 8. What do DevOps engineers do? ● Build Docker images for the team ● Develop CI/CD pipeline ● Create and maintain deployment environment ● Deploy application to the cluster ● Monitor errors, bugs and report to the team https://unsplash.com/photos/QBpZGqEMsKg
- 9. DevOps Skill Set ● Computer Infrastructure ● Tools & Technology Embracement ● Computer Security ● Communication https://unsplash.com/photos/842ofHC6MaI
- 10. Where and How we Deploy Web Application? https://unsplash.com/photos/NVnU0WIhC28
- 11. How to Deploy a Web Application? Compiled Languages - Compile the codes to executable - Upload executable to the server Interpreted Languages - Upload (or `git pull`) source codes on the server
- 12. Bare Metal Server ● Pre-cloud era ● Install OS and dependencies on a machine ● One machine - one server ● Expose the network to the internet ● Colocation/on-premise ● SSH/FTP/Git to the server
- 13. Virtualization ● One machine - many servers ● One machine multiple customers ● VPS / Cloud ● SSH/FTP/Git to the server IaaS
- 14. Containers ● Docker / Kubernetes / KNative ● Several containers in a node ● Auto deployment ● Auto scale ● Pay on number of nodes ● Infrastructure as code! (IaC) PaaS/ CaaS
- 15. Serverless Functions ● Write your code and deploy! ● Auto deploy ● Auto scale ● Pay per request/computation ● No infrastructure!! FaaS
- 16. Some Interesting Deployment Platforms CaaSPaaS FaaS
- 20. CI/CD Pipeline
- 28. Benefits of using Docker ● Consistent development, testing, and production environments ● Docker Engine can run on Linux, (MacOS, and Windows) ● Language and application dependencies are Pre-install from docker images ● Run different versions of same programming language on the same machine
- 33. Deployment & Container Orchestration
- 34. What is Kubernetes? https://blog.risingstack.com/what-is-kubernetes-how-to-get-started/ Infrastructure as Code (IaC) Design Principles - Workload Scalability - High Availability - Security - Portability Key Components - Pod - Deployment - Replica
- 36. Warning! Kubernetes Services are Complicated Please read and understand this link before your first deployment! https://medium.com/@metaphorical/internal-and-external-connectivity-in-kubernetes-space-a25cba822089
- 41. Credit Scoring Platform on Big Data Analytics creditok.co
- 42. Data Engineer & DevOps Our Technology Team Data Science Application Development
- 43. How Credit OK Deploys Application with
- 44. Dev Computer Container Registrydocker-compose DEV Source Code Source Code Container (nginx + php) Base Image Service Image (MySQL) Sidecar Container (MySQL) Web Browser mount volume volume network load load expose Use Container on Development Env MySQL Storage
- 45. GitLab Container RegistryGitLab Runner DEV Git Source Code Container (nginx + php) Base Image Service Image (MySQL) Sidecar Container (MySQL) MySQL Storage clone volume volume network load load push Result Use Container on GitLab CI
- 46. GitLab Container RegistryGitLab Runner DEV Git Source Code Container (nginx + php) Base Image clone volume load push Result COPY ./ ./ Container (nginx + php + src) App Image Build Docker Image for Production
- 47. Deployment Kubernetes (GKE)Container Registry GitLab Runner Google Cloud SDK Container Prev App Container Deploy the Docker Image Google Cloud SDK Image Patch New App Image kubectl App Image New App Container auth
- 51. The Modern Software Development Obstacles https://unsplash.com/photos/Cd2QnIKU6dk
- 52. ● Environment variables are crucial, make your application flexible with .env ● Never use persistent storage on Container if you don't truly understand what you are doing! ○ Use GCS to store users’ files ○ Store session and cache in a centralized database ○ If you can design application logic w/o cache invalidation, caching in the Pod is okay ● Write log to external logging service is a must ● Enforce Git Workflow (develop, staging, production branches) ● Please read how to make Docker Image best practice. Beware of Docker Image cache! At least, DO NOT use latest tag, it might be a time bomb, pin the version if possible. Warning! What to concern before going Container
- 53. ● Write unit testing as much as possible, you will never sleep without it ● Every git push, the pipeline run, you have to wait ● Every time you want to deploy, you have to wait (no more save and upload to deploy right away) ● Make sure you do good logging for debugging ● Make sure you can rollback the wrong deployment quickly Warning! What to concern before using CI/CD Pipeline
- 54. Summary ● DevOps is about automating development operation ● DevOps can ensure software quality by automated testing ● DevOps can improve team productivity with CI/CD ● At large scale, there is no more manual deploy ● Therefore we do IaC to automate infrastructure ● When infrastructure is automated, we need robust logging system, plan thoroughly ● Systematic logging leads to great monitoring system ● Systematic logging drives data driven culture where we make decision based on data ● DevOps engineer is in high demand with specialized skills ● GCP is free $300 first 1st year, GitLab is free forever, go register and practice. Learning DevOps is free!
- 56. Time is short, let’s utilize the networks. Feel free to connect with me via spicydog.me