Running Containerised Applications at Scale on AWS
•
3 likes•1,696 views
This document discusses running containerized applications at scale on AWS. It begins by explaining why containers are used and the challenges of scaling container workloads, such as resource and state management, monitoring, service discovery, and deployment. It then provides an overview of Amazon ECS concepts like clusters, tasks, and services. The rest of the document discusses how to address the scaling challenges with services like Application Auto Scaling, service discovery options, monitoring with CloudWatch, logging with CloudWatch Logs, task scheduling and placement strategies, IAM roles, and demoing an application on ECS.
Report
Share
Running Containerised Applications at Scale on AWS
- 1. Running Containerised Applications at scale on AWS
- 2. What to Expect From This Session • Why Containers & how they are used • Challenges of scaling containerised workloads • How to scale containerised workloads using Amazon ECS
- 8. Monolithic vs SOA vs Microservices SOA Coarse-grained Microservices Fine-grained Monolithic Single Unit
- 9. Monolithic Architecture Order UI User UI Shipping UI Order Service User Service Shipping Service Data Access
- 12. Microservices Architecture Order UI User UI Shipping UI Order Service User Service Shipping Service
- 13. Scaling Microservices Order UI User UI UI Order Service Service Shipping Service Order UI Order UI User UI UIShipping UI Order Service Order Service Service Service Service Service User Service Shipping Service
- 14. Challenges of Scaling Containerised Workloads
- 15. Going from this… Server Guest OS Bins/Libs Bins/Libs App2App1
- 16. ..to this – is hard! Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS Server Guest OS
- 17. Challenges running containers at scale Resources and State Management Monitoring Service Discovery Deployment
- 18. Challenges running containers at scale Resources and State Management Monitoring Service Discovery Deployment
- 21. Amazon ECS Concepts Regional Resource Pool Grouping of Container Instances Start Empty, Scale Dynamically
- 22. Amazon ECS Concepts - Clusters EC2 INSTANCES ECS AGENT ECS AGENT AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT ECS AGENT
- 23. Amazon ECS Concepts - Clusters EC2 INSTANCES ECS AGENT ECS AGENT AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT ECS AGENT
- 24. Amazon ECS Concepts - Tasks •Unit of work •Grouping of related containers •Run on container instances
- 25. Amazon ECS Concepts - Tasks EC2 INSTANCES ECS AGENT ECS AGENT AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT ECS AGENT TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container
- 26. Amazon ECS Concepts - Tasks EC2 INSTANCES ECS AGENT ECS AGENT AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT ECS AGENT TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container
- 27. Amazon ECS Concepts - Services •Good for long-running applications •Load balance traffic across containers •Automatically recover unhealthy containers •Discover services
- 28. Amazon ECS Concepts - Services LOAD BALANCER Internet EC2 INSTANCES ECS AGENT ECS AGENT AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT ECS AGENT TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container LOAD BALANCER
- 29. Amazon ECS Concepts - Services Internet EC2 INSTANCES ECS AGENT ECS AGENT AGENT COMMUNICATION SERVICE Amazon ECS API CLUSTER MANAGEMENT ENGINE KEY/VALUE STORE ECS AGENT ECS AGENT TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container TASK Container LOAD BALANCER LOAD BALANCER
- 30. Application Auto Scaling AMAZON ECS AMAZON CLOUDWATCH ELASTIC LOAD BALANCING Auto Scaling ECS cluster
- 31. Application Auto Scaling AMAZON ECS Publish metrics AMAZON CLOUDWATCH ELASTIC LOAD BALANCING Auto Scaling ECS cluster Scale in / out policies
- 32. Application Auto Scaling AMAZON ECS Publish metrics AMAZON CLOUDWATCH ELASTIC LOAD BALANCING Auto Scaling ECS cluster Add/remove ECS tasks Scale in / out policies
- 33. Application Auto Scaling AMAZON ECS Publish metrics AMAZON CLOUDWATCH ELASTIC LOAD BALANCING Auto Scaling ECS cluster Add/remove ECS tasks Scale in / out policies
- 34. Application Auto Scaling AMAZON ECS Publish metrics AMAZON CLOUDWATCH ELASTIC LOAD BALANCING Auto Scaling ECS cluster Add/remove ECS tasks Scale in / out policies
- 35. Application Auto Scaling AMAZON ECS Publish metrics AMAZON CLOUDWATCH ELASTIC LOAD BALANCING Auto Scaling ECS cluster Add/remove ECS tasks Scale in / out policies
- 36. Flexible Container Placement •Applications •Batch Jobs •Multiple Schedulers
- 37. Designed for Use with Other AWS Services • Elastic Load Balancing (ELB) • Application Load Balancing (ALB) • Amazon Elastic Block Store (EBS) • Amazon Virtual Private Cloud (VPC) • AWS Identity and Access Management • AWS CloudTrail
- 39. Scalable
- 41. Challenges running containers at scale Resources and State Management Monitoring Service Discovery Deployment
- 42. Service Discovery with ALB TASK A TASK A TASK A Application Load Balancer
- 43. Service Discovery with DNS https://github.com/awslabs/service-discovery-ecs-dns TASK A TASK B TASK C ecssd_agent ecssd_agent Route53 Private Hosted Zone
- 44. Service Discovery with DNS https://github.com/awslabs/service-discovery-ecs-dns TASK A TASK B TASK C ecssd_agent ecssd_agent Route53 Private Hosted Zone
- 45. Service Discovery with Weaveworks •DNS interface for cross-host container communication •Gossip protocol to share grouped updates •Overlay network between hosts
- 46. Service Discovery with Weaveworks HTTP Server Data Producer Docker Weave Router/DNS Weave Proxy ECS Agent ECS Task HTTP Server Data Producer Docker Weave Router/DNS Weave Proxy ECS Agent ECS Task HTTP Server Data Producer Docker Weave Router/DNS Weave Proxy ECS Agent ECS Task Amazon ECS Infrastructure
- 47. Service Discovery with Consul • Consul Agent – Runs on each node, responsible for checking the health of the services and of the node itself. • One or more Consul servers – Store and replicate data, leader elected using the Raft consensus algorithm • Registrator agent – Automatically register / deregisters services based on published ports and metadata from the container environment variables defined in the ECS task definition
- 48. Service Discovery with Consul Back end 1 Back end 2 Registrator Consul Agent Front end Registrator Consul Agent Consul-Server ECSClusterECSCluster
- 49. Challenges running containers at scale Resources and State Management Monitoring Service Discovery Deployment
- 50. Monitoring with Amazon CloudWatch • Metric data sent to CloudWatch in 1-minute periods and recorded for a period of two weeks • Available metrics: CPUReservation, MemoryReservation, CPUUtilization, MemoryUtilization • Available dimensions: ClusterName, ServiceName
- 51. Monitoring with Amazon CloudWatch
- 52. Monitoring with Amazon CloudWatch • Use the CloudWatch monitoring scripts to monitor additional metrics, e.g. disk space: # Edit crontab > crontab -e # Add command to report disk space utilization to CloudWatch every five minutes */5 * * * * <path_to>/mon-put-instance-data.pl --disk-space-util --disk-space-used --disk- space-avail --disk-path=/ --from-cron
- 53. Logging with CloudWatch Logs { "image": ”nginx:latest", ... "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": ”nginx", "awslogs-region": "us-east-1" } } { • Defined within the task definition • Available log drivers: • awslogs • fluentd • splunk • syslog
- 54. Challenges running containers at scale Resources and State Management Monitoring Service Discovery Deployment
- 55. Scheduling Containers: Long Running App •Deploy using least space: minimumHealthyPercent = 50%, maximumPercent = 100% Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Task 1 Task 2 Task 3 Task 4
- 56. Scheduling Containers: Long Running App •Deploy using least space: minimumHealthyPercent = 50%, maximumPercent = 100% Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Task 1 Task 2 Task 3 Task 4
- 57. Scheduling Containers: Long Running App •Deploy using least space: minimumHealthyPercent = 50%, maximumPercent = 100% Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Task 1 Task 2 Task 3 Task 4
- 58. Scheduling Containers: Long Running App •Deploy quickly without reducing service capacity:: minimumHealthyPercent = 100%, maximumPercent = 200% Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Instance 8 Task 1 Task 2 Task 3 Task 4
- 59. Scheduling Containers: Long Running App •Deploy quickly without reducing service capacity:: minimumHealthyPercent = 100%, maximumPercent = 200% Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Instance 8 Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8
- 60. Scheduling Containers: Long Running App •Deploy quickly without reducing service capacity:: minimumHealthyPercent = 100%, maximumPercent = 200% Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Instance 8 Task 5 Task 6 Task 7 Task 8
- 61. Scheduling Containers: Long Running App • Blue-Green Deployments • Define two ECS services • Each service is associated w/ ELB • Both ELBs in Route 53 record set with weighted routing policy, 100% primary, 0% secondary • Deploy to blue or green service and switch weights TASK A TASK B Route53 Record Set With Weighted Routing Policy 100% 0%
- 62. IAM Roles for Tasks DynamoDB ECS Cluster EC2 Instance EC2 Instance TASK A TASK B TASK B S3 Storage
- 64. ECS Placement Strategies Binpacking Spread Affinity Distinct Instance
- 65. Placement: Targeting Instance Type g2.2xlarge t2.small g2.2xlarge g2.2xlarge
- 66. Placement: Targeting Instance Type & Zone g2.2xlarge t2.small t2.micro t2.medium t2.medium t2.small g2.2xlarge t2.small t2.small t2.medium us-east-1aus-east-1d
- 67. Placement: Spread across Zone and Binpack g2.2xlarge t2.small t2.micro t2.medium t2.medium t2.small g2.2xlarge t2.small us-east-1aus-east-1d g2.2xlarge t2.medium t2.micro t2.small us-east-1c
- 68. Placement: Affinity and Anti-Affinity g2.2xlarge t2.small t2.micro t2.medium t2.medium t2.small g2.2xlarge t2.small us-east-1aus-east-1d g2.2xlarge t2.medium t2.micro t2.small us-east-1c
- 69. Placement: Distinct Instances t2.medium g2.2xlarge t2.micro t2.small t2.small t2.small g2.2xlarge t2.small t2.small t2.small g2.2xlarge t2.small
- 71. AWS CodeCommit AWS CodePipeline AWS CodeBuild Amazon ECR Amazon ECS instance Spot Instance AWS CloudFormation 1. Commit Code 2. Trigger Pipeline 3.Build Artifact 5. Update Stack 6. Update Service 4. Push Image Reference Architecture
- 72. New Features! • ECS adds console support for Spot fleet creation • https://aws.amazon.com/about-aws/whats-new/2017/06/amazon-ecs- adds-console-support-for-spot-fleet-creation/ • ECS adds support for time and event based scheduling • https://aws.amazon.com/about-aws/whats-new/2017/06/amazon-ecs-now- supports-time-and-event-based-task-scheduling/
- 73. Thank You