WKS401 Deploy a Deep Learning Framework on Amazon ECS and EC2 Spot Instances
•
1 like•509 views
Deep learning is an implementation of machine learning that uses neural networks to solve difficult and complex problems, such as computer vision, natural language processing, and recommendations. Due to the availability of deep learning libraries and frameworks, developers have the ability to enhance the capabilities of their applications and projects. In this workshop, you learn how to build and deploy a powerful deep learning framework called MXNet on containers. The portability and resource management benefit of containers means developers can focus less on infrastructure and more on building. The labs start by demonstrating the automation capabilities of AWS CloudFormation to stand up core infrastructure; as an added bonus, you use Spot Fleet to leverage the cost benefits of using Spot Instances, especially for developer environments. Then, you walk through creating an MXNet container in Docker and deploying it with Amazon ECS. Finally, you walk through an image classification demo of MXNet to validate that everything is working as expected. Pre-reqs: Laptop and AWS account
WKS401 Deploy a Deep Learning Framework on Amazon ECS and EC2 Spot Instances
- 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Chad Schmutzer, Solutions Architect David Kuo, Solutions Architect Shawn O’Connor, Solutions Architect Rajeev Srinivasan, Solutions Architect Kashif Imran, Solutions Architect Karthik Kumar Odapally, Solutions Architect April 2017 Deploy a Deep Learning Framework on Amazon ECS and Spot Instances
- 2. What to Expect From This Workshop • Introduce MXNet • Containers • Overview of Amazon ECS + Amazon ECR • Overview of AWS CloudFormation • Overview of Amazon EC2 Spot Fleet / Spot Instances • Hands-on workshop
- 3. • Open source, deep learning framework - https://github.com/dmlc/mxnet • Define, train, and deploy deep neural networks • Highly scalable – single/multiple hosts, CPU/GPU support • Support for multiple languages What Is MXNet? OutputInput
- 4. Why Containers? • Increase infrastructure utilization • Environment isolation and fidelity • Run diverse applications on shared hardware • Changes are tracked • Easy to deploy
- 5. Amazon ECS + Amazon ECR
- 6. Amazon ECS Benefits Cluster management made easy Flexible scheduling Integrated and extensible Security Performance at scale
- 7. Amazon ECS Architecture Docker Task Container instance Amazon ECS Container ECS agent ELB Internet ELB User / Scheduler API Cluster Management Engine Task Container Docker Task Container instance Container ECS agent Task Container Docker Task Container instance Container ECS agent Task Container AZ 1 AZ 2 Key/Value Store Agent Communication Service
- 8. What Is Amazon ECR? Amazon EC2 Container Registry (Amazon ECR) is a fully- managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. Amazon ECR is integrated with Amazon EC2 Container Service (Amazon ECS), simplifying your development to production workflow. Learn more: https://aws.amazon.com/ecr/
- 9. How Does Amazon ECS Use Amazon ECR? Amazon ECR Subnet Amazon ECS Spot Instance
- 11. Template CloudFormation Stack JSON or YAML formatted file Parameter definition Resource creation Configuration actions Configured AWS resources Comprehensive service support Service event–aware Customizable Framework Stack creation Stack updates Error detection and rollback CloudFormation – Components & Technology
- 12. CloudFormation Benefits Templated resource provisioning Infrastructure as code Declarative and flexible Easy to use
- 13. CloudFormation Use Cases Stack replication Infrastructure scale out Blue-green deployments
- 14. Why Do Customers Use CloudFormation? Developers/DevOps teams value CloudFormation for its ability to treat infrastructure as code, allowing them to apply software engineering principles, such as SOA, revision control, code reviews, integration testing to infrastructure. IT Admins and MSPs value CloudFormation as a platform to enable standardization, managed consumption, and role-specialization. ISVs value CloudFormation for its ability to support scaling out of multi-tenant SaaS products by quickly replicating or updating stacks. ISVs also value CloudFormation as a way to package and deploy their software in their customer accounts on AWS.
- 15. Amazon EC2 Spot Instances
- 16. On-Demand Pay for compute capacity by the hour with no long-term commitments For spiky workloads, or to define needs Amazon EC2 Consumption Models Reserved Make a low, one-time payment and receive a significant discount on the hourly charge For committed utilization Spot Bid for unused capacity, charged at a Spot Price that fluctuates based on supply and demand For time-insensitive or transient workloads
- 17. With Spot, the Rules Are Simple Markets where the price of compute changes based on supply and demand. You’ll never pay more than your bid. When the market exceeds your bid, you get 2 minutes to wrap up your work.
- 18. $0.27 $0.29$0.50 1b 1c1a 8XL $0.30 $0.16$0.214XL $0.07 $0.08$0.082XL $0.05 $0.04$0.04XL $0.01 $0.04$0.01L C4 $1.76 On Demand $0.88 $0.44 $.22 $0.11 Show Me the Markets! Each instance family Each instance size Each Availability Zone In every region Is a separate Spot Market
- 19. Amazon EC2 Spot Fleet
- 20. Spot Fleet Helps You… Launch thousands of Spot Instances with one single API call Get the best price Find the lowest-priced horsepower that works for you Or Get diversified resources Diversify your fleet and grow your availability Apply custom weighting Create your own capacity unit based on your application needs
- 21. Diversification with Amazon EC2 Spot Fleet Multiple Spot Instances selected Multiple Availability Zones selected Pick the instances with similar performance characteristics, e.g., c3.large, m3.large, m4.large, r3.large, c4.large
- 23. Overall Architecture Amazon S3 AWS CLI Public subnet – AZ #1 Public subnet – AZ #2 Amazon ECS Spot Instance Spot Instance Spot Fleet >_ SSH Amazon ECR AWS Management Console RunTask
- 24. Amazon S3 Public subnet – AZ #1 Public subnet – AZ #2 Amazon ECS Spot Instance Spot Instance Spot Fleet Amazon ECR AWS CloudFormation Lab 1: Set up the Workshop Environment
- 25. Amazon ECR Subnet ECS Task Definition Lab 2: Build an MXNet Docker Image Amazon ECS Spot Instance
- 26. Lab 3: Deploy MXNet Container With ECS Amazon S3 Public subnet – AZ #1 Public subnet – AZ #2 Amazon ECS Spot Instance Spot Instance Spot Fleet Amazon ECR AWS Management Console
- 27. Lab 4: Image Classification Demo
- 28. Lab 5: Wrap Image Classification in an ECS Task Amazon S3 Public subnet – AZ #1 Public subnet – AZ #2 Amazon ECS Spot Instance Spot Instance Spot Fleet Amazon ECR AWS Management Console RunTask
- 30. Appendix
- 31. Related Sessions • Deep Dive into Apache MXNet on AWS (BDA401) • An Introduction to Amazon Rekognition, for Deep Learning-based Computer Vision (BDA301) • Getting the Most Bang for Your Buck with #EC2 #Winning (SRV301) • Save 90% on Your Containerized Workload (DEM07)
- 32. Thank You!