Managing Your Infrastructure as Code

Editor's Notes

1. Choosing the right EC2 Instance type matters. Selecting an appropriate instance type for your workload can save time and money. AWS has a wide variety of EC2 compute instance types to choose from. Each Instance type or family (like T2, M3, C4,C3, G2, R3, and so on) is optimized for different workloads or use cases. Within an EC2 family, you can choose from different sizes for example micro, small, medium, large, xlarge, 2xlarge, and so on. AWS utilizes Intel® Xeon® processors for the EC2 Instances to provide customers with high performance and value for their computing needs. When you choose your instance type you should consider the several different attributes of each family; such as number of cores, amount of memory, amount & type of storage, network performance, and processor technologies. Another important consideration is TCO. A lowest-price per hour instance is not necessarily a money saver; a larger compute instance can sometimes save both money and time. It is important to evaluate all the options to see what is best for your workload.
2. AWS recently launched C4 compute-optimized instances which utilize Intel’s latest 22nm Haswell microarchitecture. C4 instances use custom Intel® Xeon® v3 processors designed and built especially for AWS. Through its relationship with Intel®, AWS provides its customers with the latest and greatest Intel® Xeon® processors that help in delivering the highest level of processor performance in EC2.
3. Intel® Xeon® processors have several other important technology features that can be leveraged by EC2 Instances. Intel® AVX is perfect for highly parallel HPC workloads such as life sciences or financial analysis. Intel® AES-NI accelerates encryption/decryption of data and therefore reduces the performance penalty that usually comes with encryption. Intel® Turbo Boost Technology automatically gives you more computing power when your workloads are not fully utilizing all CPU cores. Think of it as automatic overclocking when you have thermal headroom.
4. The matrix on the slide highlights the individual Intel® technologies that were discussed previously and the EC2 instance family that can leverage each of these technologies. A complete list of Amazon EC2 Instance types can be found here: http://aws.amazon.com/ec2/instance-types/
5. The development process that you use for developing business logic can be the same as what you when writing CloudFormation templates. You start of with your favorite IDE or Text Editor to write the code, Eclipse, VIM or VisualStudio You then commit to template to your source code repository using your usual branching strategy and then have the template reviewed as part of your typical code review process. The template is then integrated and run as part of your CI and CD pipelines. Being simply a JSON document, you can even write Unit Tests for your templates. When developing a CloudFormation template you can use all of your normal software engineering principles At the end of the day It’s all software – a template can be reused across applications – just like code library's and a stack can be shared by multiple applications.
6. The development process that you use for developing business logic can be the same as what you when writing CloudFormation templates. You start of with your favorite IDE or Text Editor to write the code, Eclipse, VIM or VisualStudio You then commit to template to your source code repository using your usual branching strategy and then have the template reviewed as part of your typical code review process. The template is then integrated and run as part of your CI and CD pipelines. Being simply a JSON document, you can even write Unit Tests for your templates. When developing a CloudFormation template you can use all of your normal software engineering principles At the end of the day It’s all software – a template can be reused across applications – just like code library's and a stack can be shared by multiple applications.
7. The development process that you use for developing business logic can be the same as what you when writing CloudFormation templates. You start of with your favorite IDE or Text Editor to write the code, Eclipse, VIM or VisualStudio You then commit to template to your source code repository using your usual branching strategy and then have the template reviewed as part of your typical code review process. The template is then integrated and run as part of your CI and CD pipelines. Being simply a JSON document, you can even write Unit Tests for your templates. When developing a CloudFormation template you can use all of your normal software engineering principles At the end of the day It’s all software – a template can be reused across applications – just like code library's and a stack can be shared by multiple applications.
8. So if you look at the components behind Cloudformation. It's starts off with a template. This is the JSON formatted script file, that deals with things like parameter definition that drive a user driven template, such as name of my databases. It deals with the resource creation, so the creation of AWS components such as EC2 instances or RDS databases. And it deals with the configuration actions I wish to apply against this resources, so it might be install software or might be creating an SQS queue for example. Than that template is deployed into the cloud formation framework. And the framework deals what we call Stack creation, updates and any error detection and rollback required in the creation of a stack. So a stack is collection of resources that you want to manage together. And the resulting artifact is what we call a Stack of configured AWS services. So this could be in an Elastic Load Balancer and Autosclaing group with EC2 instances and an RDS database. So the stack is service event aware, the stack creation actions or the changing of that environment can be feed back into Cloudfomration and trigger actions within the CloudFormation tempalte. And it is also customizable, so once you created a stack you can of course access the underlying resources and change them of modify them as you so which. Now the error detection and rollback is an interesting point. If at any time in the stack creation a problem is detected, the default behaviour of Cloudformation is to roll-back the creation of all resources and put you back in a constitent known state. So you know if your stack is working or is rolled back and is not.
9. AWS CloudFormation provides the following helpers to allow you to deploy your application code or application and OS configuration at the time you launch your EC2 instances: cfn-init: Used to retrieve and interpret the resource metadata, installing packages, creating files and starting services. cfn-signal: A simple wrapper to signal a CloudFormation WaitCondition allowing you to synchronize other resources in the stack with the application being ready. cfn-get-metadata: A wrapper script making it easy to retrieve either all metadata defined for a resource or path to a specific key or subtree of the resource metadata. cfn-hup: A daemon to check for updates to metadata and execute custom hooks when the changes are detected.
10. When we discuss infrastructure we are using it as a very broad term, it doesn’t just mean network compute and storage. As of today CloudFormation supports over 30 AWS services. A key benefit is the range of resources supported. We play catch up so that you don’t have to and are constantly expanding. Just in the last month we have extended AWS Lambda support to allow you to define event sources from kinesis or dynamoDB steams as well as setting up the right permissions so the lambda function can execute with the appropriate permissions. Now you can connect lambda functions to your data streams to process event-based data. Added the ability to create EC2 spot fleets allowing you to create and maintain a set of EC2 spot instances based on a launch specification and bid price Now support Aurora – allowing you to create an Arora database cluster along with the parameters you want to specify to configure the cluster and database SimpleAD CodeDeploy – now you can not only create your infrastructure, but also setup the deployment mechanism for your application – we allow you to define the applications and the deployment groups to use CodeDeploy to manage your application lifecycle on top of the infrastructure you create through CloudFormation Amazon workspaces – the ablity to provision Amazon workspaces in an automated and repeatable way – may be you create a new workspace for each AWS account you create. Now you can use CloudFormation to configure such things as CloudTrail, IAM users and groups, VPCs and now Workspaces for that new account On top of this. The recently release Lambda backed Custom Resources you can now use Lambda functions to support things that CloudFormation does not support – we’ll talk more about Lambda custom resources later in the session.
11. Notes: The entire application can be represented in an AWS CloudFormation template. You can use the version control system of your choice to store and track changes to this template. You can use the template to quickly build out multiple environments, such as for Development, Test, and Production.
12. Notes: The entire application can be represented in an AWS CloudFormation template. You can use the version control system of your choice to store and track changes to this template. You can use the template to quickly build out multiple environments, such as for Development, Test, and Production.
13. Notes: The entire application can be represented in an AWS CloudFormation template. You can use the version control system of your choice to store and track changes to this template. You can use the template to quickly build out multiple environments, such as for Development, Test, and Production.
14. We’re going to model our application, starting with a stack that contains all the resources we’re going to use. Next, we’ll create a layer – it’s a blueprint for how we configure ec2 instances. We’ll create a PHP layer for our PHP app. We’ll create some instances and then deploy our application. Now our app, like many, needs to connect to a database for persistence. How can we create and configure the database, and then make sure our app connects to the database? The old way is we create the database table by hand, and then bake the connection info into the code. This is error prone because you can miss a step or later forget what you did when you need to recreate it. We really want configuration information such as the database table creation, user creation and all the metadata about the database to be managed like our source code. That way we can roll back changes. Fortunately, Chef gives us a way to do this.
15. Setup is sent to the instance when it is instantiated or successfully booted. For example, the OpsWorks Rails application server layer uses the setup event to trigger a Chef recipe that installs dependencies like Apache, Ruby, Passenger, and Ruby on Rails. Configure notifies all instances whenever the state of the stack changes. For example, when a new instance is successfully added to an application server layer, the configure event triggers a Chef recipe that updates the OpsWorks Load Balancer layer configuration to reflect the added application server instance. Deploy is triggered whenever an application is deployed. For example, the OpsWorks Rails application server layer uses the deploy event to trigger a Chef recipe that executes the tasks needed to check out and download your application and tells Passenger to reload it. Undeploy is sent when you delete an application. For example, the undeploy event can trigger a custom Chef recipe that specifies any cleanup steps that need to be run, such as deleting database tables. Shutdown is sent to an instance 45 seconds before actually stopping the instance. For example, the shutdown event can trigger a custom Chef recipe that shuts down services.
16. To use AWS CodeDeploy, you start by creating an application. An application consists of different components. A revision is a specific version of deployable content such as source code, post-build artifacts, web pages, executable files. A revision can be stored in GitHub or an Amazon S3 bucket. A deployment group is a set of Amazon EC2 instances associated with an application that you target for a deployment. You can add EC2 instances to a deployment group by specifying an EC2 tag and/or an Auto Scaling group name. A deployment configuration is a constraint that determines how a deployment progresses through the EC2 instances in a deployment group. In order to do a deployment, you simply specify these three parameters which answers the questions - what to deploy, where to deploy and how to deploy. You may be thinking how do you specify what files to copy and what deployment scripts to execute. To do this, you include a CodeDeploy specific configuration file in your revision.
17. Next we prepare our deployment targets * We need to install the CodeDeploy agent on each server (Use a user-data script to bootstrap the installation or build the CodeDeploy agent into the base Amazon Machine Image.) * Then we group the target server fleets for your application by using EC2 tags, on-premises tags, or Auto Scaling Group. Integrating CodeDeploy with an Auto Scaling group is a common pattern.) * You can choose to divide your target groups however you'd like * For example, you can split between your development, testing, staging, and production environments
18. Finally, it is time to deploy. You can use the CLI, an SDK, the Console, or one of our partner integrations like Jenkins, Travis CI, and GitHub to kick off a deployment. Since it's API based, we have a number of integrated partners You can monitor the deployment through the web console, and if troubles arise, to roll back an application to a previous revision, you just need to deploy that revision. AWS CodeDeploy keeps track of the files that were copied for the current revision and removes them before starting a new deployment, so there is no difference between redeploy and roll back.
19. It’s important to note the flexibility you have when doing your deployments. When you create a deployment configuration, you control the setting for minimum-healthy-hosts. This will let you configure the speed of a deployment so you can balance availability and deployment duration For example, You could take a more conservative approach, updating one server at a time and monitoring for issues, prepared to redeploy the previous known good version if necessary. Or if it is important your entire application server fleet gets the latest revision ASAP, you can set min-healthy-hosts to 0. Hopefully you have seen the flexibility of CodeDeploy to manage applications deployments on EC2 or instances you manage on-premises.