Overview Amazon Web Services
Overview Amazon Web Services
Overview Amazon Web Services
Mikhail Vladimirov
Director, Curriculum Architecture
mikhail.vladimirov@webagesolutions.com
Web Age Solutions
Provider of a broad spectrum of regular and customized
training classes in programming, system administration and
architecture to clients across the world since 1999
©WebAgeSolutions.com 2
Overview of Talk
©WebAgeSolutions.com 3
Overview of Amazon Web Services
©WebAgeSolutions.com 5
The History of AWS
In 2003, Chris Pinkham and Benjamin Black of Amazon
presented a short internal paper "describing a vision for Amazon
infrastructure that was completely standardized, completely
automated, and relied extensively on web services for things
like storage" [http://blog.b3k.us/2009/01/25/ec2-origins.html]
The first public AWS service was Simple Queue Service (SQS)
launched in November 2004
S3 was launched in March 2006
EC2 followed with a launch in August 2006
As a cloud platform providing on-line services for clients, AWS
was launched in 2006
In November 2010, all of Amazon.com retail web services were
moved to AWS
©WebAgeSolutions.com 6
amazon.com AWS Infrastructure circa 2008
©WebAgeSolutions.com 7
Overview of Amazon Web Services
©WebAgeSolutions.com 9
Overview of Amazon Web Services
©WebAgeSolutions.com 11
The AWS Distributed Architecture
The AWS cloud platform is deployed across a number
of geographical regions (11 as of beginning 2015)
Each region includes several "Availability Zones" (AZ),
which are isolated from each other data centers
Some Amazon services can operate across AZ's (e.g. S3)
Other services require you to set up and configure
replication across AZ's to achieve service resilience against
data center outages
As of December 2014, Amazon Web Services
operated 1.4 Million servers across 28 availability
zones.[ wikipedia.org ]
©WebAgeSolutions.com 12
The Amazon Web Services Overview
Elastic Compute Cloud (EC2)
The compute service of AWS
Identity and Access Management (IAM)
Helps managing users and their permissions within your
AWS Account
IAM is natively integrated into all the AWS Services
Virtual Private Cloud (VPC)
Lets you define a private virtual network inside the AWS
Cloud with your own IP address range, subnets, route
tables, and network gateways
Enables you to build hybrid clouds by extending your
corporate network topology into a private cloud inside the
Amazon Cloud
AWS VPC uses IPSec tunnel mode for point-to-point secure
channel between your data center and the AWS Cloud
©WebAgeSolutions.com 13
The Amazon Web Services Overview
CloudWatch
Helps with monitoring Amazon EC2 instances with respect of
operational performance, including such metrics as CPU
utilization, I/O operations, network traffic (in/out), response
latencies, HTTP status codes, etc.
Auto-scaling Group
Allows automatic scaling of your capacity by configuring
resource utilization thresholds and usage conditions
Metrics are collected by the CloudWatch service
Elastic Load Balancing
Distributes incoming traffic across a cluster of EC2 instances
Amazon Elastic Block Storage (EBS)
A network-attached persistent storage volume(s) that you
can attach to EC2 instances
©WebAgeSolutions.com 14
The Amazon Web Services Overview
Simple Storage Service (S3)
Highly durable distributed data store
S3 supports the RESTful end-point as a programmatic
interface for uploading and downloading objects
CloudFront
A content delivery service for static and/or streaming
content
S3 objects (images, documents, etc.) can be replicated and
cached using CloudFront in a number of different
geographical locations (called edges)
SimpleDB
A simple NoSQL engine that operates on collections of items
stored as key-value pairs
©WebAgeSolutions.com 15
The Amazon Web Services Overview
Relational Database Service (RDS)
Backed-up by full-featured instances of a database engine of
your choice (databases currently supported are: MySQL,
Postgres, Oracle, SQL Server, and Amazon Aurora)
Database administration tasks such as backups, patch
management, etc., are taken care of by AWS support
Simple Queue Service (SQS)
A cloud-grade distributed queuing system
Supports JMS 1.1 API with minimal changes
Simple Notifications Service (SNS)
SNS allows you build applications based on the pub-sub
protocol (topics-based)
©WebAgeSolutions.com 16
The Amazon Web Services Overview
Elastic MapReduce (EMR)
A hosted Hadoop framework for running MapReduce jobs
You have a choice of Hadoop-related applications that can
be installed alongside with your Hadoop cluster:
• Hive, Pig, Hue, HBase, Impala, etc.
In addition to the Amazon's Hadoop distribution based on
the open-source (and free) Hadoop packages, you have an
option to use MapR which is a more efficient, third-party
distribution from https://www.mapr.com/
• See http://aws.amazon.com/elasticmapreduce/mapr/ for
details
EMR is tightly integrated with EC2 and S3 services
©WebAgeSolutions.com 17
The Amazon Web Services Overview
Route53
A highly scalable DNS service which you can use to manage
your DNS records for all your domains to help resolve
domain names to AWS-hosted IP addresses of your
applications
It also provides ways to buy new domains and transfer in
existing domains
Route 53 can also act as the registrar for your existing
domains that you bought from other registrars
This service can also be used to monitor the health and
performance of your web applications and redirect traffic
accordingly
©WebAgeSolutions.com 18
Overview of Amazon Web Services
©WebAgeSolutions.com 20
S3 Bucket Configuration
©WebAgeSolutions.com 21
S3 Object Lifecycle Configuration
You can configure object lifecycle events, such as archiving, or
deleting objects (e.g. log files) after a specified time period
Rules controlling lifecycle events are configured at the bucket
level and apply to the whole bucket
You can define the following actions on objects:
Archive Only
• And keep the object in S3
Permanently Delete Only
Archive and then Permanently Delete
• Essentially, move the object from S3 to the archive
Here is a visual representation of a rule for archiving a file in the
Glacier archival storage after 1 day of holding it in S3:
©WebAgeSolutions.com 22
Static Web Hosting in S3
You can host all the static content of your website on
Amazon S3
The static content includes HTML pages, JavaScript and CSS
files, images, etc.
You can refresh static content by uploading the updated
versions of those static content assets
You configure website hosting at the bucket level
Once you enable your bucket for static website
hosting, all your content in the bucket is publicly
accessible via the Amazon S3 HTTP endpoint at the
following sub-domain:
<your-bucket-name>.<your-region>.amazonaws.com
©WebAgeSolutions.com 23
Manual File Upload to S3
©WebAgeSolutions.com 24
Object Permissions
Objects in S3 are controlled by permissions and can
be exposed to the public view
©WebAgeSolutions.com 25
S3 Pricing
US Standard:
©WebAgeSolutions.com 26
Overview of Amazon Web Services
©WebAgeSolutions.com 28
AWS Marketplace for OSes
©WebAgeSolutions.com 29
AWS Marketplace for Tools & Apps
©WebAgeSolutions.com 30
EC2 Instance Choices
©WebAgeSolutions.com 31
EC2 Instance Types
AWS Cloud offers a wide selection of EC2 instance
types to help you match different use cases with
optimum computer capacity
Instance types include such parameters as:
CPU, RAM, instance storage, and networking capacity (all
resources are virtual!)
Amazon recommends to measure the actual
performance of your application by running it under
load to identify the suitable instance type and
validate your solution architecture
©WebAgeSolutions.com 32
EC2 Instance Types Matrix
EC2 instance types are grouped by instance families
that have similar base run-time profiles suitable for
specific use cases
Within each family, you have models with more vCPUs, RAM
and storage
Main families:
T2 (General Purpose Category)
M3 (General Purpose Category)
C3 (Compute Optimized Category)
R3 (Memory Optimized Category)
G2 (Graphics Optimized Category)
I2 (Storage Optimized Category)
HS1 (Storage Optimized Category)
Note: The original T1, M1, C1, CC2, M2, CR1, CG1, and HI1
instance types have been deprecated
©WebAgeSolutions.com 33
The T2 Type: Example of a Low-end Type
The T2 type is used for low compute capacity instances
This type is suitable for development, prototyping, build
servers, and small web applications
T2 instances are positioned as instances with "Burstable
Performance", meaning that they offer a baseline level
of CPU performance with the ability to burst above the
baseline for a short period of time
©WebAgeSolutions.com 34
The I2 Type: Example of a High-end Type
The I2 type is storage optimized with fast SSD-backed
instance storage optimized for random I/O performance
This type gives you an option of very cost-efficient IOPS
rates and enhanced networking with great packet per
second (PPS) rate and low latencies
Use cases: NoSQL systems (MongoDB, Cassandra,
Hadoop, etc.)
©WebAgeSolutions.com 35
VM Import / Export to/from EC2
©WebAgeSolutions.com 36
EC2 Quotas
©WebAgeSolutions.com 37
Spot Instances
The spot instance pricing fluctuates over a period of time
and availability zone
You can check the Pricing History in the EC2 dashboard to
see the trend to better prepare for pricing changes
You can select a time horizon (from 1 day up to 3
months), instance OS, and EC2 instance type per
availability zone
©WebAgeSolutions.com 38
EC2 SLA
In its SLA, AWS makes a Service Commitment to this effect:
"AWS will use commercially reasonable efforts to make Amazon
EC2 and Amazon EBS each available with a Monthly Uptime
Percentage of at least 99.95%, in each case during any
monthly billing cycle (the “Service Commitment”). In the event
Amazon EC2 or Amazon EBS does not meet the Service
Commitment, you will be eligible to receive a Service Credit as
described below."
"Service Credits are calculated as a percentage of the total
charges paid by you (excluding one-time payments such as
upfront payments made for Reserved Instances) for either
Amazon EC2 or Amazon EBS"
©WebAgeSolutions.com 39
Overview of Amazon Web Services
©WebAgeSolutions.com 41
EBS
EBS offers raw, unformatted block-level storage
After you have attached an EBS volume, you will need to
format it and create a file system on the device before
you can start using it
You may experience some I/O delays (5 to 50 %) when
starting to use a newly added EBS volume
You may also experience spikes in CPU usage during such
times
An EBS volume must be created in the same Availability
Zone as the EC2 instance you are going to attach the
volume to
Each EBS volume is automatically replicated within its
Availability Zone (AZ) offering high guarantees of data
durability and protection against loss of your data
©WebAgeSolutions.com 42
Types of EBS
EBS comes in three volume types that differ in
performance characteristics and cost
Provisioned IOPS Solid-State Drives (SSD):
The fastest option with 4,000 IOPS with single-digit millisecond
latencies
Consistent performance
Max throughput: 128MBps
General Purpose (SSD):
Have the ability to burst to 3,000 IOPS per volume
Max throughput: 128MBps
Magnetic Disk (used to be called the EBS Standard
Volume):
The lowest cost volume with the slowest I/O throughput (several
hundred IOPS)
Max throughput: 40-90MBps
©WebAgeSolutions.com 43
EBS Volume Encryption
EBS volumes can be encrypted
Applying EBS encryption may be required, for
example, in order to meet some regulatory security
compliance requirements for your data-at-rest
Encryption is done using AWS-managed keys or keys
you create and manage using the AWS Key
Management Service
Keys are protected by AWS key management infrastructure
Each newly created EBS volume is issued a unique
256-bit AES key
EBS volumes created from encrypted snapshots
inherit the key
Currently, only data volume encryption is supported
(boot volumes cannot be encrypted)
©WebAgeSolutions.com 44
Use Cases for EBS Volume Types
Provisioned IOPS (SSD)
I/O Intensive systems: relational and NoSQL Databases
General Purpose (SSD)
Boot volumes, development, testing, small to medium
relational and NoSQL databases
Magnetic Disk
Low demand data access
©WebAgeSolutions.com 45
EBS Volume Type Prices
©WebAgeSolutions.com 46
Creating an EBS Volume
1
2
©WebAgeSolutions.com 47
EBS Volume RAID Configuration
EBS volumes can be joined in any of the standard RAID
configurations the same way as you would if you use
physical hard drives
The selected RAID configuration must be supported by
the operating system installed on your EC2 instance as
RAID configuration is done at the software level
Amazon recommends
RAID 0 mode (disk striping, without mirroring or
parity) when I/O performance is more important than
fault tolerance
RAID 1 (disk mirroring, without parity or striping)
mode when fault tolerance is more important than I/O
performance
RAID 5 and RAID 6 are not recommended for Amazon
EBS because these modes take away some of the IOPS
available to your volumes for parity writing operations
©WebAgeSolutions.com 48
EBS Volume Snapshots
EBS provides the ability to save the replica of the
volume as a point-in-time snapshot
Only changes (delta) are saved in subsequent EBS
snapshots
You only pay for the incremental delta in snapshots
To fully restore the volume, you need all the active
snapshots
You can access the Public Snapshots repository from
the list in the Snapshots section of the AWS
Management Console
Snapshots are stored in S3
When you start using an EBS volume created from an EBS
Snapshot, first hit on a required data block may be delayed
(the lazy loading model is implemented)
©WebAgeSolutions.com 49
Use Cases for EBS Volume Snapshots
©WebAgeSolutions.com 50
Overview of Amazon Web Services
Our Offerings
Cloud computing and solutions offerings
©WebAgeSolutions.com 52
Summary
©WebAgeSolutions.com 53