AWS for Backup and Recovery

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
Backup and Restore with AWS
Isaiah Weiner,
Sr. Mgr. Solutions Architecture

Backup and recovery before the cloud
Application
servers
Local disk
Media
server
Tape storage
Data bunker

Primary Storage
Backup challenges in today’s age
• IDC estimates the volume of digital data
will grow 40% to 50% per year. By
2020, IDC predicts the number will have
reached 40,000 EB, or 40 Zettabytes
(ZB).
• The world’s information is doubling
every two years.

Understanding AFR and MTBF
1-((1-0.002)^1) x 100 = 0.2%
1-((1-0.002)^2) x 100 = 0.34%
1-((1-0.002)^3) x 100 = 0.60%
...
1-((1-0.002)^10) x 100 = 1.98%
...
1-((1-0.002)^1000) x 100 = 86.4935477553%
200,000 hours = 22.81 years
200,000 / 2 = 11.4 years
200,000 / 3 = 7.6 years
...
200,000 / 10 = 2.28 years
...
200,000 / 1000 = 8.3 days

Primary Storage
• File distribution services
• Gateway for IP storage protocols
• Native replication
Amazon EFS

Backup and Recovery
• Backup is not archive
• Backup represents a point in time copy of the data
• Archived data is the only authoritative copy of the data.

Archive
• End of the line for DLM!

Backup vs. Archive Summary
Backup Archive
Number of copies for one piece of
data
Many 1
Growth of the repository over time Exponential Linear
Contains “the” copy of data? No Yes
Point in time copy of data? Yes Yes
Select individual pieces of data based
on policy
Not really Yes
Backups held for “long” periods of
time ü û

Backup to the Cloud & Recovery

What should I use and when?
Economics Easy to use Reduce risk Agility, scale
§ Pay as you go
§ No upfront investment
§ No commitment
§ No risky capacity
planning
§ Self service administration
§ SDKs for simple
integration
§ Durable and secure
§ Avoid risks of physical
media handling
§ Reduce time to market
§ Focus on your business,
not your infrastructure
Amazon S3
Durable object storage
for all types of data
Amazon Glacier
Archival storage
for infrequently
accessed data
Hybrid Storage service
AWS Storage
Gateway Petabyte-scale data
transport solution
AWS Snowball

Backup and recovery to the cloud
Amazon S3
Amazon
Glacier
AWS
Direct
Connect
Internet
Amazon S3-IA
Application
servers
Cloud Gateway
Local disk
Media
server
Cloud Gateway
Application
servers
Cloud Connector
Local disk
Media server
with cloud
connector

Cloud Connector
1. Direct Amazon S3 / Glacier API/SDK
2. Amazon S3 lifecycle integration

Cloud Gateways for Backup
Amazon EBS
snapshots
Amazon S3
Amazon
Glacier
Application
server
AWS
Direct
Connect
Internet
Customer premises
Gateway
appliances
AWS
Storage Gateway
back-end
AltaVault
EC2
AMI

Hybrid Storage Service
AWS Storage Gateway

Backing up to AWS via Storage Gateway
3 options to write on-premises backups to AWS
Customer Premises or EC2
File Gateway GlacierS3-IA
Backup
Server
Customer Bucket
Volume
Gateway
iSCSI
Tape
Gateway
Volume Gateway
S3
Glacier
Tape Gateway VTL
Customer Environment
EBS Snapshots
S3
S3
Standard

Customer Premises
File
Gateway
Data (including backups) stored and retrieved from your S3 buckets
1-1 mapping from files-to-objects
File metadata stored in object metadata
Bucket access managed by IAM role you own and manage
Use S3 Lifecycle Policies, versioning, or CRR to manage data
GlacierS3 Standard S3 Standard
- Infrequent
Access
HTTPS
NFS
v3 / v4.1
Application
Server
File Gateway

Application
Server
Volume Gateway
On-premises volume storage backed by Amazon S3 with EBS snapshots
• Block storage in S3 accessed via the volume gateway
• Compression of data in-transit and at-rest
• Backup on-premises volumes to EBS snapshots
• Create on-premises volumes from EBS snapshots
• Up to 1PB of total volume storage per gateway
• Can be used by backup apps, e.g. Veeam, to write to AWS and recover in EC2
Amazon EBS
snapshots
Storage Gateway
bucket in
Amazon S3
Customer Premises
Volume
Gateway
iSCSI HTTPS

iSCSI
AWS Storage
Gateway VM
Upload
buffer
Cache
storage
TARGET
Client
Users
INITIATOR
Application
servers
INITIATOR
iSCSI
Amazon EBS
snapshots
AWS
Storage
Gateway service
Customer Data Center
HTTPS
Volume
storage backed
by Amazon S3
GATEWAY-CACHED
Volume Gateway

iSCSI
AWS Storage
Gateway VM
Upload
buffer
volume Volume
storage
TARGET
Client
Users
INITIATOR
Application
servers
INITIATOR
iSCSI
Amazon EBS
snapshots
AWS Storage
Gateway service
Customer data center
GATEWAY-STORED
Volume Gateway

Tape Gateway
Virtual tape storage in Amazon S3 and Glacier with VTL management
• Compressed of data in-transit and at-rest
• Up to 1 PB total tape storage per gateway, unlimited archive
capacity
• 3-5 hour retrieval of virtual tapes from Glacier
Archived Tapes
stored in
Amazon Glacier
MEDIA
CHANGER
TAPE
DRIVE
Customer Premises
Tape
Gateway
Virtual Tapes
stored in
Amazon S3
Backup
Server
HTTPSiSCSI

Backup, archive, and disaster recovery
Cost effective storage in AWS with local or cloud restore
“Tapes are a headache. AWS Storage Gateway
provided the most cost-effective and simple alternative.
We switched from physical to virtual tape backup simply by dropping the
gateway’s virtual appliance into our existing Veeam workflow.
Setting it all up took three hours, at most.
We even got disaster recovery by using a bi-coastal data center.”
-Jesse Martinich, Network Services Manager, SOU

Petabyte-scale data transport solution
AWS Snowball

What is AWS Snowball?
Petabyte-scale data transport
E-ink shipping
label
Ruggedized case
“8.5G impact”
All data encrypted
end-to-end
Rain- and dust-
resistant
Tamper-resistant
case and
electronics
80 TB
10 GE network

How it works

How fast is Snowball?
§ Less than 1 day to transfer 250 TB through 5 x 10G connections with 5
Snowballs, less than 1 week, including shipping
§ Number of days to transfer 250 TB through the Internet at typical utilizations
Internet connection speed
Utilization 1 Gbps 500 Mbps 300 Mbps 150 Mbps
25% 95 190 316 632
50% 47 95 158 316
75% 32 63 105 211

Archive after
30 days
My S3 bucket Amazon Glacier
rawdata1
rawdata2
rawdata3 Delete after
7 years
Customer Use Case: Backup and Archive with
Snowball

Amazon
Glacier
Amazon S3
Standard
Amazon S3
Infrequent Access
File Gateway
continuous file
access & upload,
with local cache
PetroBank
application
servers
AWS Snowball
initial bulk transfer
AWS Direct
Connect
Lifecycle policies migrate data
across storage tiers
AWS Lambda
automated functions,
including inventory
PetroBank Archive Service Migrated from Tape to Cloud
Cost effective storage in AWS with local data access
Self service loading of data
Reduced time-to-data by days or weeks
Cut storage archive costs by 90%

Backup in the cloud

What should I use and when?
Easy to useReduce risk Agility, scale
§ Self service administration
§ SDKs for simple
integration
§ Durable and secure
§ Avoid risks of physical
media handling
§ Reduce time to market
§ Focus on your business,
not your infrastructure
Amazon EBS
Block storage for use
with Amazon EC2
Amazon Glacier
Archival storage
for infrequently
accessed data
Amazon EFS
File storage for use
with Amazon EC2
Economics
§ Pay as you go
§ No upfront investment
§ No commitment
§ No risky capacity
planning
Amazon S3
Durable object storage
for all types of data

Amazon EBS
Block storage for use with Amazon EC2

Internet
AWS Cloud
Amazon EBS Lifecycle
EC2 Availability Zone
EC2
Amazon S3
EBS
EC2 EC2
EBS EBS EBS EBS EBS
EBS Snapshot
EBS Snapshot
EBS Snapshot
EBS Snapshot
EBS Snapshot
Create Snapshot
Clone From
Snapshot

EBS Volume
How Do Snapshots Work?
Time
Snapshot 1 Snapshot 2 Snapshot 3
S3
Block 1
Block 2
Block 3
Block 4
Chunk 1
Chunk 2
Chunk 3
Chunk 4

Benefits of using EBS snapshots
More durable than an EBS volume
• Stored in Amazon S3
Incremental (space-efficient)
• First snapshot is a clone
• Pay only for what you use
Availability Zone-independent
• Clone into any AZ
Can be copied efficiently across regions

AWS Database Backups
• RDS for MySQL, PostgreSQL, MariaDB, Oracle, SQL Server
• Scheduled daily backup of entire instance
• Archive database change logs
• 35 day retention for backups
• Multiple copies in each AZ where you have instances for a deployment
• Aurora
• Automatic, continuous, incremental backups
• Point-in-time restore
• No impact on database performance
• 35 day retention
• DIY on EC2
• Engine specific (RMAN, BAK)
• Third party (GoldenGate, Commvault)

Amazon S3
Durable object storage for all types of data

Use Amazon Glacier
for lowest-cost, durable cold
storage of archival data
Use Amazon S3
for reliable, durable
primary storage
Use Amazon S3
Infrequent Access
Storage
for secondary backups
at a lower cost
S3-IA
Amazon S3 Lifecycle

Key prefix “logs/”
Transition objects to Amazon Glacier 30 days after
creation
Delete 365 days after creation date
<LifecycleConfiguration>
<Rule>
<ID>archive-in-30-days</ID>
<Prefix>logs/</Prefix>
<Status>Enabled</Status>
<Transition>
<Days>30</Days>
<StorageClass>GLACIER</StorageClass>
</Transition>
<Expiration>
<Days>365</Days>
</Expiration>
</Rule>
</LifecycleConfiguration
S3 lifecycle policies

• Usual charges for storage,
requests, and inter-region
data transfer for the
replicated copy of data
• Replicate into Standard-IA or
Amazon Glacier
Cost
HEAD operation on a source
object to determine replication
status
• Replicated objects will not be re-
replicated
• Use Amazon S3 COPY to
replicate existing objects
Replication status
DELETE without object
version ID
• Marker replicated
DELETE specific object
version ID
• Marker NOT replicated
Delete operation
Cross-region replication: Details
Object ACL updates are
replicated
• Objects with Amazon
managed encryption key
replicated
• KMS encryption not
replicated
Access control

Backup & Recovery as a Service
All-in on AWS

Amazon Glacier
Archival storage for infrequently accessed data

Amazon Glacier Lifecycle
Create vault1
Configure access policies2
User policy
Effect:Allow
Resource:
arn:aws:glacier:<accountId>:vaults
Action: glacier:UploadArchive
3 Upload archives
UploadArchive(data) ->
Archive ID

Vault access policy
• Can be updated/deleted
Vault lock policy
• Lockable/Immutable policy
• Cannot be updated/deleted
after lockdown
Use vault access policy to:
• Designate third-party access
• Grant temporary read
permissions when necessary
Use vault lock policy to:
• Deploy regulatory controls such
as records retention
• Enforce data access through
multi-factor authentication only
Compliance/Governance Flexibility
Using vault lock policy with vault access policy

Vault lock best practices
• Map one vault to a single retention range
– Group regulatory data by retention: 1-year vault, 6-year vault, etc.
• Create new vault and lock it before storing production data
– Enforce the full ArchiveAgeInDays on all new archives
– Leave no “gap” on existing archives
• Thoroughly test a vault lock policy before locking it down (Abort/Initiate)
• Implement only the most restrictive controls with vault lock
– Leave the flexible controls to vault access policy

Amazon Glacier received a third-party assessment from
Cohasset Associates on how Amazon Glacier with Vault
Lock can be used to meet the requirements of SEC 17a-
4(f) and CFTC 1.31(b)-(c).

SoundCloud—leveraging Glacier for audio transcoding
• World’s leading social sound platform
• Audio files must be transcoded and
stored in multiple formats
• Stores petabytes of data
• Transcoded files served from S3
• Originals moved to Glacier for cost
savings

Amazon EFS
File storage for use with Amazon EC2

Amazon EFS Backup
• Automated EFS backups based on a
schedule that you define (for example,
hourly, daily, weekly, or monthly)
• Automated rotation of the backups, where
the oldest backup is replaced with the
newest backup based on the number of
backups that you want to retain

Amazon EFS Restore
• Restore a backup copy of an Amazon EFS
file system
• Restores can be done in parallel to meet
the recovery time objective
• Restore individual files from EFS Backups

Why EFS for Database Backup
ü Can be used with native backup commands
• - i.e. dump, RMAN, “hot-backup” mode
ü Copy is stored to another storage target for availability
• - production copy runs on EBS
• - backup copy is on EFS
ü Can be managed by the database administrators
• - to meet their specific recovery points
• - easy to restore online
ü High performance network shares provide for fast recovery vs. tape
ü Saves licensing costs and workload from traditional backup software

The Arcesium platform leverages Amazon EFS for shared data storage
between applications and for storing and analyzing operational data.
“Arcesium is a financial services SaaS platform that requires resilient, secure,
and scalable file storage. Amazon EFS offers us a powerful way to operate and
scale file storage for our Amazon EC2 instances, which has allowed us to build
out our platform quickly without compromising quality.”
-- Gaurav Suri, CEO
“We are growing by leaps and bounds, and our core offering is all about better support delivery.
During the course of developing our next-generation internal support system, we never wanted
to worry about scale again, yet we had existing architectural commitments that meant a
distributed file solution was required. Atlassian chose Amazon EFS because it was the only option
available that scaled both capacity and performance – without the up-front payments or the
management overhead of traditional models. This allows our support teams to focus on what
matters most - helping our customers.”
- Sri Viswanath, CTO

Public Sector – King County
• Most populous county in Washington State
• Replace tape solution for backup from 17 agencies
• Meet compliance requirement
• Saved $1MM in first year, no more tape refresh or management
churn
https://aws.amazon.com/solutions/case-studies/king-county/

• 2nd largest Online Video Service – 100MM+ monthly viewers
• Self managed Swift cluster out of capacity
• 5PB media assets/stats, secondary back up on Glacier
China Expansion – iQIYI
https://aws.amazon.com/cn/solutions/case-studies/iqiyi/

AWS External Resources
• AWS Storage Solution Pages
– Backup, Archive and Disaster Recovery
• AWS Storage Competency and Storage Test Drives
– AWS Storage Competency
– APN Partner-provided labs
• AWS Marketplace Storage for in-cloud use cases
– AWS Online Software Store
• Select Partner Microsites – additional in plan
– www.netapp.com/aws
– www.commvault.com/aws

Pop-up Loft
aws.amazon.com/activate
Everything and Anything Startups
Need to Get Started on AWS

AWS for Backup and Recovery

More Related Content

AWS for Backup and Recovery