Module 7: Data Management Backup, DR, Test/Dev Environments
Module 7: Data Management Backup, DR, Test/Dev Environments
Test/Dev environments
Introduction
EBS Snapshots (CLI)
AMIs (CLI/Console)
RDS Automated backups (Console)
RDS read replicas (Console)
RDS manual snapshots (CLI/Console)
S3 backups (CLI/Console)
Amazon Glacier (CLI)
Review/Assignments
Introduction
Amazon Glacier
https://aws.amazon.com/glacier/
EBS snapshots
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSSnapshots.html
RDS backing up and restoring
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_CommonTasks.BackupRestore.html
RDS read replicas
https://aws.amazon.com/rds/details/readreplicas/
S3 object life cycle management
http://docs.aws.amazon.com/AmazonS3/latest/dev/objectlifecyclemgmt.html
Cross region copy/replication
https://aws.amazon.com/blogs/aws/newcrossregionreplicationforamazons3/
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebscopysnapshot.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/CopyingAMIs.html
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CopySnapshot.html
Sharing AMIs/Snapshots
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sharingamisexplicit.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebsmodifyingsnapshotpermissions.html
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ShareSnapshot.html
CLI References
http://docs.aws.amazon.com/cli/latest/reference/glacier/index.html
EBS Snapshots (CLI)
Create snapshots
aws ec2 createsnapshot volumeid vol1509ade2 description "appsnapshot11122015" dryrun
aws ec2 createsnapshot volumeid vol1509ade2 description "appsnapshot11122015"
aws ec2 describesnapshots snapshotid snap5178250e
Create snapshots (after stopping instances)
aws ec2 describeinstances instanceids i31745bf5
aws ec2 stopinstances instanceids i31745bf5
aws ec2 startinstances instanceids i31745bf5
aws ec2 describesnapshots ownerids 560125328540
aws ec2 describeinstances instanceids i31745bf5
Copy snapshots (crossregion)
aws region useast1 ec2 copysnapshot sourceregion uswest2 sourcesnapshot snap5178250e
description "appsnapshotdr11122015"
aws region useast1 ec2 describesnapshots snapshotids snap40cc6bd6
aws region useast1 ec2 describesnapshots ownerids 560125328540
Restore: Create volume from snapshot + attach to an EC2 instance
aws ec2 createvolume snapshotid snap5178250e availabilityzone uswest2a
aws ec2 describevolumestatus volumeid vol8e06384f
aws ec2 describevolumes volumeid vol8e06384f
aws ec2 attachvolume volumeid vol8e06384f instanceid i31745bf5 device /dev/sdf
Cleanup
aws region useast1 ec2 describesnapshots ownerids 560125328540
aws region useast1 ec2 deletesnapshot snapshotid snap40cc6bd6
aws region uswest2 ec2 describesnapshots ownerids 560125328540
aws ec2 deletesnapshot snapshotid snap5178250e
aws ec2 describevolumes
aws ec2 deletevolume volumeid vol8e06384f
aws ec2 describeinstances
aws ec2 terminateinstances instanceid i31745bf5
Web console
Share Snapshot with test/devel account
AMIs (CLI/Console)
Create AMI
aws ec2 createimage instanceid i50012e94 name appami reboot
Cross AMI (cross region)
aws ec2 copyimage sourceimageid ami4d233f2c sourceregion uswest2 region useast1 name
"appamidr"
View/Verify
aws ec2 describeimages owners 560125328540
aws ec2 region useast1 describeimages owners 560125328540
Restore/launch
aws ec2 runinstances imageid ami4d233f2c count 1 instancetype t2.micro keyname edureka1
securitygroups default
Web console
Share AMI with test/devel account
RDS Automated backups (Console)
Key points:
● Restore to point in time
● Max 35 days retention
● Deleted if retention changed to 0
● Deleted if instance is terminated
● IO Freeze during backup
● MultiAZ I/O freeze is for shorter period
● Change to retention period is immediate (outage due to backup/reboot)
Demo:
● Modify retention period
● Point in time recovery
RDS read replicas (Console)
Demo:
● Create read replica in another region
● Promote read replica to replace primary
RDS manual snapshots (CLI/Console)
Create snapshot
aws rds createdbsnapshot dbinstanceidentifier edureka dbsnapshotidentifier dbsnap11122015
aws rds describedbsnapshots dbsnapshotidentifier dbsnap11122015
Web Console
Sharing snapshots
Copying snapshots
Copy snapshot (cross region)
aws rds copydbsnapshot sourcedbsnapshotidentifier
arn:aws:rds:uswest2:560125328540:snapshot:dbsnap11122015 region useast1
targetdbsnapshotidentifier dbsnap11122015dr
aws region useast1 rds describedbsnapshots dbsnapshotidentifier dbsnap11122015dr
Restore
aws rds restoredbinstancefromdbsnapshot dbinstanceidentifier edureka2 dbsnapshotidentifier
dbsnap11122015
aws rds describedbinstances dbinstanceidentifier edureka2
aws rds modifydbinstance dbinstanceidentifier edureka2 dbsecuritygroups sg530d3b37
Note: create vpc security group
Copy snapshot (from automated snapshot)
aws rds describedbsnapshots snapshottype automated
aws rds copydbsnapshot sourcedbsnapshotidentifier rds:edureka201512111449
targetdbsnapshotidentifier dbcopyedureka20151211
aws rds describedbsnapshots dbsnapshotidentifier dbcopyedureka20151211
Cleanup
aws rds describedbsnapshots
aws region useast1 rds describedbsnapshot
aws rds deletedbsnapshot dbsnapshotidentifier dbcopyedureka20151211
aws region useast1 rds deletedbsnapshot dbsnapshotidentifier dbsnapcopy
aws rds deletedbsnapshot dbsnapshotidentifier dbsnap
aws rds deletedbsnapshot dbsnapshotidentifier rds:edureka201512111449
Note: automated cannot be deleted
S3 backups (CLI/Console)
Web Console
● Create bucket for backup
● Setup crossregion replication
● Setup object lifecycle policies
Move archive to S3 bucket
aws s3 mv backup.zip s3://edurekabackup/backup.zip
aws s3 ls s3://edurekabackup
Restore: Download backup file
aws s3 mv s3://edurekabackup/backup.zip .
ls backup.zip
aws s3 ls s3://edurekabackup
Amazon Glacier (CLI)
Create archive vault
aws glacier createvault vaultname edureka accountid
Upload archive
aws glacier uploadarchive vaultname edureka accountid body backup.zip
List archives
aws glacier describevault vaultname edureka accountid
Initiate inventory retrieval job
aws glacier initiatejob accountid vaultname edureka jobparameters '{"Type": "inventoryretrieval"}'
NOTE: can’t retreive this early
Initiate archive retrieval
aws glacier initiatejob accountid vaultname edureka jobparameters '{"ArchiveId":
"T4SBmEIfLHqxlGPGis27khJ7ZHVCdjUdY0n62kfpkG5kOR7v87UPzwmjdY9NOvKx_qx_VHxBxEI74OV4Qv
U5W9fNprx5i6b8kN0v46tH3FZGwNUCBLkFvbhpcil7invEHPdm_A", "Type":"archiveretrieval"}'
View job status
aws glacier listjobs accountid vaultname edureka
Cleanup
aws glacier deletearchive accountid vaultname edureka archiveid
T4SBmEIfLHqxlGPGis27khJ7ZHVCdjUdY0n62kfpkG5kOR7v87UPzwmjdY9NOvKx_qx_VHxBxEI74OV4QvU
5W9fNprx5i6b8kN0v46tH3FZGwNUCBLkFvbhpcil7invEHPdm_A
aws glacier deletevault accountid vaultname edureka
Note: Vault cannot be deleted unless archives are deleted
Review/Assignments
Topic Review questions
RDS ● How are automated backups different from manual
snapshots/backups?
● How can read replicas be used for DR?
● How can RDS snapshots be used for creating test/development
environments?
● What is the maximum retention period of automated backups?
EBS ● How can one ensure data integrity when EBS snapshots are being
taken?
● What are the ways in which EC2 instances can be recreated in a
different region for DR?
S3 ● What are object life cycle policies used for?
● How can one protect data against failures of a region?
● What are the 3 different types of storage offered by S3?
Glacier ● How long does it take typically to retreive an archive?
● Compare Glacier pricing to S3 pricing
Assignments
● Create a complete backup, DR and test/development environment policy document for
your project/organization
● Implement your backup/DR/Environment policies using:
○ EBS: Snapshots, AMI, Sharing AMI/Snapshots, Crossregion copying of
Snapshots/AMI
○ RDS: Automated backup, manual db snapshots, read replicas
○ S3: Object lifecycle policies, crossregion replication, Glacier