Initiate Edinburgh 2019 - Migrating Data to the Cloud

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migrating Data to the Cloud
Angus McAllister
Solutions Architecture Team Lead

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
Overview of AWS Transfer Services
Data Migration
AWS Direct Connect
AWS DataSync
AWS Snowball Family
AWS Transfer for SFTP
AWS Storage Gateway Family
S3 Transfer Acceleration
AWS Partners

Why are we here?
We’ll be talking about you moving your data
• Data tied to applications
you’re migrating
• Data in bulk storage
(e.g. NAS filers)
• Data connected to
on-premises applications
• Data as part of continuous
edge & hybrid workflows
You have IT resources and want to move them to the cloud

Why move to the cloud?
Common drivers of moving data to the cloud:
Pain points of data migration:
While every cloud adoption journey is unique, all require a data migration step, which
can be hindered by blockers such as:
Cost
effectiveness
Workforce
productivity
Business
agility
Operational
resilience
Access to
innovative
services
Disruption Speed Costs Channels Cloud
compatibility

Migrating data: Five key questions
1) What kind of data, and where to?
4) How much data, and in what timescale?
2) One-off or continuous movement?
3) One-way or bi-directional?
5) How about the WAN?
Amazon S3
Amazon EC2
Amazon
EFS
Amazon
EBS
Amazon
Aurora
Amazon
EMR
Amazon
Glacier
Amazon
RDS
AWS
IoT
Amazon
Redshift
Files Block
volumes
Databases IoT Streams

Portfolio breadth and depth
Amazon Elastic
Block Store
(Amazon EBS)
Amazon Simple
Storage Service
(Amazon S3)
Amazon Glacier
Data movement
OnlineOffline
AWS Snow family
AWS Storage Gateway family
AWS Direct Connect
AWS DataSync
Amazon S3
Transfer Acceleration
Storage partners
Amazon Kinesis Data Streams,
Amazon Kinesis Video Streams, &
Amazon Kinesis Firehose
Amazon Elastic File
System (Amazon EFS)
New! Devices
New! Storage classes
New! Management features
New!
Transfer
services

AWS data transfer & hybrid storage
Online
data transfer
Hybrid
storage
Offline
data transfer
Private
network
connections
to AWS
Load
streaming
data into
Amazon S3
Ship static
data into and
out of
Amazon S3
Access AWS
storage from
on-premises
Edge
locations for
Amazon S3
enabled
applications
Online
transfer of
active data
AWS
DataSync
AWS
Transfer
for SFTP
SFTP transfers
into Amazon S3
NEW NEW
Storage and
compute in
disconnected
environments

AWS Direct Connect
Reduce
bandwidth
Costs
Consistent
network
performance
Compatible with
all AWS services
Private
connectivity to
VPC
Elastic Simple
• Establish private connectivity between AWS and your data center
• Dedicated connection can be partitioned into multiple virtual interfaces
• Maintain network separation between public and private environments
BENEFITS

What use cases need to transfer active data?
Migration of active
application data
Transferring data
for time sensitive
in-cloud analysis
Replication of data
for business
continuity
Online data transfer

What is AWS DataSync?
Online transfer service that simplifies, automates, and accelerates
moving data between on-premises storage and AWS
Fast data
transfer
Cost-
effective
Enterprise
ready
Combines the speed and reliability of network acceleration
software with the cost-effectiveness of open source tools
Easy to use Secure and
reliable

As simple as one, two, three ...
On-premises NFS file
systems, Amazon S3 buckets, or
Amazon EFS file systems
Create a task by configuring
storage locations and options
Monitor in the
console or through
Amazon CloudWatch
Start the task and
monitor progress
VMware ESXi
8 vCPU
32 GB memory
Deploy the agent and
associate with your AWS account
1 2 3

Shared
file system
NFS TLS
How AWS DataSync works
Customer Premises
Amazon S3
bucket
AWS Storage resources
AWS
DataSync
Agent deployed
on-premises for
fast access to
local storage
AWS Region
Amazon EFS
file system
AWS DataSync
agent
Data transfer
over the WAN via
efficient purpose-
built protocol
Managed from the
console or AWS
Command Line
Interface (AWS CLI)
Service in AWS
writes or reads
data from AWS
storage services

Reduce infrastructure with AWS DataSync
and Storage Gateway
AWS DataSync transfers to Amazon S3 as part of migration; incremental sync keeps Amazon S3 up to date1
2 Latency-sensitive applications can access data through file gateway, can decommission NAS
NAS
Amazon S3
bucket
Customer Premises Region
Applications 2
AWS
DataSync
AWS DataSync
agent
1

Move large volumes into the cloud
Moving large volumes of data over the internet can
take years
The cloud is not always accessible from remote
locations where connectivity is limited or
intermittent
Traditional shipping of conventional hard drives is
laborious and error prone
AWS Snowball AWS Snowball Edge

Need for edge analytics in remote and harsh environments
Various industries ranging from research to transport necessitate edge
computing capabilities to help cope with environments where data generation
is decentralised, data volumes are significant, and network connectivity is either
inaccessible or intermittent.
Research Industrial Healthcare Transport

Who's Using It?
x
ISV
PartnersCustomers

SFTP: It’s here; it’s everywhere
Protocol is deeply embedded in workflows across a variety of industries
Financial services
$
Retail
Healthcare … and more

Migrate SFTP workflows to AWS
To benefit from
• Durable storage in Amazon S3
• Processing, archival and retention of data
• Integration with AWS services
SFTP SFTP server and data
Will your hundreds of partners
change their workflows to
integrate with the cloud??

Fully managed service enabling transfer
of data over SFTP while stored in Amazon S3
Seamless migration
of existing workflows
Native integration
with AWS services
Simple
to use
Cost
effective
Fully managed
in AWS
Enterprise
ready

In as simple as three steps
Your users can now use your AWS SFTP server endpoint to transfer data
Map your hostname Select your Amazon S3
bucket(s)
Set up your users
1 2 3

Your SFTP architecture now
The same hundreds of users
now serviced by AWS Transfer,
without requiring changes
+ Enterprise ready
+ No end user disruption
+ Fully managed servers
+ Simple to use
+ Pay as you use
+ Native cloud integrations
Amazon S3AWS SFTP
AWS Transfer
for SFTP

AWS Storage Gateway
Existing Data
Archives
New Data
Corporate
Data Centres or
Offices
AWS Storage Gateway
HTTPS
NFS
SMB
iSCSI
VTL
Hybrid storage service enabling applications to seamlessly use AWS storage
On-Premises Gateway Service
Amazon S3
Amazon EBS
snapshots
Amazon Glacier

The AWS Storage Gateway family
File gateway
Store and access objects in Amazon S3 from file-based
applications with local caching
Volume gateway
Block storage on-premises backed by cloud storage with local
caching, Amazon EBS snapshots, and clones
Tape gateway
Drop-in replacement for physical tape infrastructure backed by
cloud storage with local caching
Three gateway types provide file, block, and tape storage interfaces

AWS Storage Gateway key features
Standard storage
protocols
Seamlessly connect on-premises
applications to AWS storage without
modifying applications
Fully managed cache
Frequently-used data cached locally
For low-latency access
Optimised data transfer
Secures upload of changed data and
downloads requested data
AWS-integrated
Manage on-premises gateways in AWS Management
Console and use your data across the entire portfolio
of AWS applications and services

Some AWS Storage Gateway customers

File gateway overview
Store and access objects in Amazon S3 from file-based applications with local caching
Use cases
• Backup files and
databases as objects
• Hybrid workflows for on-
premises data (e.g., data
pipelines)
• Low-latency on-premises
access to data in cloud
(e.g., media editing)
• Content distribution and
collaboration
File Gateway Objects in your
Amazon S3 bucket
NFS v3/v4
SMB v2/v3

Volume gateway overview
• Supports iSCSI block protocol
• Available in two modes: cached and stored
• Compresses and tiers partial or whole volumes to the cloud to reduce
on-premises SAN/NAS footprint
• Take EBS snapshots of on-premises volumes and store them in cloud
Cloud-backed block storage presented on-premises
Key
Capabilities
Application
server
Amazon EBS
snapshots
ServiceVolume
Gateway
Volume stored in
Amazon S3
HTTPSiSCSI
Customer Premises

• Emulates physical tape device with a media changer and tape drives
• Virtual tapes written to Amazon S3
• Ejected virtual tapes are marked “read only” and moved to “tape shelf” in
Amazon Glacier
• Recovery: Retrieve tapes to library (3–5 hours) and read data from a gateway
Tape gateway overview
Virtual tapes presented to on-premises backup applications
Key
Capabilities
Backup
server
Tape library Tape shelf
Customer Premises
iSCSI HTTPS
Media
changer
Tape Gateway
Tape
drives

Storage Gateway is available on a variety of form
factors
Virtual machines Amazon EC2
Hardware
appliance
Corporate data centers In AWS
Remote locations,
branch offices
WhereWhen
Leverage existing VM
infrastructure
In-cloud data
retrieval/access/DR
Out of the box
simplified experience

Amazon S3 Transfer Acceleration
AWS Edge
Location
Optimised
Throughput!
Change your endpoint, not your code
Leverages Amazon CloudFront
global edge locations
Optimised protocols
No firewall exceptions
No client software required

1 2 3 4 5 6 7 8 9 10 11 12
Time[hrs]
500 GB upload from these edge locations to a bucket in Singapore
Public Internet
How Fast is Amazon S3 Transfer Acceleration?
S3 Transfer Acceleration
On average, we have seen 171%
improvement over regular Amazon
S3 when uploading over long
distances

Backup & restore Archive Primary storage BC/DRData migration
AWS Partner Network (APN): Migration & storage

Thank you!

Initiate Edinburgh 2019 - Migrating Data to the Cloud

More Related Content

Initiate Edinburgh 2019 - Migrating Data to the Cloud