Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Databases - State of the Union

•
•
Amazon Web Services
Amazon Web Services

The document provides an overview of Amazon Web Services (AWS) databases and analytics services. It summarizes that AWS has significantly expanded its database and analytics offerings between 2015-2018, with over 750 new features and 10 new services launched. It highlights several core AWS database and analytics services, including Amazon DynamoDB, Amazon RDS, Amazon Aurora, Amazon Neptune, and Amazon ElastiCache. It also discusses how customers are migrating workloads from on-premises databases to AWS databases and analytics services.

1 of 43
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ric Harvey, Technical Developer Evangelist @ric__harvey AWS Builders’ Day Databases SOTU wifi: etcvenues pass: wifi8118
Databases at AWS Today
2015 2016 2017 2018 Accelerating the pace of innovation *Projected number of releases to year-end features in 2018 new database and analytics services in 2018 150 210 100 325*
Delivering capabilities to meet customer needs Glue Amazon Managed Streaming for Kafka Amazon QLDB Amazon Managed Blockchain Neptune 785+features released since 2015 Athena Amazon Timestream Redshift Spectrum Amazon RDS on VMware AWS Lake Formation QuickSight ML Insights *Projected number of launches to year-end 2017
Most startup database & analytics cloud customers
AWS databases services Purpose-built for all your app needs DynamoDB NeptuneRDS Aurora CommercialCommunity Timestream QLDBElastiCache Relational Key-value Document In-memory Graph Time series Ledger
Amazon DynamoDB
Amazon DynamoDB Fast and flexible key value database service for any scale Comprehensive security Encrypts all data by default and fully integrates with AWS Identity and Access Management for robust security Performance at scale Consistent, single-digit millisecond response times at any scale; build applications with virtually unlimited throughput Global database for global users and apps Build global applications with fast access to local data by easily replicating tables across multiple AWS Regions Serverless No hardware provisioning, software patching, or upgrades; scales up or down automatically; continuously backs up your data
Amazon DynamoDB Delivering on customer needs VPC Endpoint s April 2017 Auto Scalin g June 2017 DynamoDB Accelerator (DAX) The picture can't be displayed. Time to Live (TTL) February 2017 Global tables On-demand backup Encryption at rest February 2018 December 2017 March 2018 The picture can't be displayed . Point-in- time recovery
DynamoDB powers the world’s largest applications
Amazon RDS
Amazon RDS Managed relational database service with a choice of six popular database engines Available & durable Automatic Multi-AZ data replication; automated backup, snapshots, failover Easy to administer No need for infrastructure provisioning, installing and maintaining DB software Highly scalable Scale database compute and storage with a few clicks with no application downtime Fast & secure SSD storage and guaranteed provisioned I/O; data encryption at rest and in transit
Databases in private data centers are still difficult and expensive to set up and manage Difficult to set up and manage databases for high availability across multiple nodes Personnel needed to create the database image, install operating system, packages, and setup Burdensome to support multiple versions and applying patching ?
Amazon RDS on VMware (Preview) Managed service for on-premises databases RDS deployed as a service in on-premises VMware private data centers (vSphere) Automates management of on-premises databases and hybrid backup and scaling Available and durable Enable hybrid features and tap into AWS for high availability, backup, and restore Secure and compliant Automate management of databases for workloads that must remain on-premises to adhere to strict data policies Fully managed Easy to provision, monitor, and operate relational databases in your private data center Scalability and performance Scale storage, compute, and memory of on-premises databases from a single, simple interface
How it works Amazon RDS on VMware RDS on VMware Deploy RDS on VMware in your private data center and manage on- premises databases using a single RDS interface RDS interface Use the Amazon RDS console, APIs, or CLI to provision databases Log in or create AWS account Find RDS on VMware in the console and choose AWS region Download and install the connector Download & install the RDS connector in your VMware vSphere environment to establish secure VPN connection between AWS and your private data center Automate database management RDS on VMware automates database management tasks including provisioning, patching, backups and failover
Amazon Aurora
Amazon Aurora MySQL and PostgreSQL compatible relational database built for the cloud Performance and availability of commercial-grade databases at 1/10th the cost Availability and durability Fault-tolerant, self-healing storage; six copies of data across three AZs; continuous backup to S3 Fully managed Managed by RDS: no hardware provisioning, software patching, setup, configuration, or backups Highly secure Network isolation, encryption at rest/transit Performance and scalability 5x throughput of standard MySQL and 3x of standard PostgreSQL; scale-out up to 15 read replicas
MySQL Physical Replication - PREVIEW Serverless MySQL 5.7- compatible - Preview Parallel Query Serverless - PREVIEW TLS Support for MySQL Serverless MySQL in OSU PostgreSQL - Fast Database Cloning Parallel Query for MySQL - Preview Auto Scaling PostgreSQL Read Replicas in GovCloud MySQL launch in HKG Multiple cross- region read replicas 50+features released since re:Invent 2017 MySQL Serverless availability in all Aurora regions MySQL launch in SIN PostgreSQL in SFO PostgreSQL in PDT PostgreSQL 2.0, compatible with PostgreSQL 10.4 PostgreSQL in NRT PostgreSQL BAA Inclusion (HIPAA eligibility) Aurora PostgreSQL 1.1 (compatible with PostgreSQL 9.6.6) HTTP SQL Endpoint for Serverless Support for DB Cluster Stop / Start Aurora 5.7 – GA Backtrack MySQL launch in PDT PostgreSQL in ICN Encrypted MySQL (on-premises/EC2) to Aurora Migration Synchronous Lambda for MySQL MySQL General, Slow, and Error log Exports to CloudWatch Logs MySQL Custom Endpoints PostgreSQL 1.3, compatible with PostgreSQL 9.6.9 Auto Scaling PostgreSQL Read Replicas in Ningxia PostgreSQL 1.2, compatible with PostgreSQL 9.6.8, in GovCloud Hash joins for MySQL Encrypted Snapshot Import from RDS for PostgreSQL to Aurora PostgreSQL PostgreSQL 1.2, compatible with PostgreSQL 9.6.8 PostgreSQL in YUL, FRA, SYD, and BOM Auto Scaling PostgreSQL Read Replicas MySQL nAZ support Multi- master PostgreSQL Query Plan Management - Preview *Projected number of launches to year-end 2017 Amazon Aurora Delivering on customer needs
Aurora is still the fastest growing service in AWS history
Aurora Global Database (GA) High-performance database for globally-distributed applications Single Global Database with cross region replication Replication typically completes in less than a second No impact on database performance Write master in one region and read replicas in other regions Cross-region disaster recovery Local read latency for applications with global users Primary Region Secondary Region Application Storage Storage Replication <1s
Amazon Neptune
Use Cases For Highly Connected Data Social Networking Recommendations Knowledge Graphs Fraud Detection Life Sciences Network & IT Operations
Building Applications Over Highly Connected Data Retail Fraud DetectionRestaurant RecommendationsSocial Networks
Challenges Of Existing Graph Databases Difficult to maintain high availability Difficult to scale Limited support for open standards Too expensive
Amazon Neptune: A Fully Managed Graph Database Fast Reliable Open Query billions of relationships with millisecond latency 6 replicas of your data across 3 AZs with full backup and restore Build powerful queries easily with Gremlin and SPARQL Supports Apache TinkerPop & W3C RDF graph models Easy
Amazon Elasticache
In-memory key-value store supporting • Redis 5.0.0 (backported fixes from 5.0.1) • Memcached 1.4.34 High-performance Fully managed; zero admin Highly available and reliable Hardened by Amazon Amazon ElastiCache
Redis Use Case
Memcache Use Cache
Migration
Old world commercial relational databases Lock-inProprietary Punitive licensing Very expensive You’ve got mail
Moving to open database engines
Moving to open database engines
Customers tell us: they have three type of projects Quickly build new apps in the cloud Gain new insights “Lift and shift” existing apps to the cloud
AWS Database Migration Service M I G R A T I N G D A T A B A S E S T O A W S Migrate between on-premises and AWS Migrate between databases Automated schema conversion Data replication for zero downtime migration
Customers want to lift and shift to the cloud Relational databases Non-relational databases Data warehouses Hadoop and Spark Redshif t EM R Operational analytics Elasticsearch ServiceAuror a DynamoD B Business Intelligence QuickSightRDS
Customers are migrating their workloads to AWS Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS, several of which also include the migration of production databases to Amazon Aurora. Wappa migrated from their Oracle database to Amazon Aurora and improved their reporting time per user by 75 percent. Trimble migrated their Oracle databases to Amazon RDS and project they will pay about 1/4th of what they paid when managing their private infrastructure. Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines and get insights to decision makers faster and more frequently. Equinox Fitness migrated its Teradata on-premises data warehouse to Amazon Redshift. They went from static reports to a modern data lake that delivers dynamic reports. Eventbrite moved from Cloudera to Amazon EMR and were able to cut costs dramatically, spinning clusters up/down on-demand and using Spot (saving > 80%) and Reserved Instances. By December 2018, Amazon.com will have migrated 88% of their Oracle DBs (and 97% of critical system DBs) moved to Amazon Aurora and Amazon DynamoDB. They also migrated their 50 PB Oracle Data Warehouse to AWS (Amazon S3, Amazon Redshift, and Amazon EMR). Samsung Electronics migrated their Cassandra clusters to Amazon DynamoDB for their Samsung Cloud workload with 70% cost savings.
>100,000 databases migrated with DMS
Database Tools
Amazon Athena Zero setup cost; just point to S3 and start querying ANSI SQL interface, JDBC/ODBC drivers, multiple formats, compression types, and complex joins and data types Serverless: zero infrastructure, zero administration Integrated with QuickSight Pay only for queries run; save 30–90% on per-query costs through compression Query Instantly Open EasyPay per query Interactive query service to analyze data in Amazon S3 using standard SQL No infrastructure to set up or manage and no data to load Ability to run SQL queries on data archived in Amazon Glacier SQL
AWS Glue—Serverless Data catalog & ETL service Data Catalog ETL Job authoring Discover data and extract schema Auto-generates customizable ETL code in Python and Spark Automatically discovers data and stores schema Data searchable, and available for ETL Generates customizable code Schedules and runs your ETL jobs Serverless
Join the conversation awsdevelopers.slack.com Rooms: • Containers • Serverless • appdev • Kubernetes • Community • And more! DM @ric__Harvey on Twitter and send me your email address for an invite Get an invite:
Thank you! Ric Harvey, Technical Developer Evangelist @ric__harvey

More Related Content

Databases - State of the Union

  • 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ric Harvey, Technical Developer Evangelist @ric__harvey AWS Builders’ Day Databases SOTU wifi: etcvenues pass: wifi8118
  • 3. 2015 2016 2017 2018 Accelerating the pace of innovation *Projected number of releases to year-end features in 2018 new database and analytics services in 2018 150 210 100 325*
  • 4. Delivering capabilities to meet customer needs Glue Amazon Managed Streaming for Kafka Amazon QLDB Amazon Managed Blockchain Neptune 785+features released since 2015 Athena Amazon Timestream Redshift Spectrum Amazon RDS on VMware AWS Lake Formation QuickSight ML Insights *Projected number of launches to year-end 2017
  • 5. Most startup database & analytics cloud customers
  • 6. AWS databases services Purpose-built for all your app needs DynamoDB NeptuneRDS Aurora CommercialCommunity Timestream QLDBElastiCache Relational Key-value Document In-memory Graph Time series Ledger
  • 8. Amazon DynamoDB Fast and flexible key value database service for any scale Comprehensive security Encrypts all data by default and fully integrates with AWS Identity and Access Management for robust security Performance at scale Consistent, single-digit millisecond response times at any scale; build applications with virtually unlimited throughput Global database for global users and apps Build global applications with fast access to local data by easily replicating tables across multiple AWS Regions Serverless No hardware provisioning, software patching, or upgrades; scales up or down automatically; continuously backs up your data
  • 9. Amazon DynamoDB Delivering on customer needs VPC Endpoint s April 2017 Auto Scalin g June 2017 DynamoDB Accelerator (DAX) The picture can't be displayed. Time to Live (TTL) February 2017 Global tables On-demand backup Encryption at rest February 2018 December 2017 March 2018 The picture can't be displayed . Point-in- time recovery
  • 10. DynamoDB powers the world’s largest applications
  • 12. Amazon RDS Managed relational database service with a choice of six popular database engines Available & durable Automatic Multi-AZ data replication; automated backup, snapshots, failover Easy to administer No need for infrastructure provisioning, installing and maintaining DB software Highly scalable Scale database compute and storage with a few clicks with no application downtime Fast & secure SSD storage and guaranteed provisioned I/O; data encryption at rest and in transit
  • 13. Databases in private data centers are still difficult and expensive to set up and manage Difficult to set up and manage databases for high availability across multiple nodes Personnel needed to create the database image, install operating system, packages, and setup Burdensome to support multiple versions and applying patching ?
  • 14. Amazon RDS on VMware (Preview) Managed service for on-premises databases RDS deployed as a service in on-premises VMware private data centers (vSphere) Automates management of on-premises databases and hybrid backup and scaling Available and durable Enable hybrid features and tap into AWS for high availability, backup, and restore Secure and compliant Automate management of databases for workloads that must remain on-premises to adhere to strict data policies Fully managed Easy to provision, monitor, and operate relational databases in your private data center Scalability and performance Scale storage, compute, and memory of on-premises databases from a single, simple interface
  • 15. How it works Amazon RDS on VMware RDS on VMware Deploy RDS on VMware in your private data center and manage on- premises databases using a single RDS interface RDS interface Use the Amazon RDS console, APIs, or CLI to provision databases Log in or create AWS account Find RDS on VMware in the console and choose AWS region Download and install the connector Download & install the RDS connector in your VMware vSphere environment to establish secure VPN connection between AWS and your private data center Automate database management RDS on VMware automates database management tasks including provisioning, patching, backups and failover
  • 17. Amazon Aurora MySQL and PostgreSQL compatible relational database built for the cloud Performance and availability of commercial-grade databases at 1/10th the cost Availability and durability Fault-tolerant, self-healing storage; six copies of data across three AZs; continuous backup to S3 Fully managed Managed by RDS: no hardware provisioning, software patching, setup, configuration, or backups Highly secure Network isolation, encryption at rest/transit Performance and scalability 5x throughput of standard MySQL and 3x of standard PostgreSQL; scale-out up to 15 read replicas
  • 18. MySQL Physical Replication - PREVIEW Serverless MySQL 5.7- compatible - Preview Parallel Query Serverless - PREVIEW TLS Support for MySQL Serverless MySQL in OSU PostgreSQL - Fast Database Cloning Parallel Query for MySQL - Preview Auto Scaling PostgreSQL Read Replicas in GovCloud MySQL launch in HKG Multiple cross- region read replicas 50+features released since re:Invent 2017 MySQL Serverless availability in all Aurora regions MySQL launch in SIN PostgreSQL in SFO PostgreSQL in PDT PostgreSQL 2.0, compatible with PostgreSQL 10.4 PostgreSQL in NRT PostgreSQL BAA Inclusion (HIPAA eligibility) Aurora PostgreSQL 1.1 (compatible with PostgreSQL 9.6.6) HTTP SQL Endpoint for Serverless Support for DB Cluster Stop / Start Aurora 5.7 – GA Backtrack MySQL launch in PDT PostgreSQL in ICN Encrypted MySQL (on-premises/EC2) to Aurora Migration Synchronous Lambda for MySQL MySQL General, Slow, and Error log Exports to CloudWatch Logs MySQL Custom Endpoints PostgreSQL 1.3, compatible with PostgreSQL 9.6.9 Auto Scaling PostgreSQL Read Replicas in Ningxia PostgreSQL 1.2, compatible with PostgreSQL 9.6.8, in GovCloud Hash joins for MySQL Encrypted Snapshot Import from RDS for PostgreSQL to Aurora PostgreSQL PostgreSQL 1.2, compatible with PostgreSQL 9.6.8 PostgreSQL in YUL, FRA, SYD, and BOM Auto Scaling PostgreSQL Read Replicas MySQL nAZ support Multi- master PostgreSQL Query Plan Management - Preview *Projected number of launches to year-end 2017 Amazon Aurora Delivering on customer needs
  • 19. Aurora is still the fastest growing service in AWS history
  • 20. Aurora Global Database (GA) High-performance database for globally-distributed applications Single Global Database with cross region replication Replication typically completes in less than a second No impact on database performance Write master in one region and read replicas in other regions Cross-region disaster recovery Local read latency for applications with global users Primary Region Secondary Region Application Storage Storage Replication <1s
  • 22. Use Cases For Highly Connected Data Social Networking Recommendations Knowledge Graphs Fraud Detection Life Sciences Network & IT Operations
  • 23. Building Applications Over Highly Connected Data Retail Fraud DetectionRestaurant RecommendationsSocial Networks
  • 24. Challenges Of Existing Graph Databases Difficult to maintain high availability Difficult to scale Limited support for open standards Too expensive
  • 25. Amazon Neptune: A Fully Managed Graph Database Fast Reliable Open Query billions of relationships with millisecond latency 6 replicas of your data across 3 AZs with full backup and restore Build powerful queries easily with Gremlin and SPARQL Supports Apache TinkerPop & W3C RDF graph models Easy
  • 27. In-memory key-value store supporting • Redis 5.0.0 (backported fixes from 5.0.1) • Memcached 1.4.34 High-performance Fully managed; zero admin Highly available and reliable Hardened by Amazon Amazon ElastiCache
  • 31. Old world commercial relational databases Lock-inProprietary Punitive licensing Very expensive You’ve got mail
  • 32. Moving to open database engines
  • 33. Moving to open database engines
  • 34. Customers tell us: they have three type of projects Quickly build new apps in the cloud Gain new insights “Lift and shift” existing apps to the cloud
  • 35. AWS Database Migration Service M I G R A T I N G D A T A B A S E S T O A W S Migrate between on-premises and AWS Migrate between databases Automated schema conversion Data replication for zero downtime migration
  • 36. Customers want to lift and shift to the cloud Relational databases Non-relational databases Data warehouses Hadoop and Spark Redshif t EM R Operational analytics Elasticsearch ServiceAuror a DynamoD B Business Intelligence QuickSightRDS
  • 37. Customers are migrating their workloads to AWS Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS, several of which also include the migration of production databases to Amazon Aurora. Wappa migrated from their Oracle database to Amazon Aurora and improved their reporting time per user by 75 percent. Trimble migrated their Oracle databases to Amazon RDS and project they will pay about 1/4th of what they paid when managing their private infrastructure. Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines and get insights to decision makers faster and more frequently. Equinox Fitness migrated its Teradata on-premises data warehouse to Amazon Redshift. They went from static reports to a modern data lake that delivers dynamic reports. Eventbrite moved from Cloudera to Amazon EMR and were able to cut costs dramatically, spinning clusters up/down on-demand and using Spot (saving > 80%) and Reserved Instances. By December 2018, Amazon.com will have migrated 88% of their Oracle DBs (and 97% of critical system DBs) moved to Amazon Aurora and Amazon DynamoDB. They also migrated their 50 PB Oracle Data Warehouse to AWS (Amazon S3, Amazon Redshift, and Amazon EMR). Samsung Electronics migrated their Cassandra clusters to Amazon DynamoDB for their Samsung Cloud workload with 70% cost savings.
  • 40. Amazon Athena Zero setup cost; just point to S3 and start querying ANSI SQL interface, JDBC/ODBC drivers, multiple formats, compression types, and complex joins and data types Serverless: zero infrastructure, zero administration Integrated with QuickSight Pay only for queries run; save 30–90% on per-query costs through compression Query Instantly Open EasyPay per query Interactive query service to analyze data in Amazon S3 using standard SQL No infrastructure to set up or manage and no data to load Ability to run SQL queries on data archived in Amazon Glacier SQL
  • 41. AWS Glue—Serverless Data catalog & ETL service Data Catalog ETL Job authoring Discover data and extract schema Auto-generates customizable ETL code in Python and Spark Automatically discovers data and stores schema Data searchable, and available for ETL Generates customizable code Schedules and runs your ETL jobs Serverless
  • 42. Join the conversation awsdevelopers.slack.com Rooms: • Containers • Serverless • appdev • Kubernetes • Community • And more! DM @ric__Harvey on Twitter and send me your email address for an invite Get an invite:
  • 43. Thank you! Ric Harvey, Technical Developer Evangelist @ric__harvey