Serverless Big Data Analytics with Amazon Athena and Amazon Quicksight - May 2017 AWS Online Tech Talks
•Download as PPTX, PDF•
5 likes•4,913 views
Learning Objectives:
- Learn how to use Amazon Athena to query various data formats in Amazon S3
- Learn how to use Amazon QuickSight to visualize the results of your Athena query with and without using SPICE
Querying and analyzing big data can be complicated and expensive. It requires you to setup and manage databases, data warehouses, and business intelligence applications; all of which require time, effort, and resources. Using Amazon Athena and Amazon QuickSight, you can avoid the cost and complexity by creating a fast, scalable and serverless cloud analytics solution without the need to invest in databases, data warehouses, complex ETL solutions, and BI applications. In this tech talk, we will demonstrate how you can build a serverless big data analytics solution using Amazon Athena and Amazon QuickSight.
1 of 27
More Related Content
Serverless Big Data Analytics with Amazon Athena and Amazon Quicksight - May 2017 AWS Online Tech Talks
2. 1990 2000 2010 2020
Generated Data
Available for Analysis
Sources:
Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011
IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares
Data Volume
Year
The Dark Data Problem
Most generated data is unavailable for analysis
7. Athena is Serverless
• No Infrastructure or
administration
• Zero Spin up time
• Transparent upgrades
8. Amazon Athena is Easy To Use
• Log into the Console
• Create a table
• Type in a Hive DDL Statement
• Use the console Add Table wizard
• Start querying
9. Amazon Athena is Highly Available
• You connect to a service endpoint or log into the console
• Athena uses warm compute pools across multiple
Availability Zones
• Your data is in Amazon S3, which is also highly available
and designed for 99.999999999% durability
10. Query Data Directly from Amazon S3
• No loading of data
• Query data in its raw format
• Text, CSV, JSON, weblogs, AWS service logs
• Convert to an optimized form like ORC or Parquet for the best
performance and lowest cost
• No ETL required
• Stream data from directly from Amazon S3
• Take advantage of Amazon S3 durability and availability
11. Use ANSI SQL
• Start writing ANSI SQL
• Support for complex joins, nested
queries & window functions
• Support for complex data types
(arrays, structs)
• Support for partitioning of data by
any key
• (date, time, custom keys)
• e.g., Year, Month, Day, Hour or
Customer Key, Date
12. Simple Pricing - $5/TB Scanned
• Pay by the amount of data scanned per query
• Ways to save costs
• Compress
• Convert to Columnar format
• Use partitioning
• Free: DDL Queries, Failed Queries
Dataset Size on Amazon S3 Query Run time Data Scanned Cost
Logs stored as Text
files
1 TB 237 seconds 1.15TB $5.75
Logs stored in
Apache Parquet
format*
130 GB 5.13 seconds 2.69 GB $0.013
Savings 87% less with Parquet 34x faster 99% less data scanned 99.7% cheaper
14. Amazon QuickSight is a Business Analytics Service that lets business users
quickly and easily visualize, explore, and share insights from their data.
15. Who Is QuickSight For?
DATA
PROFESSIONALS
BUSINESS
PROFESSIONALS
DATA CONSUMERS
21. New Connectors, Enhanced Visualizations, Expansion to new regions, SPICE updates, and more
New capabilities in QuickSight
Std. Edition
KPI Charts
Export to
CSV
AD
Connector
US Ohio
East Region
CloudTrail
Audit Logs
Ent. Edition
Sched.
Refresh
Excel Range
Detection
Presto
Connector
SparkSQL
Connector
Nov ’16
GA
Dec ‘16 Feb ‘17 Apr ‘17 May ‘17