Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

A developer's introduction to big data processing with Azure Databricks

•Download as PPTX, PDF•
•

The document discusses how companies can use big data analytics and Azure Databricks to improve their customer experiences and grow their business. It provides an overview of how Wide World Importers seeks to expand its customers through an omni-channel strategy using analytics from data across its retail stores, website, and mobile apps. The document also outlines logical architectures for ingesting, storing, preparing, training models on, and serving data using Azure Databricks and other Azure services.

1 of 21
A developer's introduction to big data processing with Azure Databricks
A developer's introduction to big data processing with Azure Databricks
“More than any other factor, customer experiences determine whether companies thrive and profit, or struggle and fade.” – Forrester Research
Speed 79% won’t return to a slow website Personalization 38% won't call again if they have to repeat themselves Consistency 65% get frustrated with inconsistent device experiences
Harness the power of Big Data analytics apps to exceed customer needs
Enhance any type of app with Big Data analytics
Introducing Wide World Importers
Online shopping through company website E-commerce directly in the consumer’s hand, anywhere Wide World ImportersWide World Importers Retail stores
Wide World Importers Web e-commerce Mobile e-commerce Wide World Importers seeks to expand customers through an omni-channel strategy Retail stores
The solutions needed to reach more customers and grow the business 1. Scale with ease to reach more consumers 2. Unlock business insights from unstructured data 3. Enhance user experience with advanced analytics 4. Apply real-time analytics for instant updates 5. Infuse AI into apps to actively engage with customers
A Z U R E D A T A B R I C K S Microsoft Azure
Optimized Databricks Runtime Engine DATABRICKS I/O SERVERLESS Collaborative Workspace Cloud storage Data warehouses Hadoop storage IoT / streaming data Rest APIs Machine learning models BI tools Data exports Data warehouses Azure Databricks Enhance Productivity Deploy Production Jobs & Workflows APACHE SPARK MULTI-STAGE PIPELINES DATA ENGINEER JOB SCHEDULER NOTIFICATION & LOGS DATA SCIENTIST BUSINESS ANALYST Build on secure & trusted cloud Scale without limits A Z U R E D A T A B R I C K S
A developer's introduction to big data processing with Azure Databricks
Azure Databricks Logical Architectures
INGEST STORE PREP & TRAIN MODEL & SERVE Azure Blob Storage Logs, files and media (unstructured) Azure SQL Data Warehouse Azure Data Factory Azure Analysis Services Azure Databricks (Python, Scala, Spark SQL) Polybase Business/custom apps (Structured) Power BI Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.
INGEST STORE PREP & TRAIN MODEL & SERVE Azure Blob Storage Logs, files and media (unstructured) Azure SQL Data Warehouse Azure Data Factory Azure Analysis Services Polybase Business/custom apps (Structured) Power BI Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs. Azure Databricks (Python, Scala, Spark SQL) Azure Databricks (Spark ML, Spark R, SparklyR) Intelligent Apps Cosmos DB
INGEST STORE PREP & TRAIN MODEL & SERVE Logs, files and media (unstructured) Sensors and IoT (unstructured) HDInsight (Kafka) Power BIAzure Databricks (Python, Scala, Spark SQL) Intelligent Apps Cosmos DBEvent Hub IoT Hub Azure Databricks (Spark ML, Spark R, SparklyR) Azure Blob Storage Batch Data (Apps, logs) Azure Data Factory
Azure Databricks at //BUILD 2018 Tuesday: • BRK3320 The Developer Data Scientist – Creating New Analytics Driven Applications using Apache Spark with Azure Databricks • WRK2601 Using Databricks to Analyze Telemetry Data Stored in Azure Blob Storage Wednesday: • BRK4102 ETL 2.0 - Data Engineering for developers • BRK3314 Leveraging Azure Databricks to minimize time to insight by combining Batch and Stream processing pipelines. • BRK3708 Machine learning at scale
A developer's introduction to big data processing with Azure Databricks
A developer's introduction to big data processing with Azure Databricks
A developer's introduction to big data processing with Azure Databricks

More Related Content

A developer's introduction to big data processing with Azure Databricks

  • 3. “More than any other factor, customer experiences determine whether companies thrive and profit, or struggle and fade.” – Forrester Research
  • 4. Speed 79% won’t return to a slow website Personalization 38% won't call again if they have to repeat themselves Consistency 65% get frustrated with inconsistent device experiences
  • 5. Harness the power of Big Data analytics apps to exceed customer needs
  • 6. Enhance any type of app with Big Data analytics
  • 8. Online shopping through company website E-commerce directly in the consumer’s hand, anywhere Wide World ImportersWide World Importers Retail stores
  • 9. Wide World Importers Web e-commerce Mobile e-commerce Wide World Importers seeks to expand customers through an omni-channel strategy Retail stores
  • 10. The solutions needed to reach more customers and grow the business 1. Scale with ease to reach more consumers 2. Unlock business insights from unstructured data 3. Enhance user experience with advanced analytics 4. Apply real-time analytics for instant updates 5. Infuse AI into apps to actively engage with customers
  • 11. A Z U R E D A T A B R I C K S Microsoft Azure
  • 12. Optimized Databricks Runtime Engine DATABRICKS I/O SERVERLESS Collaborative Workspace Cloud storage Data warehouses Hadoop storage IoT / streaming data Rest APIs Machine learning models BI tools Data exports Data warehouses Azure Databricks Enhance Productivity Deploy Production Jobs & Workflows APACHE SPARK MULTI-STAGE PIPELINES DATA ENGINEER JOB SCHEDULER NOTIFICATION & LOGS DATA SCIENTIST BUSINESS ANALYST Build on secure & trusted cloud Scale without limits A Z U R E D A T A B R I C K S
  • 15. INGEST STORE PREP & TRAIN MODEL & SERVE Azure Blob Storage Logs, files and media (unstructured) Azure SQL Data Warehouse Azure Data Factory Azure Analysis Services Azure Databricks (Python, Scala, Spark SQL) Polybase Business/custom apps (Structured) Power BI Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.
  • 16. INGEST STORE PREP & TRAIN MODEL & SERVE Azure Blob Storage Logs, files and media (unstructured) Azure SQL Data Warehouse Azure Data Factory Azure Analysis Services Polybase Business/custom apps (Structured) Power BI Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs. Azure Databricks (Python, Scala, Spark SQL) Azure Databricks (Spark ML, Spark R, SparklyR) Intelligent Apps Cosmos DB
  • 17. INGEST STORE PREP & TRAIN MODEL & SERVE Logs, files and media (unstructured) Sensors and IoT (unstructured) HDInsight (Kafka) Power BIAzure Databricks (Python, Scala, Spark SQL) Intelligent Apps Cosmos DBEvent Hub IoT Hub Azure Databricks (Spark ML, Spark R, SparklyR) Azure Blob Storage Batch Data (Apps, logs) Azure Data Factory
  • 18. Azure Databricks at //BUILD 2018 Tuesday: • BRK3320 The Developer Data Scientist – Creating New Analytics Driven Applications using Apache Spark with Azure Databricks • WRK2601 Using Databricks to Analyze Telemetry Data Stored in Azure Blob Storage Wednesday: • BRK4102 ETL 2.0 - Data Engineering for developers • BRK3314 Leveraging Azure Databricks to minimize time to insight by combining Batch and Stream processing pipelines. • BRK3708 Machine learning at scale

Editor's Notes

  1. 4
  2. Microsoft Connect 2016
  3. 12