Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
79 views

Azure SQL Trainings: Contact: +91 90 32 82 44 67

This document provides an overview of training courses related to Azure SQL and big data technologies. It lists topics that will be covered including introductions to Azure, storage, SQL, data factory, data lake, U-SQL, streaming, HD Insight, Databricks, Spark, database migration, Sqoop, Hive, MapReduce, Scala, and Spark. The training aims to help participants learn about these Azure and big data technologies through explanations and hands-on exercises.

Uploaded by

jeffa123
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
79 views

Azure SQL Trainings: Contact: +91 90 32 82 44 67

This document provides an overview of training courses related to Azure SQL and big data technologies. It lists topics that will be covered including introductions to Azure, storage, SQL, data factory, data lake, U-SQL, streaming, HD Insight, Databricks, Spark, database migration, Sqoop, Hive, MapReduce, Scala, and Spark. The training aims to help participants learn about these Azure and big data technologies through explanations and hands-on exercises.

Uploaded by

jeffa123
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 6

Azure SQL Trainings

Contact : +91 90 32 82 44 67

Introduction to Azure

1) Introduction to Azure Cloud


2) What is difference between Azure Cloud and On-Premises
3) What is Subscriptions and Resource Groups
4) Different offerings of Cloud IaaS, PaaS and SaaS
5) Creation of Virtual Machine

Introduction to Storage

1) Azure Storage
Azure Blob
Azure Table
Azure Message
Azure Queue
2) Azure Data Lake Store Gen (1 and 2)

Introduction to Azure SQL

1) Introduction to Azure SQL Database


2) Introduction to Azure SQL Data Warehouse
3) Installation of SQL Server 2016 and above in Virtual Machine
4) Creation of External Table or Polybase in On-Premise SQL Server
Creation of Master Key
Creation of Database Scoped Credential
Creation of External Data Source
Creation of External File Format
Creation of External Table
5) Creation of External Table or Polybase in Azure SQL Data Warehouse
Creation of Master Key
Creation of Database Scoped Credential
Creation of External Data Source
Creation of External File Format
Creation of External Table
6) Different Distribution or Shredding Patterns
ROUND ROBIN
HASH
REPLICATION
7) Cross Query Databases in Azure SQL Database
Creation of Master Key
Creation of Database Scoped Credential
Creation of External Data Source
Creation of External Table

8) Creation of Elastic Pools in Azure SQL Server between Databases

Azure Data Factory

1) Creation of Azure Data Factory


2) Creation of Linked Services
3) Creation of Datasets
4) Creation of Pipelines
5) Creation of Integration Runtime and different types
6) Copy Activity
7) Stored Procedure Activity
8) Lookup Activity
9) For Each Activity
9) Spark Activity
10) U-SQL Activity
11) Notebooks Activity
13) Web Activity
14) Data Flow Activity (with different transformations join, filter, exists, condition etc.)
15) Dynamic Queries in ADF
16) Sending mails through Logic Apps
17) Get Metadata Activity
18) If Condition Activity
19) Few more Activities ......

Azure Data Lake and Azure Data Lake Analytics

1) How to create App Registration


2) How to create ADL and ADLA
3) How to integrate Azure data lake analytics with ADF USQL Activity
4) How to create jobs and submit U-SQL Queries
5) How to integrate Azure data lake store with ADF

U-SQL Language

Introduction to U-SQL
U-SQL vs. SQL
Transforming Row Sets
Declare Parameters
Data Types
Expressions
Reading and Writing files
File sets
Grouping and Aggregation
U-SQL Catalog
Window functions
Set Operations
Joins
Complex Types
Extending U-SQL

Azure Streaming

Azure Event Hub


Azure Stream Analytics
Scheduling Jobs

Azure HD Insight -- Part 1

1) How to create Azure HDInsight


2) How to attach extra storage to HDInsight
3) How to SSH to the cluster and Use of Spark Activity
4) How to Monitor Job execution in Yarn
5) Introduction to Ambari, Hive and Jupyter

Azure Databricks

1) How to create Cluster


2) How to work with Databricks File System
3) How to create notebooks and Integrate with ADF
4) How to import and export the Notebooks
5) How to connect to blob, SQL DB from Databricks
6) How to read data files from Azure Blob and Azure Data Lake Store
Using Scala
Using R
Using Python
Using Spark SQL
7) Creating Data Frames
8) Converting Data Frames into Temporary Table or Temporary View
9) Incremental and Full Load with Azure SQL Data Warehouse

Azure Spark

1) Introduction to Apache Spark


2) Why Spark
3) Batch Vs Real Time Big Data Analytics
4) Disk and In-Memory Processing
5) Spark Execution Architecture
6) What is RDD
7) Different ways to create the RDDs
8) Transformation using Map, flatMap, Filter
9) Grouping using reduceByKey, groupByKey
10) Spark Actions
11) Data frame, different ways to create Data frame
12) Spark SQL with CSV
13) Spark SQL with Parquet
14) Spark SQL with JSON
15) Spark SQL With Database
16) Different ways of creating temp tables.

On-Premise Databases Migration


1) DMS -- Database Migration Service
2) On-Premise SQL Server to Azure Virtual Machine
3) On-Premise SQL Server to Azure SQL Server

Azure HD Insight -- Part 2

 Sqoop
 Oozie
 Hive
 Scala
 Spark
 Spark SQL

BIG DATA

Evolution of Data – Introduction to Big data – Classification - Size Hierarchy - Why Big data is Trending
(IOT, Devops, Cloud Computing, Enterprise Mobility) - Challenges in Big Data – Characteristics - Tools for
Big Data - Why Big Data draws attention in IT Industry - What do we do with Big data - How Big Data can
be analyzed - Typical Distributed System - Draw backs in Traditional distributed System

LINUX

History and Evolution - Architecture – Development Commands – Env Variables - File Management –
Directories Management – Admin Commands – Advanced Commands – Shell Scripting – Groups and
User managements – Permissions – Important directory structure – Disk utilities – Compression
Techniques – Misc Commands

HADOOP: HDFS (1 and 2)

What is Hadoop? - Evolution of Hadoop - Features of Hadoop - Characteristic of Hadoop - Hadoop


compared with Traditional Dist. Systems - When to use Hadoop - When not to use Hadoop -
Components of Hadoop (HDFS & MapReduce) - Hadoop Architecture - Daemons in Hadoop Version 1 &
2 -How Data is stored in Hadoop (Cluster, Datacenter, Spilt, Block, Rack Awareness, Replication, Hear
beat) - Hadoop 1.0 Limitation - NameNode High Availability - NameNode federation - How Metadata is
stored in Disk (FSImage & Editlog file) -Role of Secondary Name Node - Anatomy of File read & File Write
- Data Integrity - Serialization - Compression - What happens when copying data in Hadoop cluster? -
Centos Linux Commands Exercise - Hadoop Next Gen (ver 2) single node Pseudo mode Custer
installation - Hadoop commands Exercise

SQOOP – RDBMS

Introduction & History – History - Installation and configuration - Why Sqoop - Indepth Architecture -
Sqoop Import Properties - Sqoop Export Architecture - Commands (Import – HDSF, HIVE, HBase from
MySQL) - Export – Incremental Import - Saved Jobs - Import All tables - Sqoop installation and
configuration - Sqoop workouts - Sqoop best practices & performance tuning - Sqoop import/export use
cases - Mock test on Sqoop

HIVE – SQL & OLAP Layer on Hadoop

Introduction – Architecture - Hive Vs RDBMS - Detailed Installation (Metastore, Integrating with Hue)-
Starting Metastore and Hive Server - Data types (Primitive, Collection) - Create Tables (Managed,
external) and DML operations (load, insert, export) - Managed Vs External tables - QL Queries (select,
where, group by, having, sort by, order by) - Hive access through Hive Client, Beeline and Hue - File
Formats (RC, ORC, Sequence)- Partitioning (static and dynamic), partition with external table, dropping
partitions and corresponding configuration parameters - Bucketing, Partitioning Vs Bucketing - Views,
different types of joins (inner, outer) - Queries (Union, union all, intersection, minus) - Add files to the
distributed cache, jars to the class path - Optimized joins (MapSide join, Bucketing join) - Compressions
on tables (LZO, Snappy) - Serde (XML Serde, JsonSerde) - Parallel execution, Sampling data, Speculative
execution -Two POCs using the large dataset on the above topics -Mock Test on Hive and Its
Architecture

HADOOP – PROCESSING ARCHITECTURE

Hadoop Ecosystems ROAD MAP-MAP REDUCE FLOW-MapReduce Job submission in YARN Cluster in
details -What is MapReduce? - How MapReduce works on high level - Types of Input and Output Format
- MapReduce in details -Different types of files supported (Text, Sequence, map and Avro) - STORAGE &
PROCESSING DAEMONS Architecture Version 1 - PROCESSING DAEMONS Architecture Version 1 - Role of
Job Tracker and Task Tracker - Manager, Application Master, Node Manager), Architecture and Failure
handling – Schedulers - Resource Manager High availability -YARN Architecture

SCALA

Scala Introduction – History - Why Scala - Scala Installation - Get deep insights into the functioning of
Scala - Execute Pattern Matching in Scala - OOPs concepts (Classes, Objects, Collections, Inheritance,
Abstraction and Encapsulation) - Functional Programming in Scala (Closures, Currying, Expressions,
Anonymous Functions) - Know the concepts of classes in Scala - Object Orientation in Scala (Primary,
Auxiliary Constructors, Singleton Objects, Companion Objects) - Traits - Abstract classes
SPARK

Introduction – Scala/Python – History – Overview – MR vs Spark – Spark Libraries – Why Spark – RDDs –
Spark Internals – Transformations – Actions – DAG – Fault Tolerance – Lineage – Terminologies – Cluster
types – Hadoop Integration – Spark SQL – Data frames – DataSets – Optimizers – AST – Session –
Structured Streaming– RDDs to Relations – Spark Streaming – Why Spark Streaming– Data masking
techniques – SCD implementation - Real time use cases – End to end realtime integration with NIFI,
Kafka, Spark Streaming, EC2, Cassandra, RDBMS, Different Filesystems, Hive, Oozie & HBase

You might also like