Azure data platform overview

Azure Data Overview
…dove metto i miei dati?...

Agenda
Azure SQL
Database
Azure
DocumentDB
Azure Table
Storage
Azure Redis Cache Azure Search
…almeno fino a ieri

Agenda (post //build/)
Azure SQL
Database
Azure CosmosDB
(DocumentDB, MongoDB, Graph, Key-value)
Azure Table
Storage
Azure Redis Cache Azure Search
Azure Database
for MySQL
Azure Database
for PostgreSQL

Azure SQL Database
Azure SQL Database is a relational database-as-a-service (DBaaS)
hosted in the Azure cloud that falls into the industry categories
of Software-as-a-Service (SaaS) and Platform-as-a-Service (PaaS).
https://docs.microsoft.com/en-us/azure/sql-database/sql-database-paas-vs-sql-server-iaas

Azure SQL Database
Azure SQL Database is optimized to reduce overall costs to the
minimum for provisioning and managing many databases. It reduces
ongoing administration costs because you do not have to manage any
virtual machines, operating system or database software.

Azure SQL Database
You do not have to manage:

Azure SQL Database
upgrades

Azure SQL Database
high availability

Azure SQL Database
backups

Service tiers
• Basic: Best suited for a small database, supporting typically one single active operation at a
given time. Examples include databases used for development or testing, or small-scale
infrequently used applications.
• Standard: The go-to option for cloud applications with low to medium IO performance
requirements, supporting multiple concurrent queries. Examples include workgroup or web
applications.
• Premium: Designed for high transactional volume with high IO performance requirements,
supporting many concurrent users. Examples are databases supporting mission critical
applications.
• Premium RS: Designed for IO-intensive workloads that do not require the highest availability
guarantees. Examples include testing high-performance workloads, or an analytical workload
where the database is not the system of record.

Service tiers
• After initially picking a service tier and performance level, you can
scale a single database up or down dynamically
• Changing the service tier and/or performance level of a database
creates a replica of the original database at the new performance
level, and then switches connections over to the replica.
• No data is lost during this process but during the brief moment when
we switch over to the replica, connections to the database are
disabled, so some transactions in flight may be rolled back.

Create Azure Sql Database
• Azure Portal
• Azure CLI
• Powershell

Elastic pools
• SQL Database elastic pools are a simple, cost-effective solution for
managing and scaling multiple databases that have varying and
unpredictable usage demands
• Within the pool, individual databases are given the flexibility to auto-
scale within set parameters. Under heavy load, a database can
consume more eDTUs to meet demand. Databases under light loads
consume less, and databases under no load consume no eDTUs.

Business continuity
• Backup
• Active Geo-Replication

Business continuity: backup
• SQL Database automatically performs a combination of full database
backups weekly, differential database backups hourly, and transaction
log backups every five minutes to protect your business from data
loss. These backups are stored in geo-redundant storage
• Use automated backups as your business continuity and recovery
mechanism if your application:
• Is not considered mission critical
• Has a low rate of data change (low transactions per hour) and losing up to an
hour of change is an acceptable data loss
• Is cost sensitive

Business continuity: active geo-replication
• Is possible to use Active Geo-Replication to configure a database to
have up to four readable secondary databases in the regions of your
choice
• These secondary databases are kept synchronized with the primary
database using an asynchronous replication mechanism
• Active Geo-Replication can also be used to provide better query
performance for read-only queries to geographically dispersed users.

Monitoring
• Metrics
• SQL Database advisor
• Query Performance Insight

Azure Database Migration Service
• Using database migration service simplifies the migration of existing
on-premises SQL Server, Oracle, and MySQL databases to Azure,
whether your target database is Azure SQL Database, Azure SQL
Database Managed Instance or Microsoft SQL Server in an Azure
virtual machine.

Azure database for PostgreSQL/MySQL
• Azure Database for PostgreSQL/MySQL is a relational database service
in the Microsoft cloud built for developers based on the community
version of open source PostgreSQL and MySQL Community
Edition database engine. It provides:
• Predictable performance at multiple service levels
• Dynamic scalability with no application downtime
• Built-in high availability
• Data protection

Azure Table Storage
Azure Table storage is a service that stores structured NoSQL data in
the cloud, providing a key/attribute store with a schemaless design
https://docs.microsoft.com/en-us/azure/storage/storage-dotnet-how-to-use-tables

Table service concepts
• Storage Account
• Table: a table is a collection of entities
• Entitiy: An entity is a set of properties (similar
to a database row). An entity can be up to 1MB
in size
• Properties: A property is a name-value pair.
Each entity can include up to 252 properties to
store data

Table entities
• Table entities represent the units of data stored in a table and are
similar to rows in a typical relational database table
• Each entity defines a collection of properties
• Entities must define the following three system properties as part of
the property collection:
• PartitionKey
• RowKey
• Timestamp

Azure Storage: paper
http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11-calder.pdf

Azure CosmosDB
Azure Cosmos DB is Microsoft's globally distributed,
multi-model database
https://docs.microsoft.com/en-us/azure/cosmos-db/introduction

Global distribution
• Within a given region, all resources
are horizontally partitioned using
resource partitions (local
distribution).
• Each resource partition is also
replicated across geographical regions
(global distribution).

Multi-model, multi API-support
• The core content-model of Cosmos DB’s
database engine is based on atom-
record-sequence (ARS)
• Atoms consist of a small set of primitive
types like string, bool, and number. Records
are structs composed of these types.
Sequences are arrays consisting of atoms,
records, or sequences.
• The database engine can efficiently
translate and project different data
models onto the ARS-based data model.

Multiple consistency models
• It’s possible to configure the default consistency level on Cosmos DB
account, and later override the consistency on a specific read request

Guaranteed SLA
• Availability: 99.99% uptime availability SLA for each of the data and control plane
operations.
• Throughput: 99.99% of requests complete successfully
• Latency: 99.99% of <10 ms latencies at the 99th percentile
• Consistency: 100% of read requests will meet the consistency guarantee for the consistency
level requested by you.

CosmosDB: indexing
• CosmosDB indexes, by default, each document’s attribute
• Automatic indexing of documents without requiring schema or
secondary indexes is a key capability of CosmosDB and is enabled by
write-optimized, lock-free and log-structured index maintenance
techniques

CosmosDB: indexing policy
• The indexing policy of each collection allows you to make
performance and storage trade-offs associated with indexing
• Choose whether the collection automatically indexes all of the documents or
not
• Choose whether to include or exclude specific paths or patterns in your
documents from the index
• Choose between synchronous (consistent) and asynchronous (lazy) index
updates
• The indexing policy can be changed

Azure Redis Cache
• Redis is an advanced key-value store, where keys can contain data
structures such as strings, hashes, lists, sets, and sorted sets. Redis
supports a set of atomic operations on these data types.
• Azure Redis Cache uses Redis authentication and also supports SSL
connections to Redis.

Service tiers
• Basic: Single node, multiple sizes, ideal for development/test and non-critical workloads. The
Basic tier has no SLA.
• Standard: A replicated cache in a two-node primary/secondary configuration managed by
Microsoft, with a high-availability SLA.
• Premium: All of the Standard tier features, plus:
• Redis data persistence: allows you to persist the cache data in an Azure Storage account
• Redis cluster
• Enhanced security and isolation
• Reboot
• Schedule updates

Azure Search
Azure Search is a cloud search-as-a-service solution that delegates
server and infrastructure management to Microsoft, leaving you with a
ready-to-use service that you can populate with your data and then use
to add search to your web or mobile application.
https://docs.microsoft.com/en-us/azure/search/search-what-is-azure-search

Azure Search
• Full text search and text analysis
• Search suggestions
• Facet queries
• Scoring
• …
• Language support
• Data integration

Service tiers
• Free: A shared service, at no charge, used for evaluation, investigation, or small workloads.
• Basic: Small production workloads on dedicated hardware. Highly available.
• Standard:
• S1
• S2
• S3
• S3 HD

Indexers
• An indexer in Azure Search is a crawler that extracts searchable data
and metadata from an external data source and populates an index
based on field-to-field mappings between the index and your data
source.
• You can use an indexer as the sole means for data ingestion, or use a
combination of techniques
• You can run indexers on demand or on a recurring data refresh
schedule

Indexer types
• Azure Blob Storage indexer
• Azure Table Storage indexer
• Azure SQL indexer
• Azure CosmosDB indexer

SQL IaaS vs SQL Database
• Existing applications that
require fast migration to the
cloud with minimal changes.
• SQL Server instances with up to
64 TB of storage. The instance
can support as many databases
as needed.
• Migrating and building
enterprise and hybrid
applications.
• New cloud-designed
applications that have time
constraints in development and
marketing.
• Databases of up to 1 TB, or
larger databases that can
be horizontally or vertically
partitioned using a scale-out
pattern.
• Building Software-as-a-Service
(SaaS) applications.

SQL IaaS vs SQL Database: features
• https://docs.microsoft.com/en-us/azure/sql-database/sql-database-
features

Transient fault handling
Transient faults can occur in any environment, on any platform or
operating system, and in any kind of application

Transient fault handling: cloud
• Many resources in a cloud environment are shared, and access to
these resources is subject to throttling in order to protect the
resource
• Cloud environments are built using vast numbers of commodity
hardware units
• There are often more hardware components, including network
infrastructure such as routers and load balancers, between the
application and the resources and services it uses
• Network conditions between the client and the server may be
variable

Transient fault handling: challanges
• The application must be able to detect faults when they occur, and
determine if these faults are likely to be transient, more long-lasting,
or are terminal failures
• The application must be able to retry the operation if it determines
that the fault is likely to be transient
• The application must use an appropriate strategy for the retries:
• Exponential back-off
• Incremental intervals
• Regular intervals
• Immediate retry
• Randomization

About me
Alessandro Melchiori
@amelchiori
http://melkio.codiceplastico.com

Azure data platform overview

More Related Content

Azure data platform overview