0% found this document useful (0 votes)

3K views

SQL Database Azure

Uploaded by

Gopal Arunachalam

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

3K views

SQL Database Azure

Uploaded by

Gopal Arunachalam

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 800

SQL DB Documentation
Overview
About SQL DB
Quickstart
Create DB - Portal
Create DB - Azure CLI
Create DB - PowerShell
Manage
SSMS
VS Code
Connect
.NET
PHP
Node.js
Java
Python
Ruby
Tutorials
Design a database
Migrate a database
Samples
Azure CLI
PowerShell
Concepts
DBs and servers
Databases
Servers
Elastic pools
Resources
Service tiers
DTUs and eDTUs
DTU benchmark
Limits
PaaS or IaaS
Features
Tools
Partition data
Sharded databases
Elastic client library
Shard maps
Query routing
Manage credentials
Shard querying
Elastic tools
Move sharded data
Elastic tools FAQ
Manage multiple DBs
Elastic queries
Horizontal data
Vertical data
Transactions
Elastic jobs
Security
Overview
Access control
Firewall rules
Azure AD
Logins and users
Multi-factor auth
Auditing
Auditing and TDS
Threat detection
TDE with SQL DB
Always encrypted
Data masking
Business continuity
Overview
Database backups
Backup retention
Database recovery
Geo-replication
Logins
App design
Elastic pools
App upgrades
Development
Overview
Connectivity
App design - SaaS
Security - SaaS
Ports - ADO.NET
Authenticate App
Error messages
Dapper
JSON data
In-memory
Temporal tables
Retention policies
Database migration
SQL Server DB
T-SQL changes
Data movement
Copy a DB
Import a DB
Export a DB
Monitor and tune
Single databases
Database advisor
Query insight
Operate query store
DMVs
Perf guidance
Batching
Extended events
Compatibility levels
How-to guides
Manage elastic pools
Portal
PowerShell
Transact-SQL
C
DB Access
SQL Server
Azure AD
Secure data
Azure AD auth
Encrypt - cert store
Encrypt - key vault
Configure masking
Recover single table
Configure vault for backups
Geo-replicate data
Portal
T-SQL - Configure
T-SQL - Failover
Recover - outage
Perform drills
Move data
Load data with BCP
Load data with ADF
Sync data
Connect applications
C and C ++
Excel
Guidance
Issues
Create DB with C
Configure In-Memory
Manage multiple DBs
Create sharded app
Move sharded data
Security configuration
Add a shard
Fix shard map problems
Migrate sharded DB
Create counters
Use entity framework
Upgrade clients
Install elastic job
Create job - Portal
Create job - PowerShell
Create job - Sample app
Uninstall elastic job
Query horizontal data
Query vertical data
Monitor and tune
Use Database Advisor
Use QPI
Evaluate and tune
Create alerts
Monitor in-memory
Extended events - event file
Extended events - ring buffer
Manage
Azure Automation
Azure RemoteApp
SSMS - MFA
Reference
PowerShell
PowerShell (Elastic DB)
Azure CLI 2.0
.NET
Java
Node.js
Python
Ruby
PHP
T-SQL
REST
SQL Server tools
SQL Server Management Studio (SSMS)
SQL Server Data Tools (SSDT)
BCP
SQLCMD
SqlPackage
SQL Database Management Library package
SQL Server drivers
ADO.NET
JDBC
ODBC
Resources
FAQ
Public data sets
Troubleshoot connectivity
Pricing
MSDN forum
Stack Overflow
Videos
Service updates
Architecture guidance
Customer implementations
Sample applications
Tailspin Surveys
Contoso Clinic
Elastic pool telemetry
Elastic pool custom dashboard
SQL Database Documentation
Azure SQL Database is a relational database-as-a service using the Microsoft SQL Server Engine. SQL Database is a high-
performance, reliable, and secure database you can use to build data-driven applications and websites in the programming
language of your choice, without needing to manage infrastructure. Learn how to use SQL Database with our quickstarts, tutorials,
and samples.

Quickstarts
Create a SQL DB using:
Azure Portal
Azure CLI
Azure PowerShell

Tutorials
Build and deploy SQL Database applications.
Design a database
Migrate a database

Samples
Find scripts to manage common tasks.
Azure CLI
Azure PowerShell

Free Video Training

Free Pluralsight video training – Developing with .NET

Reference
Command- Line
Azure PowerShell
Azure PowerShell (Elastic DB)
Azure CLI 2.0

Languages
.NET
Java
Node.js
Python
Ruby
PHP
T-SQL

REST
REST API Reference

SQL Server Tools

SQL Server Management Studio (SSMS)
SQL Server Data Tools (SSDT)
BCP
SQLCMD
SqlPackage
SQL Database Management Library package

SQL Server Drivers

ADO.NET
JDBC
ODBC
What is SQL Database? Introduction to SQL
Database
4/14/2017 • 5 min to read • Edit Online

SQL Database is a relational database service in the Microsoft cloud based on the market-leading Microsoft SQL
Server engine and capable of handling mission-critical workloads. SQL Database delivers predictable performance
at multiple service levels, dynamic scalability with no downtime, built-in business continuity, and data protection
— all with near-zero administration. These capabilities allow you to focus on rapid app development and
accelerating your time to market, rather than allocating precious time and resources to managing virtual machines
and infrastructure. Because SQL Database is based on the SQL Server engine, SQL Database supports existing SQL
Server tools, libraries, and APIs. As a result, it is easy for you to develop new solutions, to move your existing SQL
Server solutions, and to extend your existing SQL Server solutions to the Microsoft cloud without having to learn
new skills.
This article is an introduction to SQL Database core concepts and features related to performance, scalability, and
manageability, with links to explore details. See these quick starts to get you started:
Create a SQL database in the Azure portal
Create a SQL database with the Azure CLI
Create a SQL database using PowerShell
For a set of Azure CLI and PowerShell samples, see:
Azure CLI samples for Azure SQL Database
Azure PowerShell samples for Azure SQL Database

Adjust performance and scale without downtime

The SQL Database service offers three service tiers: Basic, Standard, Premium, and Premium RS. Each service tier
offers different levels of performance and capabilities to support lightweight to heavyweight database workloads.
You can build your first app on a small database for a few bucks a month and then change its service tier manually
or programmatically at any time to meet the needs of your solution. You can do this without downtime to your
app or to your customers. Dynamic scalability enables your database to transparently respond to rapidly changing
resource requirements and enables you to only pay for the resources that you need when you need them.

Elastic pools to maximize resource utilization

For many businesses and apps, being able to create single databases and dial performance up or down on
demand is enough, especially if usage patterns are relatively predictable. But if you have unpredictable usage
patterns, it can make it hard to manage costs and your business model. Elastic pools are designed to solve this
problem. The concept is simple. You allocate performance resources to a pool rather than an individual database,
and pay for the collective performance resources of the pool rather than for single database performance. With
elastic pools, you don’t need to focus on dialing database performance up and down as demand for resources
fluctuates. The pooled databases consume the performance resources of the elastic pool as needed. Pooled
databases consume but don’t exceed the limits of the pool, so your cost remains predictable even if individual
database usage doesn’t. What’s more, you can add and remove databases to the pool, scaling your app from a
handful of databases to thousands, all within a budget that you control. Finally, you can also control the minimum
and maximum resources available to databases in the pool to ensure that no database in the pool uses all the pool
resource and that every pooled database has a guaranteed minimum amount of resources. To learn more about
design patterns for SaaS applications using elastic pools, see Design Patterns for Multi-tenant SaaS Applications
with Azure SQL Database.

Blend single databases with pooled databases

Either way you go — single databases or elastic pools — you are not locked in. You can blend single databases
with elastic pools, and change the service tiers of single databases and elastic pools quickly and easily to adapt to
your situation. Moreover, with the power and reach of Azure, you can mix-and-match other Azure services with
SQL Database to meet your unique app design needs, drive cost and resource efficiencies, and unlock new
business opportunities.

Monitoring and alerting

But how can you compare the relative performance of single databases and elastic pools? How do you know the
right click-stop when you dial up and down? You use the built-in performance monitoring and alerting tools,
combined with the performance ratings based on Database Transaction Units (DTUs) for single databases and
elastic DTUs (eDTUs) for elastic pools. Using these tools, you can quickly assess the impact of scaling up or down
based on your current or project performance needs. See SQL Database options and performance: Understand
what's available in each service tier for details.

Keep your app and business running

Azure's industry leading 99.99% availability service level agreement (SLA), powered by a global network of
Microsoft-managed datacenters, helps keep your app running 24/7. With every SQL database, you take advantage
of built-in security, fault tolerance, and data protection that you would otherwise have to buy or design, build, and
manage. With SQL Database, each service tier offers a comprehensive set of business continuity features and
options that you can use to get up and running and stay that way. You can use point-in-time restore to return a
database to an earlier state, as far back as 35 days. You can configure long-term backup retention to store backups
in a secure vault for up to 10 years. In addition, if the datacenter hosting your databases experiences an outage,
you can restore databases from geo-redundant copies of recent backups. If needed, you can also configure geo-
redundant readable replicas in one or more regions for rapid failover in the event of a data center outage. You can
also use these replicas for faster read performance in different geographic regions or for application upgrades
without downtime.

See Business Continuity for details about the different business continuity features available for different service
tiers.

Secure your data

SQL Server has a tradition of data security that SQL Database upholds with features that limit access, protect data,
and help you monitor activity. See Securing your SQL database for a quick rundown of security options you have
in SQL Database. See the Security Center for SQL Server Database Engine and SQL Database for a more
comprehensive view of security features. And visit the Azure Trust Center for information about Azure's platform
security.

Next steps
Now that you've read an introduction to SQL Database and answered the question "What is SQL Database?",
you're ready to:
See the pricing page for single database and elastic pools cost comparisons and calculators.
Learn about elastic pools.
Get started by creating your first database.
Build your first app in C#, Java, Node.js, PHP, Python, or Ruby: Connection libraries for SQL Database and SQL
Server
Create an Azure SQL database in the Azure portal
4/17/2017 • 5 min to read • Edit Online

This quick start tutorial walks through how to create a SQL database in Azure. Azure SQL Database is a
“Database-as-a-Service” offering that enables you to run and scale highly available SQL Server databases in the
cloud. This quick start shows you how to get started by creating a SQL database using the Azure portal.
If you don't have an Azure subscription, create a free account before you begin.

Log in to the Azure portal

Create a SQL database

An Azure SQL database is created with a defined set of compute and storage resources. The database is created
within an Azure resource group and in an Azure SQL Database logical server.
Follow these steps to create a SQL database containing the Adventure Works LT sample data.
1. Click the New button found on the upper left-hand corner of the Azure portal.
2. Select Databases from the New page, and select SQL Database from the Databases page.

3. Fill out the SQL Database form with the following information, as shown on the preceding image:
Database name: mySampleDatabase
Resource group: myResourceGroup
Source source: Sample (AdventureWorksLT)
IMPORTANT
You must select the sample database on this form because it is used in the remainder of this quick start.

4. Click Server and then fill out the New server form specifying a globally unique server name, provide a
name for the server admin login, and then specify the password of your choice.

IMPORTANT
The server admin login and password that you specify here are required to log in to the server and its databases
later in this quick start. Remember or record this information for later use.

5. When you have completed the form, click Select.

6. Click Pricing tier to specify the service tier and performance level for your new database. Use the slider
to select 20 DTUs and 250 GB of storage. For more information on DTUs, see What is a DTU?.
7. After selected the amount of DTUs, click Apply.
8. Now that you have completed the SQL Database form, click Create to provision the database.
Provisioning takes a few minutes.
9. On the toolbar, click Notifications to monitor the deployment process.

Create a server-level firewall rule

NOTE
SQL Database communicates over port 1433. If you are trying to connect from within a corporate network, outbound
traffic over port 1433 may not be allowed by your network's firewall. If so, you will not be able to connect to your Azure
SQL Database server unless your IT department opens port 1433.

1. After the deployment completes, click SQL databases from the left-hand menu and then click
mySampleDatabase on the SQL databases page. The overview page for your database opens, showing
you the fully qualified server name (such as mynewserver20170411.database.windows.net) and
provides options for further configuration.
IMPORTANT
You will need this fully qualified server name to connect to your server and its databases in subsequent quick
starts.

2. Click Set server firewall on the toolbar as shown in the previous image. The Firewall settings page for
the SQL Database server opens.

3. Click Add client IP on the toolbar to add your current IP address to a new firewall rule. A firewall rule
can open port 1433 for a single IP address or a range of IP addresses.
4. Click Save. A server-level firewall rule is created for your current IP address opening port 1433 on the
logical server.

5. Click OK and then close the Firewall settings page.

You can now connect to the SQL Database server and its databases using SQL Server Management Studio or
another tool of your choice from this IP address using the server admin account created previously.

IMPORTANT
By default, access through the SQL Database firewall is enabled for all Azure services. Click OFF on this page to disable for
all Azure services.

Query the SQL database

Now that you have created a sample database in Azure, let’s use the built-in query tool within the Azure portal
to confirm that you can connect to the database and query the data.
1. On the SQL Database page for your database, click Tools on the toolbar. The Tools page opens.

2. Click Query editor (preview), click the Preview terms checkbox, and then click OK. The Query editor
page opens.
3. Click Login and then, when prompted, select SQL server authentication and then provide the server
admin login and password that you created earlier.

4. Click OK to log in.

5. After you are authenticated, type the following query in the query editor pane.

SELECT TOP 20 pc.Name as CategoryName, p.name as ProductName

FROM SalesLT.ProductCategory pc
JOIN SalesLT.Product p
ON pc.productcategoryid = p.productcategoryid;

6. Click Run and then review the query results in the Results pane.

7. Close the Query editor page and the Tools page.

Clean up resources
Other quick starts in this collection build upon this quick start. If you plan to continue on to work with
subsequent quick starts, do not clean up the resources created in this quick start. If you do not plan to continue,
use the following steps to delete all resources created by this quick start in the Azure portal.
1. From the left-hand menu in the Azure portal, click Resource groups and then click myResourceGroup.
2. On your resource group page, click Delete, type myResourceGroup in the text box, and then click Delete.

Next steps
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio Code, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Create a single Azure SQL database using the Azure
CLI
4/18/2017 • 3 min to read • Edit Online

The Azure CLI is used to create and manage Azure resources from the command line or in scripts. This guide
details using the Azure CLI to deploy an Azure SQL database in an Azure resource group in an Azure SQL
Database logical server.
To complete this quick start, make sure you have installed the latest Azure CLI 2.0.
If you don't have an Azure subscription, create a free account before you begin.

az login

Define variables
Define variables for use in the scripts in this quick start.

# The data center and resource name for your resources

resourcegroupname = myResourceGroup
location = westeurope
# The logical server name: Use a random value or replace with your own value (do not capitalize)
servername = server-$RANDOM
# Set an admin login and password for your database
adminlogin = ServerAdmin
password = ChangeYourAdminPassword1
# The ip address range that you want to allow to access your DB
startip = "0.0.0.0"
endip = "0.0.0.1"
# The database name
databasename = mySampleDatabase

Create a resource group

Create an Azure resource group using the az group create command. A resource group is a logical container into
which Azure resources are deployed and managed as a group. The following example creates a resource group
named myResourceGroup in the westeurope location.

az group create --name $resourcegroupname --location $location

Create a logical server

Create an Azure SQL Database logical server using the az sql server create command. A logical server contains a
group of databases managed as a group. The following example creates a randomly named server in your
resource group with an admin login named ServerAdmin and a password of ChangeYourAdminPassword1 . Replace
these pre-defined values as desired.

az sql server create --name $servername --resource-group $resourcegroupname --location $location \

--admin-user $adminlogin --admin-password $password

Configure a server firewall rule

Create an Azure SQL Database server-level firewall rule using the az sql server firewall create command. A server-
level firewall rule allows an external application, such as SQL Server Management Studio or the SQLCMD utility to
connect to a SQL database through the SQL Database service firewall. In the following example, the firewall is only
opened for other Azure resources. To enable external connectivity, change the IP address to an appropriate
address for your environment. To open all IP addresses, use 0.0.0.0 as the starting IP address and
255.255.255.255 as the ending address.

az sql server firewall-rule create --resource-group $resourcegroupname --server $servername \

-n AllowYourIp --start-ip-address $startip --end-ip-address $endip

NOTE
SQL Database communicates over port 1433. If you are trying to connect from within a corporate network, outbound traffic
over port 1433 may not be allowed by your network's firewall. If so, you will not be able to connect to your Azure SQL
Database server unless your IT department opens port 1433.

Create a database in the server with sample data

Create a database with an S0 performance level in the server using the az sql db create command. The following
example creates a database called mySampleDatabase and loads the AdventureWorksLT sample data into this
database. Replace these predefined values as desired (other quick starts in this collection build upon the values in
this quick start).

az sql db create --resource-group $resourcegroupname --server $servername \

--name $databasename --sample-name AdventureWorksLT --service-objective S0

Clean up resources
Other quick starts in this collection build upon this quick start. If you plan to continue on to work with subsequent
quick starts or with the tutorials, do not clean up the resources created in this quick start. If you do not plan to
continue, use the following command to delete all resources created by this quick start.

az group delete --name $resourcegroupname

Next steps
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio Code, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Create a single Azure SQL database using PowerShell
4/18/2017 • 3 min to read • Edit Online

PowerShell is used to create and manage Azure resources from the command line or in scripts. This guide details
using PowerShell to deploy an Azure SQL database in an Azure resource group in an Azure SQL Database logical
server.
To complete this tutorial, make sure you have installed the latest Azure PowerShell.
If you don't have an Azure subscription, create a free account before you begin.

Log in to Azure
Log in to your Azure subscription using the Add-AzureRmAccount command and follow the on-screen directions.

Add-AzureRmAccount

Create variables
Define variables for use in the scripts in this quick start.

# The data center and resource name for your resources

$resourcegroupname = "myResourceGroup"
$location = "WestEurope"
# The logical server name: Use a random value or replace with your own value (do not capitalize)
$servername = "server-$(Get-Random)"
# Set an admin login and password for your database
# The login information for the server
$adminlogin = "ServerAdmin"
$password = "ChangeYourAdminPassword1"
# The ip address range that you want to allow to access your server - change as appropriate
$startip = "0.0.0.0"
$endip = "0.0.0.1"
# The database name
$databasename = "mySampleDatabase"

Create a resource group

Create an Azure resource group using the New-AzureRmResourceGroup command. A resource group is a logical
container into which Azure resources are deployed and managed as a group. The following example creates a
resource group named myResourceGroup in the westeurope location.

New-AzureRmResourceGroup -Name $resourcegroupname -Location $location

Create a logical server

Create an Azure SQL Database logical server using the New-AzureRmSqlServer command. A logical server
contains a group of databases managed as a group. The following example creates a randomly named server in
your resource group with an admin login named ServerAdmin and a password of ChangeYourAdminPassword1 . Replace
these pre-defined values as desired.
New-AzureRmSqlServer -ResourceGroupName $resourcegroupname `
-ServerName $servername `
-Location $location `
-SqlAdministratorCredentials $(New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $adminlogin,
$(ConvertTo-SecureString -String $password -AsPlainText -Force))

Configure a server firewall rule

Create an Azure SQL Database server-level firewall rule using the New-AzureRmSqlServerFirewallRule command.
A server-level firewall rule allows an external application, such as SQL Server Management Studio or the SQLCMD
utility to connect to a SQL database through the SQL Database service firewall. In the following example, the
firewall is only opened for other Azure resources. To enable external connectivity, change the IP address to an
appropriate address for your environment. To open all IP addresses, use 0.0.0.0 as the starting IP address and
255.255.255.255 as the ending address.

New-AzureRmSqlServerFirewallRule -ResourceGroupName $resourcegroupname `

-ServerName $servername `
-FirewallRuleName "AllowSome" -StartIpAddress $startip -EndIpAddress $endip

NOTE
SQL Database communicates over port 1433. If you are trying to connect from within a corporate network, outbound traffic
over port 1433 may not be allowed by your network's firewall. If so, you will not be able to connect to your Azure SQL
Database server unless your IT department opens port 1433.

Create a blank database

Create a blank SQL database with an S0 performance level in the server using the New-AzureRmSqlDatabase
command. The following example creates a database called mySampleDatabase . Replace this predefined value as
desired.

New-AzureRmSqlDatabase -ResourceGroupName $resourcegroupname `

-ServerName $servername `
-DatabaseName databasename `
-RequestedServiceObjectiveName "S0"

Remove-AzureRmResourceGroup -ResourceGroupName $resourcegroupname

Next steps
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio Code, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use SQL Server Management
Studio to connect and query data
4/19/2017 • 3 min to read • Edit Online

SQL Server Management Studio (SSMS) is a management tool used to create and manage SQL Server resources
from the user interface or in scripts. This quick start demonstrates how to use SSMS to connect to an Azure SQL
database, and then use Transact-SQL statements to query, insert, update, and delete data in the database.
This quick start uses as its starting point the resources created in one of these quick starts:
Create DB - Portal
Create DB - CLI
Before you start, make sure you have installed the newest version of SSMS.

Get connection information

4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.

Connect to your database in the SQL Database logical server

Use SQL Server Management Studio to establish a connection to your Azure SQL Database server.

IMPORTANT
An Azure SQL Database logical server listens on port 1433. If you are attempting to connect to an Azure SQL Database
logical server from within a corporate firewall, this port must be open in the corporate firewall for you to successfully
connect.

1. Open SQL Server Management Studio.

2. In the Connect to Server dialog box, enter the following information:
Server type: Specify Database engine
Server name: Enter your fully qualified server name, such as
mynewserver20170313.database.windows.net
Authentication: Specify SQL Server Authentication
Login: Enter your server admin account
Password: Enter the password for your server admin account

3. Click Options in the Connect to server dialog box. In the Connect to database section, enter
mySampleDatabase to connect to this database.

4. Click Connect. The Object Explorer window opens in SSMS.

5. In Object Explorer, expand Databases and then expand mySampleDatabase to view the objects in the
sample database.

Query data
Use the following code to query for the top 20 products by category using the SELECT Transact-SQL statement.
1. In Object Explorer, right-click mySampleDatabase and click New Query. A blank query window opens that
is connected to your database.
2. In the query window, enter the following query:

SELECT pc.Name as CategoryName, p.name as ProductName

FROM [SalesLT].[ProductCategory] pc
JOIN [SalesLT].[Product] p
ON pc.productcategoryid = p.productcategoryid;

3. On the toolbar, click Execute to retrieve data from the Product and ProductCategory tables.
Insert data
Use the following code to insert a new product into the SalesLT.Product table using the INSERT Transact-SQL
statement.
1. In the query window, replace the previous query with the following query:

INSERT INTO [SalesLT].[Product]

( [Name]
, [ProductNumber]
, [Color]
, [ProductCategoryID]
, [StandardCost]
, [ListPrice]
, [SellStartDate]
)
VALUES
('myNewProduct'
,123456789
,'NewColor'
,1
,100
,100
,GETDATE() );

2. On the toolbar, click Execute to insert a new row in the Product table.

Update data
Use the following code to update the new product that you previously added using the UPDATE Transact-SQL
statement.
1. In the query window, replace the previous query with the following query:

UPDATE [SalesLT].[Product]
SET [ListPrice] = 125
WHERE Name = 'myNewProduct';

2. On the toolbar, click Execute to update the specified row in the Product table.
Delete data
Use the following code to delete the new product that you previously added using the DELETE Transact-SQL
statement.
1. In the query window, replace the previous query with the following query:

DELETE FROM [SalesLT].[Product]

WHERE Name = 'myNewProduct';

2. On the toolbar, click Execute to delete the specified row in the Product table.

Next steps
For information about SSMS, see Use SQL Server Management Studio.
To connect and query using Visual Studio Code, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use Visual Studio Code to
connect and query data
4/19/2017 • 4 min to read • Edit Online

Visual Studio Code is a graphical code editor for Linux, macOS, and Windows that supports extensions, including
the mssql extension for querying Microsoft SQL Server, Azure SQL Database, and SQL Data Warehouse. This
quick start demonstrates how to use Visual Studio Code to connect to an Azure SQL database, and then use
Transact-SQL statements to query, insert, update, and delete data in the database.
This quick start uses as its starting point the resources created in one of these quick starts:
Create DB - Portal
Create DB - CLI
Before you start, make sure you have installed the newest version of Visual Studio Code and loaded the mssql
extension. For installation guidance for the mssql extension, see Install VS Code and see mssql for Visual Studio
Code.

Configure VS Code
Mac OS
For macOS, you need to install OpenSSL which is a prerequiste for DotNet Core that mssql extention uses. Open
your terminal and enter the following commands to install brew and OpenSSL.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew update
brew install openssl
mkdir -p /usr/local/lib
ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/

Linux (Ubuntu)
No special configuration needed.
Windows
No special configuration needed.

Get connection information

Get the connection information needed to connect to the Azure SQL database. You will need the fully qualified
server name, database name, and login information in the next procedures.
1. Log in to the Azure portal.
2. Select SQL Databases from the left-hand menu, and click your database on the SQL databases page.
3. On the Overview page for your database, review the fully qualified server name as shown in the following
image. You can hover over the server name to bring up the Click to copy option.
4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.

Set language mode to SQL

Set the language mode is set to SQL in Visual Studio Code to enable mssql commands and T-SQL IntelliSense.
1. Open a new Visual Studio Code window.
2. Click Plain Text in the lower right-hand corner of the status bar.
3. In the Select language mode drop-down menu that opens, type SQL, and then press ENTER to set the
language mode to SQL.

Connect to your database in the SQL Database logical server

Use Visual Studio Code to establish a connection to your Azure SQL Database server.

IMPORTANT
Before continuing, make sure that you have your server, database, and login information ready. Once you begin entering
the connection profile information, if you change your focus from Visual Studio Code, you have to restart creating the
connection profile.

1. In VS Code, press CTRL+SHIFT+P (or F1) to open the Command Palette.

2. Type sqlcon and press ENTER.
3. Press ENTER to select Create Connection Profile. This creates a connection profile for your SQL Server
instance.
4. Follow the prompts to specify the connection properties for the new connection profile. After specifying
each value, press ENTER to continue.
The following table describes the Connection Profile properties.

SETTING DESCRIPTION

Server name Enter your fully qualified server name, such as

mynewserver20170313.database.windows.net

Database name Enter your database name, such as mySampleDatabase

Authentication Select SQL Login

User name Enter your server admin account

Password (SQL Login) Enter the password for your server admin account

Save Password? Select Yes or No

[Optional] Enter a name for this profile Enter a connection profile name, such as
mySampleDatabase.

5. Press the ESC key to close the info message that informs you that the profile is created and connected.
6. Verify your connection in the status bar.

Query data
Use the following code to query for the top 20 products by category using the SELECT Transact-SQL statement.
1. In the Editor window, enter the following query in the empty query window:

SELECT pc.Name as CategoryName, p.name as ProductName

FROM [SalesLT].[ProductCategory] pc
JOIN [SalesLT].[Product] p
ON pc.productcategoryid = p.productcategoryid;
2. Press CTRL+SHIFT+E to retrieve data from the Product and ProductCategory tables.

Insert data
Use the following code to insert a new product into the SalesLT.Product table using the INSERT Transact-SQL
statement.
1. In the Editor window, delete the previous query and enter the following query:

INSERT INTO [SalesLT].[Product]

( [Name]
, [ProductNumber]
, [Color]
, [ProductCategoryID]
, [StandardCost]
, [ListPrice]
, [SellStartDate]
)
VALUES
('myNewProduct'
,123456789
,'NewColor'
,1
,100
,100
,GETDATE() );

2. Press CTRL+SHIFT+E to insert a new row in the Product table.

Update data
Use the following code to update the new product that you previously added using the UPDATE Transact-SQL
statement.
1. In the Editor window, delete the previous query and enter the following query:

UPDATE [SalesLT].[Product]
SET [ListPrice] = 125
WHERE Name = 'myNewProduct';

2. Press CTRL+SHIFT+E to update the specified row in the Product table.

Delete data
Use the following code to delete the new product that you previously added using the DELETE Transact-SQL
statement.
1. In the Editor window, delete the previous query and enter the following query:

DELETE FROM [SalesLT].[Product]

WHERE Name = 'myNewProduct';

2. Press CTRL+SHIFT+E to delete the specified row in the Product table.

Next steps
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use .NET (C#) to connect and
query data
4/20/2017 • 5 min to read • Edit Online

This quick start demonstrates how to use C# and ADO.NET to connect to an Azure SQL database, and then use
Transact-SQL statements to query, insert, update, and delete data in the database from the Windows, Mac OS,
and Ubuntu Linux platforms.
This quick start uses as its starting point the resources created in one of these quick starts:
Create DB - Portal
Create DB - CLI

Install .NET
The steps in this section assume that you are familar with developing using .NET and are new to working with
Azure SQL Database. If you are new to developing with .NET, go the Build an app using SQL Server and select C#
and then select your operating system.
Windows .NET framework and .NET core
Visual Studio 2017 Community is a fully-featured, extensible, free IDE for creating modern applications for
Android, iOS, Windows, as well as web & database applications and cloud services. You can install either the full
.NET framework or just .NET core. The code snippets in the quick start work with either. If you already have Visual
Studio installed on your machine, skip the next few steps.
1. Download the installer.
2. Run the installer and follow the installation prompts to complete the installation.
Mac OS
Open your terminal and navigate to a directory where you plan on creating your .NET Core project. Enter the
following commands to install brew, OpenSSL, and .NET Core.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew update
brew install openssl
mkdir -p /usr/local/lib
ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/

Install .NET Core on macOS. Download the official installer. This installer will install the tools and put them on
your PATH so you can run dotnet from the Console
Linux (Ubuntu)
Open your terminal and navigate to a directory where you plan on creating your .NET Core project. Enter the
following commands to install .NET Core.
sudo sh -c 'echo "deb [arch=amd64] https://apt-mo.trafficmanager.net/repos/dotnet-release/ xenial main" >
/etc/apt/sources.list.d/dotnetdev.list'
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 417A0893
sudo apt-get update
sudo apt-get install dotnet-dev-1.0.1

Get connection information

4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.
5. Click Show database connection strings.
6. Review the completeADO.NET connection string.

Add System.Data.SqlClient
When using .NET core, add System.Data.SqlClient to your project's csproj file as a dependency.
<ItemGroup>
<PackageReference Include="System.Data.SqlClient" Version="4.3.0" />
</ItemGroup>

Select data
1. In your development environment, open a blank code file.
2. Add using System.Data.SqlClient to your code file (System.Data.SqlClient namespace).
3. Use the following code to query for the top 20 products by category with the SqlCommand.ExecuteReader
command with a SELECT Transact-SQL statement. Add the appropriate values for your server, database,
user and password.
using System;
using System.Data;
using System.Data.SqlClient;

namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
try
{
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder();
builder.DataSource = "your_server.database.windows.net";
builder.UserID = "your_user";
builder.Password = "your_password";
builder.InitialCatalog = "your_database";

using (SqlConnection connection = new SqlConnection(builder.ConnectionString))

{
Console.WriteLine("\nQuery data example:");
Console.WriteLine("=========================================\n");

connection.Open();
StringBuilder sb = new StringBuilder();
sb.Append("SELECT TOP 20 pc.Name as CategoryName, p.name as ProductName ");
sb.Append("FROM [SalesLT].[ProductCategory] pc ");
sb.Append("JOIN [SalesLT].[Product] p ");
sb.Append("ON pc.productcategoryid = p.productcategoryid;");
String sql = sb.ToString();

using (SqlCommand command = new SqlCommand(sql, connection))

{
using (SqlDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
Console.WriteLine("{0} {1}", reader.GetString(0), reader.GetString(1));
}
}
}
}
}
catch (SqlException e)
{
Console.WriteLine(e.ToString());
}
}
}
}

Insert data
Use the following code to insert a new product into the SalesLT.Product table using the
SqlCommand.ExecuteNonQuery with an INSERT Transact-SQL statement. Add the appropriate values for your
server, database, user and password.
using System;
using System.Data;
using System.Data.SqlClient;

using (SqlConnection connection = new SqlConnection(builder.ConnectionString))

{
Console.WriteLine("\nInsert data example:");
Console.WriteLine("=========================================\n");

connection.Open();
StringBuilder sb = new StringBuilder();
sb.Append("INSERT INTO [SalesLT].[Product] ([Name], [ProductNumber], [Color], [StandardCost], [ListPrice], [SellStartDate]) ");
sb.Append("VALUES (@Name, @ProductNumber, @Color, @StandardCost, @ListPrice, @SellStartDate);");
String sql = sb.ToString();
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.Parameters.AddWithValue("@Name", "BrandNewProduct");
command.Parameters.AddWithValue("@ProductNumber", "200989");
command.Parameters.AddWithValue("@Color", "Blue");
command.Parameters.AddWithValue("@StandardCost", 75);
command.Parameters.AddWithValue("@ListPrice", 80);
command.Parameters.AddWithValue("@SellStartDate", "7/1/2016");
int rowsAffected = command.ExecuteNonQuery();
Console.WriteLine(rowsAffected + " row(s) inserted");
}
}
}
catch (SqlException e)
{
Console.WriteLine(e.ToString());
}
}
}
}

Update data
Use the following code to update the new product that you previously added using the
SqlCommand.ExecuteNonQuery with an UPDATE Transact-SQL statement. Add the appropriate values for your
server, database, user and password.
using System;
using System.Data;
using System.Data.SqlClient;

using (SqlConnection connection = new SqlConnection(builder.ConnectionString))

{
Console.WriteLine("\nUpdate data example");
Console.WriteLine("=========================================\n");

connection.Open();
StringBuilder sb = new StringBuilder();
sb.Append("UPDATE [SalesLT].[Product] SET ListPrice = @ListPrice WHERE Name = @Name;");
String sql = sb.ToString();
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.Parameters.AddWithValue("@Name", "BrandNewProduct");
command.Parameters.AddWithValue("@ListPrice", 500);
int rowsAffected = command.ExecuteNonQuery();
Console.WriteLine(rowsAffected + " row(s) updated");
}
}
}
catch (SqlException e)
{
Console.WriteLine(e.ToString());
}
}
}
}

Delete data
Use the following code to delete the new product that you previously added using the
SqlCommand.ExecuteNonQuery with a DELETE Transact-SQL statement . Add the appropriate values for your
server, database, user and password.
using System;
using System.Data;
using System.Data.SqlClient;

using (SqlConnection connection = new SqlConnection(builder.ConnectionString))

{
Console.WriteLine("\nDelete data example");
Console.WriteLine("=========================================\n");

connection.Open();
StringBuilder sb = new StringBuilder();
sb.Append("DELETE FROM SalesLT.Product WHERE Name = @Name;");
String sql = sb.ToString();
using (SqlCommand command = new SqlCommand(sql, connection))
{
command.Parameters.AddWithValue("@Name", "BrandNewProduct");
int rowsAffected = command.ExecuteNonQuery();
Console.WriteLine(rowsAffected + " row(s) deleted");
}
}
}
catch (SqlException e)
{
Console.WriteLine(e.ToString());
}
}
}
}

Next steps
For .NET documentation, see .NET documentation.
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio, see Connect and query with Visual Studio Code.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use PHP to connect and query
data
4/20/2017 • 5 min to read • Edit Online

This quick start demonstrates how to use PHP to connect to an Azure SQL database, and then use Transact-SQL
statements to query, insert, update, and delete data in the database from Mac OS, Ubuntu Linux, and Windows
platforms.
This quick start uses as its starting point the resources created in one of these quick starts:
Create DB - Portal
Create DB - CLI

Install PHP and database communications software

The steps in this section assume that you are familar with developing using PHP and are new to working with
Azure SQL Database. If you are new to developing with PHP, go the Build an app using SQL Server and select
PHP and then select your operating system.
Mac OS
Open your terminal and enter the following commands to install brew, Microsoft ODBC Driver for Mac and
the Microsoft PHP Drivers for SQL Server.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew tap microsoft/msodbcsql https://github.com/Microsoft/homebrew-msodbcsql-preview
brew update
brew install msodbcsql
#for silent install ACCEPT_EULA=y brew install msodbcsql
pecl install sqlsrv-4.1.7preview
pecl install pdo_sqlsrv-4.1.7preview

Linux (Ubuntu)
Enter the following commands to install the Microsoft ODBC Driver for Linux and the Microsoft PHP Drivers
for SQL Server.

sudo su
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql.list
exit
sudo apt-get update
sudo apt-get install msodbcsql mssql-tools unixodbc-dev gcc g++ php-dev
sudo pecl install sqlsrv pdo_sqlsrv
sudo echo "extension= pdo_sqlsrv.so" >> `php --ini | grep "Loaded Configuration" | sed -e "s|.*:\s*||"`
sudo echo "extension= sqlsrv.so" >> `php --ini | grep "Loaded Configuration" | sed -e "s|.*:\s*||"`

Windows
Install PHP 7.1.1 (x64) from WebPlatform Installer
Install the Microsoft ODBC Driver 13.1.
Download the non-thread safe dll's for the Microsoft PHP Driver for SQL Server and place the binaries in the
PHP\v7.x\ext folder.
Next edit your php.ini (C:\Program Files\PHP\v7.1\php.ini) file by adding the reference to the dll. For
example:

extension=php_sqlsrv.dll
extension=php_pdo_sqlsrv.dll

At this point you should have the dll's registered with PHP.

Get connection information

4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.

Select data
Use the following code to query for the top 20 products by category using the sqlsrv_query() function with a
SELECT Transact-SQL statement. The sqlsrv_query function is used to retrieve a result set from a query against a
SQL database. This function accepts a query and returns a result set that can be iterated over with the use of
sqlsrv_fetch_array(). Replace the server, database, username, and password parameters with the values that you
specified when you created the database with the AdventureWorksLT sample data.
<?php
$serverName = "your_server.database.windows.net";
$connectionOptions = array(
"Database" => "your_database",
"Uid" => "your_username",
"PWD" => "your_password"
);
//Establishes the connection
$conn = sqlsrv_connect($serverName, $connectionOptions);
$tsql= "SELECT TOP 20 pc.Name as CategoryName, p.name as ProductName
FROM [SalesLT].[ProductCategory] pc
JOIN [SalesLT].[Product] p
ON pc.productcategoryid = p.productcategoryid";
$getResults= sqlsrv_query($conn, $tsql);
echo ("Reading data from table" . PHP_EOL);
if ($getResults == FALSE)
echo (sqlsrv_errors());
while ($row = sqlsrv_fetch_array($getResults, SQLSRV_FETCH_ASSOC)) {
echo ($row['CategoryName'] . " " . $row['ProductName'] . PHP_EOL);
}
sqlsrv_free_stmt($getResults);
?>

Insert data
Use the following code to insert a new product into the SalesLT.Product table using the sqlsrv_query() function
and the INSERT Transact-SQL statement. Replace the server, database, username, and password parameters with
the values that you specified when you created the database with the AdventureWorksLT sample data.

<?php
$serverName = "your_server.database.windows.net";
$connectionOptions = array(
"Database" => "your_database",
"Uid" => "your_username",
"PWD" => "your_password"
);
//Establishes the connection
$conn = sqlsrv_connect($serverName, $connectionOptions);
$tsql= "INSERT INTO SalesLT.Product (Name, ProductNumber, Color, StandardCost, ListPrice, SellStartDate) VALUES (?,?,?,?,?,?);";
$params = array("BrandNewProduct", "200989", "Blue", 75, 80, "7/1/2016");
$getResults= sqlsrv_query($conn, $tsql, $params);
if ($getResults == FALSE)
echo print_r(sqlsrv_errors(), true);
else{
$rowsAffected = sqlsrv_rows_affected($getResults);
echo ($rowsAffected. " row(s) inserted" . PHP_EOL);
sqlsrv_free_stmt($getResults);
}
?>

Update data
Use the following code to update data in your Azure SQL database using the sqlsrv_query() function and the
UPDATE Transact-SQL statement. Replace the server, database, username, and password parameters with the
values that you specified when you created the database with the AdventureWorksLT sample data.
<?php
$serverName = "your_server.database.windows.net";
$connectionOptions = array(
"Database" => "your_database",
"Uid" => "your_username",
"PWD" => "your_password"
);
//Establishes the connection
$conn = sqlsrv_connect($serverName, $connectionOptions);
$tsql= "UPDATE SalesLT.Product SET ListPrice =? WHERE Name = ?";
$params = array(50,"BrandNewProduct");
$getResults= sqlsrv_query($conn, $tsql, $params);
if ($getResults == FALSE)
echo print_r(sqlsrv_errors(), true);
else{
$rowsAffected = sqlsrv_rows_affected($getResults);
echo ($rowsAffected. " row(s) updated" . PHP_EOL);
sqlsrv_free_stmt($getResults);
}
?>

Delete data
Use the following code to delete the new product that you previously added using the sqlsrv_query() function
and the DELETE Transact-SQL statement. Replace the server, database, username, and password parameters with
the values that you specified when you created the database with the AdventureWorksLT sample data.

<?php
$serverName = "your_server.database.windows.net";
$connectionOptions = array(
"Database" => "your_database",
"Uid" => "your_username",
"PWD" => "your_password"
);
//Establishes the connection
$conn = sqlsrv_connect($serverName, $connectionOptions);
$tsql= "DELETE FROM SalesLT.Product WHERE Name = ?";
$params = array("BrandNewProduct");
$getResults= sqlsrv_query($conn, $tsql, $params);
if ($getResults == FALSE)
echo print_r(sqlsrv_errors(), true);
else{
$rowsAffected = sqlsrv_rows_affected($getResults);
echo ($rowsAffected. " row(s) deleted" . PHP_EOL);
sqlsrv_free_stmt($getResults);
}

Next steps
More information on the Microsoft PHP Driver for SQL Server.
File issues/ask questions.
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use Node.js to connect and
query data
4/20/2017 • 5 min to read • Edit Online

This quick start demonstrates how to connect to an Azure SQL database using Node.js and then use Transact-
SQL statements to query, insert, update, and delete data in the database from Windows, Ubuntu Linux, and Mac
platforms.
This quick start uses as its starting point the resources created in any of these guides:
Create DB - Portal
Create DB - CLI

Install Node.js
The steps in this section assume that you are familar with developing using Node.js and are new to working with
Azure SQL Database. If you are new to developing with Node.js, go the Build an app using SQL Server and select
Node.js and then select your operating system.
Mac OS
Enter the following commands to install brew, an easy-to-use package manager for Mac OS X and Node.js.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew install node

Linux (Ubuntu)
Enter the following commands to install Node.js and npm the package manager for Node.js.

sudo apt-get install -y nodejs npm

Windows
Visit the Node.js downloads page and select your desired Windows installer option.

Install the Tedious SQL Server driver for Node.js

The recommended driver for Node.js is tedious. Tedious is an open-source initiative that Microsoft is
contributing to for Node.js applications on any platform. For this tutorial you need an empty directory to contain
your code and the npm dependencies that we'll install.
To install the tedious driver run the following command inside your directory:

npm install tedious

Get connection information

4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.

Select data
Use the following code to query your Azure SQL database for the top 20 products by category. First import the
driver Connect and Request classes from the tedious driver library. Afterwards create the configuration object
and replace the username, password, server and database variables with the values that you specified when
you created the database with the AdventureWorksLT sample data. Create a Connection object using the specified
config object. After that, define callback for the connect event of the connection object to execute the
queryDatabase() function.
var Connection = require('tedious').Connection;
var Request = require('tedious').Request;

// Create connection to database

var config = {
userName: 'your_username', // update me
password: 'your_password', // update me
server: 'your_server.database.windows.net', // update me
options: {
database: 'your_database' //update me
}
}
var connection = new Connection(config);

// Attempt to connect and execute queries if connection goes through

connection.on('connect', function(err) {
if (err) {
console.log(err)
}
else{
queryDatabase()
}
});

function queryDatabase(){
console.log('Reading rows from the Table...');

// Read all rows from table

request = new Request(
"SELECT TOP 1 pc.Name as CategoryName, p.name as ProductName FROM [SalesLT].[ProductCategory] pc JOIN [SalesLT].[Product] p
ON pc.productcategoryid = p.productcategoryid",
function(err, rowCount, rows) {
console.log(rowCount + ' row(s) returned');
}
);

request.on('row', function(columns) {
columns.forEach(function(column) {
console.log("%s\t%s", column.metadata.colName, column.value);
});
});

connection.execSql(request);
}

Insert data into the database

Use the following code to insert a new product into the SalesLT.Product table using in the insertIntoDatabase()
function and the INSERT Transact-SQL statement. Replace the username, password, server and database
variables with the values that you specified when you created the database with the AdventureWorksLT sample
data.
var Connection = require('tedious').Connection;
var Request = require('tedious').Request;

// Create connection to database

var connection = new Connection(config);

// Attempt to connect and execute queries if connection goes through

connection.on('connect', function(err) {
if (err) {
console.log(err)
}
else{
insertIntoDatabase()
}
});

function insertIntoDatabase(){
console.log("Inserting a brand new product into database...");
request = new Request(
"INSERT INTO SalesLT.Product (Name, ProductNumber, Color, StandardCost, ListPrice, SellStartDate) OUTPUT INSERTED.ProductID
VALUES ('BrandNewProduct', '200989', 'Blue', 75, 80, '7/1/2016')",
function(err, rowCount, rows) {
console.log(rowCount + ' row(s) inserted');
}
);
connection.execSql(request);
}

Update data in the database

Use the following code to delete the new product that you previously added using the updateInDatabase() function
and the UPDATE Transact-SQL statement. Replace the username, password, server and database variables
with the values that you specified when you created the database with the AdventureWorksLT sample data. This
sample uses the Product name inserted in the previous example.
var Connection = require('tedious').Connection;
var Request = require('tedious').Request;

// Create connection to database

var connection = new Connection(config);

// Attempt to connect and execute queries if connection goes through

connection.on('connect', function(err) {
if (err) {
console.log(err)
}
else{
updateInDatabase()
}
});

function updateInDatabase(){
console.log("Updating the price of the brand new product in database...");
request = new Request(
"UPDATE SalesLT.Product SET ListPrice = 50 WHERE Name = 'BrandNewProduct'",
function(err, rowCount, rows) {
console.log(rowCount + ' row(s) updated');
}
);
connection.execSql(request);
}

Delete data from the database

Use the following code to delete data from the database. Replace the username, password, server and
database variables with the values that you specified when you created the database with the
AdventureWorksLT sample data. This time, use an DELETE statement in the deleteFromDatabase() function. This
sample also uses the Product name inserted in the previous example.
var Connection = require('tedious').Connection;
var Request = require('tedious').Request;

// Create connection to database

var connection = new Connection(config);

// Attempt to connect and execute queries if connection goes through

connection.on('connect', function(err) {
if (err) {
console.log(err)
}
else{
deleteFromDatabase()
}
});

function deleteFromDatabase(){
console.log("Deleting the brand new product in database...");
request = new Request(
"DELETE FROM SalesLT.Product WHERE Name = 'BrandNewProduct'",
function(err, rowCount, rows) {
console.log(rowCount + ' row(s) returned');
}
);
connection.execSql(request);
}

Next Steps
More information on the Microsoft Node.js Driver for SQL Server
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use Java to connect and query
data
4/20/2017 • 5 min to read • Edit Online

This quick start demonstrates how to use Java to connect to an Azure SQL database, and then use Transact-SQL
statements to query, insert, update, and delete data in the database from Mac OS, Ubuntu Linux, and Windows
platforms.
This quick start uses as its starting point the resources created in one of these quick starts:
Create DB - Portal
Create DB - CLI

Install Java software

The steps in this section assume that you are familar with developing using Java and are new to working with
Azure SQL Database. If you are new to developing with Java, go the Build an app using SQL Server and select
Java and then select your operating system.
Mac OS
Open your terminal and navigate to a directory where you plan on creating your Java project. Enter the following
commands to install brew and Maven.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew update
brew install maven

Linux (Ubuntu)
Open your terminal and navigate to a directory where you plan on creating your Java project. Enter the following
commands to install Maven.

sudo apt-get install maven

Windows
Install Maven using the official installer. Maven can be used to help manage dependencies, build, test and run
your Java project.

Get connection information

Get the connection information needed to connect to the Azure SQL database. You will need the fully qualified
server name, database name, and login information in the next procedures.
1. Log in to the Azure portal.
2. Select SQL Databases from the left-hand menu, and click your database on the SQL databases page.
3. On the Overview page for your database, review the fully qualified server name as shown in the image
below. You can hover over the server name to bring up the Click to copy option.
4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.
5. Click Show database connection strings.
6. Review the complete JDBC connection string.

Create Maven project

From the terminal, create a new Maven project.

mvn archetype:generate "-DgroupId=com.sqldbsamples" "-DartifactId=SqlDbSample" "-DarchetypeArtifactId=maven-archetype-quickstart" "-

Dversion=1.0.0"

Add the Microsoft JDBC Driver for SQL Server to the dependencies in pom.xml.

<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>6.1.0.jre8</version>
</dependency>

Select data
Use the following code to query for the top 20 products by category using the connection class with a SELECT
Transact-SQL statement. Replace the hostHame, dbName, user, and password parameters with the values that
you specified when you created the database with the AdventureWorksLT sample data.
package com.sqldbsamples;

import java.sql.Connection;
import java.sql.Statement;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.DriverManager;

public class App {

public static void main(String[] args) {

// Connect to database
String hostName = "your_server.database.windows.net";
String dbName = "your_database";
String user = "your_username";
String password = "your_password";
String url =
String.format("jdbc:sqlserver://%s.database.windows.net:1433;database=%s;user=%s;password=%s;encrypt=true;hostNameInCertificate=*.d
atabase.windows.net;loginTimeout=30;", hostName, dbName, user, password);
Connection connection = null;

try {
connection = DriverManager.getConnection(url);
String schema = connection.getSchema();
System.out.println("Successful connection - Schema: " + schema);

System.out.println("Query data example:");

System.out.println("=========================================");

// Create and execute a SELECT SQL statement.

String selectSql = "SELECT TOP 20 pc.Name as CategoryName, p.name as ProductName "
+ "FROM [SalesLT].[ProductCategory] pc "
+ "JOIN [SalesLT].[Product] p ON pc.productcategoryid = p.productcategoryid";

try (Statement statement = connection.createStatement();

ResultSet resultSet = statement.executeQuery(selectSql)) {

// Print results from select statement

System.out.println("Top 20 categories:");
while (resultSet.next())
{
System.out.println(resultSet.getString(1) + " "
+ resultSet.getString(2));
}
}
}
catch (Exception e) {
e.printStackTrace();
}
}
}

Insert data
Use the following code to insert a new product into the SalesLT.Product table using the Prepared Statements
class with an INSERT Transact-SQL statement. Replace the hostHame, dbName, user, and password parameters
with the values that you specified when you created the database with the AdventureWorksLT sample data.
package com.sqldbsamples;

import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.DriverManager;

public class App {

public static void main(String[] args) {

try {
connection = DriverManager.getConnection(url);
String schema = connection.getSchema();
System.out.println("Successful connection - Schema: " + schema);

System.out.println("Insert data example:");

System.out.println("=========================================");

// Prepared statement to insert data

String insertSql = "INSERT INTO SalesLT.Product (Name, ProductNumber, Color, "
+ " StandardCost, ListPrice, SellStartDate) VALUES (?,?,?,?,?,?);";

java.util.Date date = new java.util.Date();

java.sql.Timestamp sqlTimeStamp = new java.sql.Timestamp(date.getTime());

try (PreparedStatement prep = connection.prepareStatement(insertSql)) {

prep.setString(1, "BrandNewProduct");
prep.setInt(2, 200989);
prep.setString(3, "Blue");
prep.setDouble(4, 75);
prep.setDouble(5, 80);
prep.setTimestamp(6, sqlTimeStamp);

int count = prep.executeUpdate();

System.out.println("Inserted: " + count + " row(s)");
}
}
catch (Exception e) {
e.printStackTrace();
}
}
}

Update data
Use the following code to update the new product that you previously added using the Prepared Statements
class with an UPDATE Transact-SQL statement to update data in your Azure SQL database. Replace the
hostHame, dbName, user, and password parameters with the values that you specified when you created the
database with the AdventureWorksLT sample data.
package com.sqldbsamples;

import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.DriverManager;

public class App {

public static void main(String[] args) {

try {
connection = DriverManager.getConnection(url);
String schema = connection.getSchema();
System.out.println("Successful connection - Schema: " + schema);

System.out.println("Update data example:");

System.out.println("=========================================");

// Prepared statement to update data

String updateSql = "UPDATE SalesLT.Product SET ListPrice = ? WHERE Name = ?";

try (PreparedStatement prep = connection.prepareStatement(updateSql)) {

prep.setString(1, "500");
prep.setString(2, "BrandNewProduct");

int count = prep.executeUpdate();

System.out.println("Updated: " + count + " row(s)")
}
}
catch (Exception e) {
e.printStackTrace();
}
}
}

Delete data
Use the following code to delete the new product that you previously added using the Prepared Statements with
a DELETE Transact-SQL statement . Replace the hostHame, dbName, user, and password parameters with the
values that you specified when you created the database with the AdventureWorksLT sample data.
package com.sqldbsamples;

import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.DriverManager;

public class App {

public static void main(String[] args) {

try {
connection = DriverManager.getConnection(url);
String schema = connection.getSchema();
System.out.println("Successful connection - Schema: " + schema);

System.out.println("Delete data example:");

System.out.println("=========================================");

// Prepared statement to delete data

String deleteSql = "DELETE SalesLT.Product WHERE Name = ?";

try (PreparedStatement prep = connection.prepareStatement(deleteSql)) {

prep.setString(1, "BrandNewProduct");

int count = prep.executeUpdate();

System.out.println("Deleted: " + count + " row(s)");
}
}
catch (Exception e) {
e.printStackTrace();
}
}
}

Next steps
GitHub repository for Microsoft JDBC Driver for SQL Server.
File issues/ask questions.
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use Python to connect and
query data
4/20/2017 • 4 min to read • Edit Online

This quick start demonstrates how to use Python to connect to an Azure SQL database, and then use Transact-
SQL statements to query, insert, update, and delete data in the database from Mac OS, Ubuntu Linux, and
Windows platforms.
This quick start uses as its starting point the resources created in one of these quick starts:
Create DB - Portal
Create DB - CLI

Install the Python and database communication libraries

The steps in this section assume that you are familar with developing using Python and are new to working with
Azure SQL Database. If you are new to developing with Python, go the Build an app using SQL Server and select
Python and then select your operating system.
Mac OS
Open your terminal and navigate to a directory where you plan on creating your python script. Enter the
following commands to install brew, Microsoft ODBC Driver for Mac and pyodbc. pyodbc uses the Microsoft
ODBC Driver on Linux to connect to SQL Databases.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Linux (Ubuntu)
Open your terminal and navigate to a directory where you plan on creating your python script. Enter the
following commands to install the Microsoft ODBC Driver for Linux and pyodbc. pyodbc uses the Microsoft
ODBC Driver on Linux to connect to SQL Databases.

Windows
Install the Microsoft ODBC Driver 13.1 (upgrade the driver if prompted). Pyodbc uses the Microsoft ODBC Driver
on Linux to connect to SQL Databases.
Then install pyodbc using pip.
pip install pyodbc==3.1.1

Instructions to enable the use pip can be found here

Get connection information

4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.

Select Data
Use the following code to query for the top 20 products by category using the pyodbc.connect function with a
SELECT Transact-SQL statement. The cursor.execute function is used to retrieve a result set from a query against
SQL Database. This function accepts a query and returns a result set that can be iterated over with the use of
cursor.fetchone(). Replace the server, database, username, and password parameters with the values that you
specified when you created the database with the AdventureWorksLT sample data.

import pyodbc
server = 'your_server.database.windows.net'
database = 'your_database'
username = 'your_username'
password = 'your_password'
driver= '{ODBC Driver 13 for SQL Server}'
cnxn =
pyodbc.connect('DRIVER='+driver+';PORT=1433;SERVER='+server+';PORT=1443;DATABASE='+database+';UID='+username+';PWD='+
password)
cursor = cnxn.cursor()
cursor.execute("SELECT TOP 20 pc.Name as CategoryName, p.name as ProductName FROM [SalesLT].[ProductCategory] pc JOIN [SalesLT].
[Product] p ON pc.productcategoryid = p.productcategoryid")
row = cursor.fetchone()
while row:
print str(row[0]) + " " + str(row[1])
row = cursor.fetchone()

Insert data
Use the following code to insert a new product into the SalesLT.Product table using the cursor.execute function
and the INSERT Transact-SQL statement. Replace the server, database, username, and password parameters with
the values that you specified when you created the database with the AdventureWorksLT sample data.

import pyodbc
server = 'your_server.database.windows.net'
database = 'your_database'
username = 'your_username'
password = 'your_password'
driver= '{ODBC Driver 13 for SQL Server}'
cnxn =
pyodbc.connect('DRIVER='+driver+';PORT=1433;SERVER='+server+';PORT=1443;DATABASE='+database+';UID='+username+';PWD='+
password)
cursor = cnxn.cursor()
with cursor.execute("INSERT INTO SalesLT.Product (Name, ProductNumber, Color, StandardCost, ListPrice, SellStartDate) OUTPUT
INSERTED.ProductID VALUES ('BrandNewProduct', '200989', 'Blue', 75, 80, '7/1/2016')"):
print ('Successfuly Inserted!')
cnxn.commit()

Update data
Use the following code to update the new product that you previously added using the cursor.execute function
and the UPDATE Transact-SQL statement. Replace the server, database, username, and password parameters
with the values that you specified when you created the database with the AdventureWorksLT sample data.

import pyodbc
server = 'your_server.database.windows.net'
database = 'your_database'
username = 'your_username'
password = 'your_password'
driver= '{ODBC Driver 13 for SQL Server}'
cnxn =
pyodbc.connect('DRIVER='+driver+';PORT=1433;SERVER='+server+';PORT=1443;DATABASE='+database+';UID='+username+';PWD='+
password)
cursor = cnxn.cursor()
tsql = "UPDATE SalesLT.Product SET ListPrice = ? WHERE Name = ?"
with cursor.execute(tsql,50,'BrandNewProduct'):
print ('Successfuly Updated!')
cnxn.commit()

Delete data
Use the following code to delete the new product that you previously added using the cursor.execute function
and the DELETE Transact-SQL statement. Replace the server, database, username, and password parameters with
the values that you specified when you created the database with the AdventureWorksLT sample data.
import pyodbc
server = 'your_server.database.windows.net'
database = 'your_database'
username = 'your_username'
password = 'your_password'
driver= '{ODBC Driver 13 for SQL Server}'
cnxn =
pyodbc.connect('DRIVER='+driver+';PORT=1433;SERVER='+server+';PORT=1443;DATABASE='+database+';UID='+username+';PWD='+
password)
cursor = cnxn.cursor()
tsql = "DELETE FROM SalesLT.Product WHERE Name = ?"
with cursor.execute(tsql,'BrandNewProduct'):
print ('Successfuly Deleted!')
cnxn.commit()

Next steps
More information on the Microsoft Python Driver for SQL Server.
Visit the Python Developer Center.
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Ruby, see Connect and query with Ruby.
Azure SQL Database: Use Ruby to connect and
query data
4/20/2017 • 5 min to read • Edit Online

This quick start demonstrates how to use Ruby to connect to an Azure SQL database, and then use Transact-SQL
statements to query, insert, update, and delete data in the database from Mac OS and Ubuntu Linux platforms.
This quick start uses as its starting point the resources created in one of these quick starts:
Create DB - Portal
Create DB - CLI

Install Ruby and database communication libraries

The steps in this section assume that you are familar with developing using Ruby and are new to working with
Azure SQL Database. If you are new to developing with Ruby, go the Build an app using SQL Server and select
Ruby and then select your operating system.
Mac OS
Open your terminal and navigate to a directory where you plan on creating your Ruby script. Enter the following
commands to install brew, FreeTDS, and TinyTDS.

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

brew tap microsoft/msodbcsql https://github.com/Microsoft/homebrew-msodbcsql-preview
brew update
brew install FreeTDS
gem install tiny_tds

Linux (Ubuntu)
Open your terminal and navigate to a directory where you plan on creating your Ruby script. Enter the following
commands to install the FreeTDS and TinyTDS.

wget ftp://ftp.freetds.org/pub/freetds/stable/freetds-1.00.27.tar.gz
tar -xzf freetds-1.00.27.tar.gz
cd freetds-1.00.27
./configure --prefix=/usr/local --with-tdsver=7.3
make
make install
gem install tiny_tds

Get connection information

Get the connection information needed to connect to the Azure SQL database. You will need the fully qualified
server name, database name, and login information in the next procedures.
1. Log in to the Azure portal.
2. Select SQL Databases from the left-hand menu, and click your database on the SQL databases page.
3. On the Overview page for your database, review the fully qualified server name as shown in the image
below. You can hover over the server name to bring up the Click to copy option.
4. If you have forgotten the login information for your Azure SQL Database server, navigate to the SQL
Database server page to view the server admin name and, if necessary, reset the password.

Select data
Use the following code to query for the top 20 products by category using the TinyTDS::Client function with a
SELECT Transact-SQL statement. The TinyTDS::Client function accepts a query and returns a result set. The results
set is iterated over by using result.each do |row|. Replace the server, database, username, and password
parameters with the values that you specified when you created the database with the AdventureWorksLT
sample data.

require 'tiny_tds'
server = 'your_server.database.windows.net'
database = 'your_database'
username = 'your_username'
password = 'your_password'
client = TinyTds::Client.new username: username, password: password,
host: server, port: 1433, database: database, azure: true

puts "Reading data from table"

tsql = "SELECT TOP 20 pc.Name as CategoryName, p.name as ProductName
FROM [SalesLT].[ProductCategory] pc
JOIN [SalesLT].[Product] p
ON pc.productcategoryid = p.productcategoryid"
result = client.execute(tsql)
result.each do |row|
puts row
end

Insert data
Use the following code to insert a new product into the SalesLT.Product table using the TinyTDS::Client function
with an INSERT Transact-SQL statement. Replace the server, database, username, and password parameters with
the values that you specified when you created the database with the AdventureWorksLT sample data.
Tis example demonstrates how to execute an INSERT statement safely, pass parameters which protect your
application from SQL injection vulnerability, and retrieve the auto-generated Primary Key value.
To use TinyTDS with Azure, it is recommended that you execute several SET statements to change how the
current session handles specific information. Recommended SET statements are provided in the code sample.
For example, SET ANSI_NULL_DFLT_ON will allow new columns created to allow null values even if the nullability
status of the column is not explicitly stated.
To align with the Microsoft SQL Server datetime format, use the strftime function to cast to the corresponding
datetime format.
require 'tiny_tds'
server = 'your_server.database.windows.net'
database = 'your_database'
username = 'your_username'
password = 'your_password'
client = TinyTds::Client.new username: username, password: password,
host: server, port: 1433, database: database, azure: true

# settings for Azure

result = client.execute("SET ANSI_NULLS ON")
result = client.execute("SET CURSOR_CLOSE_ON_COMMIT OFF")
result = client.execute("SET ANSI_NULL_DFLT_ON ON")
result = client.execute("SET IMPLICIT_TRANSACTIONS OFF")
result = client.execute("SET ANSI_PADDING ON")
result = client.execute("SET QUOTED_IDENTIFIER ON")
result = client.execute("SET ANSI_WARNINGS ON")
result = client.execute("SET CONCAT_NULL_YIELDS_NULL ON")

def insert(name, productnumber, color, standardcost, listprice, sellstartdate)

tsql = "INSERT INTO SalesLT.Product (Name, ProductNumber, Color, StandardCost, ListPrice, SellStartDate)
VALUES (N'#{name}', N'#{productnumber}',N'#{color}',N'#{standardcost}',N'#{listprice}',N'#{sellstartdate}')"
result = client.execute(tsql)
result.each
puts "#{result.affected_rows} row(s) affected"
end
insert('BrandNewProduct', '200989', 'Blue', 75, 80, '7/1/2016')

Update data
Use the following code to update the new product that you previously added using the TinyTDS::Client function
with a UPDATE Transact-SQL statement. Replace the server, database, username, and password parameters with
the values that you specified when you created the database with the AdventureWorksLT sample data.

def update(name, listPrice, client)

tsql = "UPDATE SalesLT.Product SET ListPrice = N'#{listPrice}' WHERE Name =N'#{name}'";
result = client.execute(tsql)
result.each
puts "#{result.affected_rows} row(s) affected"
end
update('BrandNewProduct', 500, client)

Delete data
Use the following code to delete the new product that you previously added using the TinyTDS::Client function
with a DELETE Transact-SQL statement. Replace the server, database, username, and password parameters with
the values that you specified when you created the database with the AdventureWorksLT sample data.
require 'tiny_tds'
server = 'your_server.database.windows.net'
database = 'your_database'
username = 'your_username'
password = 'your_password'
client = TinyTds::Client.new username: username, password: password,
host: server, port: 1433, database: database, azure: true

# settings for Azure

def delete(name, client)

tsql = "DELETE FROM SalesLT.Product WHERE Name = N'#{name}'"
result = client.execute(tsql)
result.each
puts "#{result.affected_rows} row(s) affected"
end
delete('BrandNewProduct', client)

Next Steps
Github repository for TinyTDS.
File issues/ask questions.
More information on the Ruby Driver for SQL Server.
To connect and query using SQL Server Management Studio, see Connect and query with SSMS
To connect and query using Visual Studio, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
Design your first Azure SQL database
4/21/2017 • 8 min to read • Edit Online

In this tutorial, you build a database for a university to track student grades and courses enrollment. This tutorial
demonstrates how to use the Azure portal and SQL Server Management Studio (SSMS) to create an Azure SQL
database on an Azure SQL Database logical server, add tables to the database, load data into the tables, and query
the database. It also demonstrates how to use SQL Database point in time restore capabilities to restore the
database to an earlier point in time.
To complete this tutorial, make sure you have installed the newest version of SQL Server Management Studio
(SSMS).

Log in to the Azure portal

Create a blank SQL database in Azure

An Azure SQL database is created with a defined set of compute and storage resources. The database is created
within an Azure resource group and in an Azure SQL Database logical server.
Follow these steps to create a blank SQL database.
1. Click the New button found on the upper left-hand corner of the Azure portal.
2. Select Databases from the New page, and select SQL Database from the Databases page.

3. Fill out the SQL Database form with the following information, as shown on the preceding image:
Database name: mySampleDatabase
Resource group: myResourceGroup
Source: Blank database
4. Click Server to create and configure a new server for your new database. Fill out the New server form
specifying a globally unique server name, provide a name for the Server admin login, and then specify the
password of your choice.

5. Click Select.
6. Click Pricing tier to specify the service tier and performance level for your new database. For this tutorial,
select 20 DTUs and 250 GB of storage.

7. Click Apply.
8. Click Create to provision the database. Provisioning takes about a minute and a half to complete.
9. On the toolbar, click Notifications to monitor the deployment process.
Create a server-level firewall rule
Azure SQL Databases are protected by a firewall. By default, all connections to the server and the databases inside
the server are rejected. Follow these steps to create a SQL Database server-level firewall rule for your server to
allow connections from your client's IP address.
1. After the deployment completes, click SQL databases from the left-hand menu and click your new database,
mySampleDatabase, on the SQL databases page. The overview page for your database opens, showing
you the fully qualified server name (such as mynewserver-20170313.database.windows.net) and
provides options for further configuration.

2. Click Set server firewall on the toolbar as shown in the previous image. The Firewall settings page for the
SQL Database server opens.
3. Click Add client IP on the toolbar and then click Save. A server-level firewall rule is created for your current
IP address.
4. Click OK and then click the X to close the Firewall settings page.
You can now connect to the database and its server using SQL Server Management Studio or another tool of your
choice.

NOTE
SQL Database communicates over port 1433. If you are trying to connect from within a corporate network, outbound traffic
over port 1433 may not be allowed by your network's firewall. If so, you will not be able to connect to your Azure SQL
Database server unless your IT department opens port 1433.

Get connection information

Get the fully qualified server name for your Azure SQL Database server in the Azure portal. You use the fully
qualified server name to connect to your server using SQL Server Management Studio.
1. Log in to the Azure portal.
2. Select SQL Databases from the left-hand menu, and click your database on the SQL databases page.
3. In the Essentials pane in the Azure portal page for your database, locate and then copy the Server name.

Connect to your database using SQL Server Management Studio

Use SQL Server Management Studio to establish a connection to your Azure SQL Database server.
1. Open SQL Server Management Studio.
2. In the Connect to Server dialog box, enter the following information:
Server type: Specify Database engine
Server name: Enter your fully qualified server name, such as
mynewserver20170313.database.windows.net
Authentication: Specify SQL Server Authentication
Login: Enter your server admin account
Password: Enter the password for your server admin account

1. Click Options in the Connect to server dialog box. In the Connect to database section, enter
mySampleDatabase to connect to this database.

2. Click Connect. The Object Explorer window opens in SSMS.

3. In Object Explorer, expand Databases and then expand mySampleDatabase to view the objects in the
sample database.

Create tables in the database

Create a database schema with four tables that model a student management system for universities using
Transact-SQL:
Person
Course
Student
Credit that model a student management system for universities
The following diagram shows how these tables are related to each other. Some of these tables reference columns in
other tables. For example, the Student table references the PersonId column of the Person table. Study the
diagram to understand how the tables in this tutorial are related to one another. For an in-depth look at how to
create effective database tables, see Create effective database tables. For information about choosing data types,
see Data types.

NOTE
You can also use the table designer in SQL Server Management Studio to create and design your tables.
1. In Object Explorer, right-click mySampleDatabase and click New Query. A blank query window opens that
is connected to your database.
2. In the query window, execute the following query to create four tables in your database:
-- Create Person table

CREATE TABLE Person

(
PersonId INT IDENTITY PRIMARY KEY,
FirstName NVARCHAR(128) NOT NULL,
MiddelInitial NVARCHAR(10),
LastName NVARCHAR(128) NOT NULL,
DateOfBirth DATE NOT NULL
)

-- Create Student table

CREATE TABLE Student

(
StudentId INT IDENTITY PRIMARY KEY,
PersonId INT REFERENCES Person (PersonId),
Email NVARCHAR(256)
)

-- Create Course table

CREATE TABLE Course

(
CourseId INT IDENTITY PRIMARY KEY,
Name NVARCHAR(50) NOT NULL,
Teacher NVARCHAR(256) NOT NULL
)

-- Create Credit table

CREATE TABLE Credit

(
StudentId INT REFERENCES Student (StudentId),
CourseId INT REFERENCES Course (CourseId),
Grade DECIMAL(5,2) CHECK (Grade <= 100.00),
Attempt TINYINT,
CONSTRAINT [UQ_studentgrades] UNIQUE CLUSTERED
(
StudentId, CourseId, Grade, Attempt
)
)
1. Expand the 'tables' node in the SQL Server Management Studio Object explorer to see the tables you
created.
Load data into the tables
1. Create a folder called SampleTableData in your Downloads folder to store sample data for your database.
2. Right-click the following links and save them into the SampleTableData folder.
SampleCourseData
SamplePersonData
SampleStudentData
SampleCreditData
3. Open a command prompt window and navigate to the SampleTableData folder.
4. Execute the following commands to insert sample data into the tables replacing the values for ServerName,
DatabaseName, UserName, and Password with the values for your environment.

bcp Course in SampleCourseData -S <ServerName>.database.windows.net -d <DatabaseName> -U <Username> -P <password> -q -c -t ","

bcp Person in SamplePersonData -S <ServerName>.database.windows.net -d <DatabaseName> -U <Username> -P <password> -q -c -t ","
bcp Student in SampleStudentData -S <ServerName>.database.windows.net -d <DatabaseName> -U <Username> -P <password> -q -c -t
","
bcp Credit in SampleCreditData -S <ServerName>.database.windows.net -d <DatabaseName> -U <Username> -P <password> -q -c -t ","

You have now loaded sample data into the tables you created earlier.

Query the tables

Execute the following queries to retrieve information from the database tables. See Writing SQL Queries to learn
more about writing SQL queries. The first query joins all four tables to find all the students taught by 'Dominick
Pope' who have a grade higher than 75% in his class. The second query joins all four tables and finds all courses in
which 'Noe Coleman' has ever enrolled.
1. In a SQL Server Management Studio query window, execute the following query:

-- Find the students taught by Dominick Pope who have a grade higher than 75%

SELECT person.FirstName,
person.LastName,
course.Name,
credit.Grade
FROM Person AS person
INNER JOIN Student AS student ON person.PersonId = student.PersonId
INNER JOIN Credit AS credit ON student.StudentId = credit.StudentId
INNER JOIN Course AS course ON credit.CourseId = course.courseId
WHERE course.Teacher = 'Dominick Pope'
AND Grade > 75

2. In a SQL Server Management Studio query window, execute following query:

-- Find all the courses in which Noe Coleman has ever enrolled

SELECT course.Name,
course.Teacher,
credit.Grade
FROM Course AS course
INNER JOIN Credit AS credit ON credit.CourseId = course.CourseId
INNER JOIN Student AS student ON student.StudentId = credit.StudentId
INNER JOIN Person AS person ON person.PersonId = student.PersonId
WHERE person.FirstName = 'Noe'
AND person.LastName = 'Coleman'

Restore a database to a previous point in time

Imagine you have accidentally deleted a table. This is something you cannot easily recover from. Azure SQL
Database allows you to go back to any point in time in the last up to 35 days and restore this point in time to a new
database. You can you this database to recover your deleted data. The following steps restore the sample database
to a point before the tables were added.
1. On the SQL Database page for your database, click Restore on the toolbar. The Restore page opens.

2. Fill out the Restore form with the required information:

Database name: Provide a database name
Point-in-time: Select the Point-in-time tab on the Restore form
Restore point: Select a time that occurs before the database was changed
Target server: You cannot change this value when restoring a database
Elastic database pool: Select None
Pricing tier: Select 20 DTUs and 250 GB of storage.

3. Click OK to restore the database to restore to a point in time before the tables were added. Restoring a
database to a different point in time creates a duplicate database in the same server as the original database
as of the point in time you specify, provided that it is within the retention period for your service tier.

Next Steps
For PowerShell samples for common tasks, see SQL Database PowerShell samples
Migrate your SQL Server database to Azure SQL
Database
4/21/2017 • 8 min to read • Edit Online

In this tutorial, you migrate an existing SQL Server database to Azure SQL Database using the Microsoft Data
Migration Assistant and go through the required steps from preparing for migration to performing the actual data
migration, and connecting to the migrated database after completed migration.

IMPORTANT
To fix compatibility issues, use Visual Studio Data Tools.

If you don't have an Azure subscription, create a free account before you begin.
To complete this tutorial, make sure you have:
The newest version of SQL Server Management Studio (SSMS). Installing SSMS also installs the newest version
of SQLPackage, a command-line utility that can be used to automate a range of database development tasks.
The Data Migration Assistant (DMA).
A database to migrate. This tutorial uses the SQL Server 2008R2 AdventureWorks OLTP database on an
instance of SQL Server 2008R2 or newer, but you can use any database of your choice.

Prepare for migration

You are ready to prepare for migration. Follow these steps to use the Data Migration Assistant to assess the
readiness of your database for migration to Azure SQL Database.
1. Open the Data Migration Assistant. You can run DMA on any computer with connectivity to the SQL
Server instance containing the database that you plan to migrate, you do not need to install it on the
computer hosting the SQL Server instance.
2. In the left-hand menu, click + New to create an Assessment project. Fill in the form with a Project name
(all other values should be left at their default values) and click Create. The Options page opens.

3. On the Options page, click Next. The Select sources page opens.

4. On the Select sources page, enter the name of SQL Server instance containing the server you plan to
migrate. Change the other values on this page if necessary. Click Connect.
5. In the Add sources portion of the Select sources page, select the checkboxes for the databases to be tested
for compatibility. Click Add.

6. Click Start Assessment.

7. When the assessment completes, first look to see if the database is sufficiently compatible to migrate. Look
for the checkmark in a green circle.

8. Review the results, beginning with SQL Server feature parity. Specifically review the information about
unsupported and partially supported features, and the provided information about recommended actions.
9. Click Compatibility issues. Specifically review the information about migration blockers, behavior changes,
and deprecated features for each compatibility level. For the AdventureWorks2008R2 database, review the
changes to Full-Text Search since SQL Server 2008 and the changes to SERVERPROPERTY('LCID') since SQL
Server 2000. For details on these changes, links for more information is provided. Many search options and
settings for Full-Text Search have changed

IMPORTANT
After you migrate your database to Azure SQL Database, you can choose to operate the database at its current
compatibility level (level 100 for the AdventureWorks2008R2 database) or at a higher level. For more information on
the implications and options for operating a database at a specific compatibility level, see ALTER DATABASE
Compatibility Level. See also ALTER DATABASE SCOPED CONFIGURATION for information about additional
database-level settings related to compatibility levels.

10. Optionally, click Export report to save the report as a JSON file.
11. Close the Data Migration Assistant.

Export to BACPAC file

A BACPAC file is a ZIP file with an extension of BACPAC containing the metadata and data from a SQL Server
database. A BACPAC file can be stored in Azure blob storage or in local storage for archiving or for migration -
such as from SQL Server to Azure SQL Database. For an export to be transactionally consistent, you must ensure
either that no write activity is occurring during the export.
Follow these steps to use the SQLPackage command-line utility to export the AdventureWorks2008R2 database to
local storage.
1. Open a Windows command prompt and change your directory to a folder in which you have the 130
version of SQLPackage - such as C:\Program Files (x86)\Microsoft SQL Server\130\DAC\bin.
2. Execute the following SQLPackage command at the command prompt to export the
AdventureWorks2008R2 database from localhost to AdventureWorks2008R2.bacpac. Change any of
these values as appropriate to your environment.

sqlpackage.exe /Action:Export /ssn:localhost /sdn:AdventureWorks2008R2 /tf:AdventureWorks2008R2.bacpac

Once the execution is complete the generated BCPAC file is stored in the directory where the sqlpackage
executable is located. In this example C:\Program Files (x86)\Microsoft SQL Server\130\DAC\bin.

Log in to the Azure portal

Log in to the Azure portal. Logging on from the computer from which you are running the SQLPackage command-
line utility eases the creation of the firewall rule in step 5.

Create a SQL Database logical server

An Azure SQL Database logical server acts as a central administrative point for multiple databases. Follow these
steps to create a SQL Database logical server to contain the migrated Adventure Works OLTP SQL Server database.
1. Click the New button found on the upper left-hand corner of the Azure portal.
2. Type server in the search window on the New page, and select SQL database (logical server) from the
filtered list.

3. On the Everything page, click SQL server (logical server) and then click Create on the SQL Server
(logical server) page.

4. Fill out the SQL server (logical server) form with the following information, as shown on the preceding
image:
Server name: Specify a globally unique server name
Server admin login: Provide a name for the Server admin login
Password: Specify the password of your choice
Resource group: Select Create new and specify myResourceGroup
Location: Select a data center location

5. Click Create to provision the logical server. Provisioning takes a few minutes.

Create a server-level firewall rule

The SQL Database service creates a firewall at the server-level that prevents external applications and tools from
connecting to the server or any databases on the server unless a firewall rule is created to open the firewall for
specific IP addresses. Follow these steps to create a SQL Database server-level firewall rule for the IP address of the
computer from which you are running the SQLPackage command-line utility. This enables SQLPackage to connect
to the SQL Database logical server through the Azure SQL Database firewall.
1. Click All resources from the left-hand menu and click your new server on the All resources page. The
overview page for your server opens, showing you the fully qualified server name (such as
mynewserver20170403.database.windows.net) and provides options for further configuration.
2. Click Firewall in the left-hand menu under Settings on the overview page. The Firewall settings page for
the SQL Database server opens.
3. Click Add client IP on the toolbar to add the IP address of the computer you are currently using and then
click Save. A server-level firewall rule is created for this IP address.

4. Click OK.
You can now connect to all databases on this server using SQL Server Management Studio or another tool of your
choice from this IP address using the server admin account created previously.
NOTE
SQL Database communicates over port 1433. If you are trying to connect from within a corporate network, outbound traffic
over port 1433 may not be allowed by your network's firewall. If so, you will not be able to connect to your Azure SQL
Database server unless your IT department opens port 1433.

Import BACPAC file to Azure SQL Database

The newest versions of the SQLPackage command-line utility provide support for creating an Azure SQL database
at a specified service tier and performance level. For best performance during import, select a high service tier and
performance level and then scale down after import if the service tier and performance level is higher than you
need immediately.
Follow these steps use the SQLPackage command-line utility to import the AdventureWorks2008R2 database to
Azure SQL Database.
Execute the following SQLPackage command at the command prompt to import the
AdventureWorks2008R2 database from local storage to the Azure SQL Database logical server that you
previously created with a database name of myMigratedDatabase, a service tier of Premium, and a
Service Objective of P6. Change any of these three values as appropriate to your environment.

SqlPackage.exe /a:import /tcs:"Data Source=mynewserver20170403.database.windows.net;Initial Catalog=myMigratedDatabase;User

Id=<change_to_your_admin_user_account>;Password=<change_to_your_password>" /sf:AdventureWorks2008R2.bacpac
/p:DatabaseEdition=Premium /p:DatabaseServiceObjective=P6
IMPORTANT
An Azure SQL Database logical server listens on port 1433. If you are attempting to connect to an Azure SQL Database
logical server from within a corporate firewall, this port must be open in the corporate firewall for you to successfully connect.

Connect using SQL Server Management Studio (SSMS)

Use SQL Server Management Studio to establish a connection to your Azure SQL Database server and newly
migrated database. If you are running SQL Server Management Studio on a different computer from which you
ran SQLPackage, create a firewall rule for this computer using the steps in the previous procedure.
1. Open SQL Server Management Studio.
2. In the Connect to Server dialog box, enter the following information:
Server type: Specify Database engine
Server name: Enter your fully qualified server name, such as
mynewserver20170403.database.windows.net
Authentication: Specify SQL Server Authentication
Login: Enter your server admin account
Password: Enter the password for your server admin account

3. Click Connect. The Object Explorer window opens.

4. In Object Explorer, expand Databases and then expand myMigratedDatabase to view the objects in the
sample database.

Change database properties

You can change the service tier, performance level, and compatibility level using SQL Server Management Studio.
1. In Object Explorer, right-click myMigratedDatabase and click New Query. A query window opens
connected to your database.
2. Execute the following command to set the service tier to Standard and the performance level to S1.

ALTER DATABASE myMigratedDatabase

MODIFY
(
EDITION = 'Standard'
, MAXSIZE = 250 GB
, SERVICE_OBJECTIVE = 'S1'
);
3. Execute the following command to change the database compatibility level to 130.

ALTER DATABASE myMigratedDatabase

SET COMPATIBILITY_LEVEL = 130;

Next steps
For an overview of migration, see Database migration.
For a discussion of T-SQL differences, see Resolving Transact-SQL differences during migration to SQL
Database.
To connect and query using Visual Studio Code, see Connect and query with Visual Studio Code.
To connect and query using .NET, see Connect and query with .NET.
To connect and query using PHP, see Connect and query with PHP.
To connect and query using Node.js, see Connect and query with Node.js.
To connect and query using Java, see Connect and query with Java.
To connect and query using Python, see Connect and query with Python.
To connect and query using Ruby, see Connect and query with Ruby.
Azure CLI samples for Azure SQL Database
3/29/2017 • 1 min to read • Edit Online

The following table includes links to sample Azure CLI scripts for Azure SQL Database.

Create a single database and an elastic pool

Create a single database and configure a firewall rule Creates a single Azure SQL database and configures a server-
level firewall rule.

Create elastic pools and move pooled databases Creates elastic pools, and moves pooled Azure SQL databases,
and changes performance levels.

Scale a single database and an elastic pool

Scale a single database Scales a single Azure SQL database to a different performance
level after querying the size information for the database.

Scale an elastic pool Scales an elastic pool to a different performance level.

Azure PowerShell samples for Azure SQL Database
4/5/2017 • 1 min to read • Edit Online

The following table includes links to sample Azure PowerShell scripts for Azure SQL Database.

Create a single database and an elastic pool

Create a single database and configure a firewall rule Creates a single Azure SQL database and configures a server-
level firewall rule.

Create elastic pools and move pooled databases Creates elastic pools, and moves pooled databases, and
changes performance levels.

Configure geo-replication and failover

Configure and failover a single database using Active Geo- Configures Active Geo-Replication for a single Azure SQL
Replication database and fails it over to the secondary replica.

Configure and failover a pooled database using Active Geo- Configures Active Geo-Replication for an Azure SQL database
Replication in an elastic pool and fails it over to the secondary replica.

Scale a single databases and an elastic pool

Scale a single database Monitors the performance metrics of an Azure SQL database,
scales it to a higher performance level and creates an alert rule
on one of the performance metrics.

Scale an elastic pool Monitors the performance metrics of an elastic pool, scales it
to a higher performance level, and creates an alert rule on one
of the performance metrics.

Auditing and threat detection

Configure auditing and threat-detection Configures auditing and threat detection policies for an Azure
SQL database.

Restore, copy, and import a database

Restore a database Restores an Azure SQL database from a geo-redundant

backup and restores a deleted Azure SQL database to the
latest backup.

Copy a database to new server Creates a copy of an existing Azure SQL database in a new
Azure SQL server.

Import a database from a bacpac file Imports a database to an Azure SQL server from a bacpac file.
Azure SQL database overview
4/14/2017 • 2 min to read • Edit Online

This topic provides an overview of Azure SQL databases. For information about Azure SQL logical servers, see
Logical servers.

What is Azure SQL database?

Each database in Azure SQL Database is associated with a logical server. The database can be:
A single database with its own set of resources (DTUs)
Part of an elastic pool that shares a set of resources (eDTUs)
Part of a scaled-out set of sharded databases, which can be either single or pooled databases
Part of a set of databases participating in a multitenant SaaS design pattern, and whose databases can either be
single or pooled databases (or both)

How do I connect and authenticate to an Azure SQL database?

Authentication and authorization: Azure SQL Database supports SQL authentication and Azure Active
Directory Authentication (with certain limitations - see Connect to SQL Database with Azure Active Directory
Authentication for authentication. You can connect and authenticate to Azure SQL databases through the
server's master database or directly to a user database. For more information, see Managing Databases and
Logins in Azure SQL Database. Windows Authentication is not supported.
TDS: Microsoft Azure SQL Database supports tabular data stream (TDS) protocol client version 7.3 or later.
TCP/IP: Only TCP/IP connections are allowed.
SQL Database firewall: To help protect your data, a SQL Database firewall prevents all access to your database
server or its databases until you specify which computers have permission. See Firewalls

What collations are supported?

The default database collation used by Microsoft Azure SQL Database is SQL_LATIN1_GENERAL_CP1_CI_AS,
where LATIN1_GENERAL is English (United States), CP1 is code page 1252, CI is case-insensitive, and AS is
accent-sensitive. For more information about how to set the collation, see COLLATE (Transact-SQL).

What are the naming requirements for database objects?

Names for all new objects must comply with the SQL Server rules for identifiers. For more information, see
Identifiers.

What features are supported by Azure SQL databases?

For information about supported features, see Features. See also Azure SQL Database Transact-SQL differences for
more background on the reasons for lack of support for certain types of features.

How do I manage an Azure SQL database?

You can manage Azure SQL Database logical servers using several methods:
Azure portal
PowerShell
Transact-SQL
Visual Studio Code
REST

Next steps
For information about the Azure SQL Database service, see What is SQL Database?
For information about supported features, see Features
For an overview of Azure SQL logical servers, see SQL Database logical server overview
For information about Transact-SQL support and differences, see Azure SQL Database Transact-SQL differences.
For information about specific resource quotas and limitations based on your service tier. For an overview of
service tiers, see SQL Database service tiers.
For an overview of security, see Azure SQL Database Security Overview.
For information on driver availability and support for SQL Database, see Connection Libraries for SQL Database
and SQL Server.
Azure SQL Database logical servers
4/14/2017 • 3 min to read • Edit Online

This topic provides considerations and guidelines for working with Azure SQL logical servers. For information
about Azure SQL databases, see SQL databases.

What is an Azure SQL Database logical server?

An Azure SQL Database logical server acts as a central administrative point for multiple databases. In SQL
Database, a server is a logical construct that is distinct from a SQL Server instance that you may be familiar with in
the on-premises world. Specifically, the SQL Database service makes no guarantees regarding location of the
databases in relation to their logical servers, and exposes no instance-level access or features.
An Azure Database logical server:
Is created within an Azure subscription, but can be moved with its contained resources to another subscription
Is the parent resource for databases, elastic pools, and data warehouses
Provides a namespace for databases, elastic pools, data warehouses
Is a logical container with strong lifetime semantics - delete a server and it deletes the contained databases,
elastic pools, data warehouses
Participates in Azure role-based access control (RBAC); databases, elastic pools within a server inherit access
rights from the server
Is a high-order element of the identity of databases and elastic pools for Azure resource management purposes
(see the URL scheme for databases and pools)
Collocates resources in a region
Provides a connection endpoint for database access (.database.windows.net)
Provides access to metadata regarding contained resources via DMVs by connecting to a master database
Provides the scope for management policies that apply to its databases: logins, firewall, audit, threat detection,
etc.
Is restricted by a quota within the parent subscription (six servers per subscription - see Subscription limits
here)
Provides the scope for database quota and DTU quota for the resources it contains (such as 45000 DTU)
Is the versioning scope for capabilities enabled on contained resources
Server-level principal logins can manage all databases on a server
Can contain logins similar to those in instances of SQL Server on your premises that are granted access to one
or more databases on the server, and can be granted limited administrative rights. For more information, see
Logins.

How do I connect and authenticate to an Azure SQL Database logical

server?
Authentication and authorization: Azure SQL Database supports SQL authentication and Azure Active
Directory Authentication (with certain limitations - see Connect to SQL Database with Azure Active Directory
Authentication for authentication. You can connect and authenticate to Azure SQL databases through the
server's master database or directly to a user database. For more information, see Managing Databases and
Logins in Azure SQL Database. Windows Authentication is not supported.
TDS: Microsoft Azure SQL Database supports tabular data stream (TDS) protocol client version 7.3 or later.
TCP/IP: Only TCP/IP connections are allowed.
SQL Database firewall: To help protect your data, a SQL Database firewall prevents all access to your database
server or its databases until you specify which computers have permission. See Firewalls

What collations are supported?

The default database collation used by Microsoft Azure SQL Database (including the master database) is
SQL_LATIN1_GENERAL_CP1_CI_AS, where LATIN1_GENERAL is English (United States), CP1 is code page 1252,
CI is case-insensitive, and AS is accent-sensitive. It is not recommended to alter the collation after databases are
created. For more information about collations, see COLLATE (Transact-SQL).

What are the naming requirements for database objects?

Names for all new objects must comply with the SQL Server rules for identifiers. For more information, see
Identifiers.

What features are supported?

For information about supported features, see Features. See also Azure SQL Database Transact-SQL differences for
more background on the reasons for lack of support for certain types of features.

How do I manage a logical server?

You can manage Azure SQL Database logical servers using several methods:
Azure portal
PowerShell
REST

Next steps
For information about the Azure SQL Database service, see What is SQL Database?
For information about supported features, see Features
For an overview of Azure SQL databases, see SQL database overview
For information about Transact-SQL support and differences, see Azure SQL Database Transact-SQL
differences.
For information about specific resource quotas and limitations based on your service tier. For an overview of
service tiers, see SQL Database service tiers.
For an overview of security, see Azure SQL Database Security Overview.
For information on driver availability and support for SQL Database, see Connection Libraries for SQL Database
and SQL Server.
Elastic pools help you manage and scale multiple
SQL databases
4/19/2017 • 14 min to read • Edit Online

SQL Database elastic pools are a simple, cost-effective solution for managing and scaling multiple databases
that have varying and unpredictable usage demands. The databases in an elastic pool are on a single Azure
SQL Database server and share a set number of resources (elastic Database Transaction Units (eDTUs)) at a
set price. Elastic pools in Azure SQL Database enable SaaS developers to optimize the price performance for a
group of databases within a prescribed budget while delivering performance elasticity for each database.

NOTE
Elastic pools are generally available (GA) in all Azure regions except West India where it is currently in preview. GA of
elastic pools in this region will occur as soon as possible.

Overview of elastic pools

SaaS developers build applications on top of large scale data-tiers consisting of multiple databases. A
common application pattern is to provision a single database for each customer. But different customers
often have varying and unpredictable usage patterns, and it is difficult to predict the resource requirements of
each individual database user. Traditionally, you had two options:
Over-provision resources based on peak usage and over pay, or
Under-provision to save cost, at the expense of performance and customer satisfaction during peaks.
Elastic pools solve this problem by ensuring that databases get the performance resources they need when
they need it. They provide a simple resource allocation mechanism within a predictable budget. To learn more
about design patterns for SaaS applications using elastic pools, see Design Patterns for Multi-tenant SaaS
Applications with Azure SQL Database.

Elastic pools enable the developer to purchase elastic Database Transaction Units (eDTUs) for a pool shared
by multiple databases to accommodate unpredictable periods of usage by individual databases. The eDTU
requirement for a pool is determined by the aggregate utilization of its databases. The number of eDTUs
available to the pool is controlled by the developer budget. The developer simply adds databases to the pool,
sets the minimum and maximum eDTUs for the databases, and then sets the eDTU of the pool based on their
budget. A developer can use pools to seamlessly grow their service from a lean startup to a mature business
at ever-increasing scale.
Within the pool, individual databases are given the flexibility to auto-scale within set parameters. Under
heavy load, a database can consume more eDTUs to meet demand. Databases under light loads consume
less, and databases under no load consume no eDTUs. Provisioning resources for the entire pool rather than
for single databases simplifies your management tasks. Plus you have a predictable budget for the pool.
Additional eDTUs can be added to an existing pool with no database downtime, except that the databases
may need to be moved to provide the additional compute resources for the new eDTU reservation. Similarly,
if extra eDTUs are no longer needed they can be removed from an existing pool at any point in time. And you
can add or subtract databases to the pool. If a database is predictably under-utilizing resources, move it out.
You can create and manage an elastic pool using the Azure portal, PowerShell, Transact-SQL, C#, and the
REST API.

When to consider a pool

Pools are well suited for a large number of databases with specific utilization patterns. For a given database,
this pattern is characterized by low average utilization with relatively infrequent utilization spikes.
The more databases you can add to a pool the greater your savings become. Depending on your application
utilization pattern, it is possible to see savings with as few as two S3 databases.
The following sections help you understand how to assess if your specific collection of databases can benefit
from being in a pool. The examples use Standard pools but the same principles also apply to Basic and
Premium pools.
Assessing database utilization patterns
The following figure shows an example of a database that spends much time idle, but also periodically spikes
with activity. This is a utilization pattern that is suited for a pool:

For the five-minute period illustrated, DB1 peaks up to 90 DTUs, but its overall average usage is less than five
DTUs. An S3 performance level is required to run this workload in a single database, but this leaves most of
the resources unused during periods of low activity.
A pool allows these unused DTUs to be shared across multiple databases, and so reduces the DTUs needed
and overall cost.
Building on the previous example, suppose there are additional databases with similar utilization patterns as
DB1. In the next two figures below, the utilization of four databases and 20 databases are layered onto the
same graph to illustrate the non-overlapping nature of their utilization over time:
The aggregate DTU utilization across all 20 databases is illustrated by the black line in the preceding figure.
This shows that the aggregate DTU utilization never exceeds 100 DTUs, and indicates that the 20 databases
can share 100 eDTUs over this time period. This results in a 20x reduction in DTUs and a 13x price reduction
compared to placing each of the databases in S3 performance levels for single databases.
This example is ideal for the following reasons:
There are large differences between peak utilization and average utilization per database.
The peak utilization for each database occurs at different points in time.
eDTUs are shared between many databases.
The price of a pool is a function of the pool eDTUs. While the eDTU unit price for a pool is 1.5x greater than
the DTU unit price for a single database, pool eDTUs can be shared by many databases and fewer total
eDTUs are needed. These distinctions in pricing and eDTU sharing are the basis of the price savings
potential that pools can provide.
The following rules of thumb related to database count and database utilization help to ensure that a pool
delivers reduced cost compared to using performance levels for single databases.
Minimum number of databases
If the sum of the DTUs of performance levels for single databases is more than 1.5x the eDTUs needed for the
pool, then an elastic pool is more cost effective. For available sizes, see eDTU and storage limits for elastic
pools and elastic databases.
Example
At least two S3 databases or at least 15 S0 databases are needed for a 100 eDTU pool to be more cost-
effective than using performance levels for single databases.
Maximum number of concurrently peaking databases
By sharing eDTUs, not all databases in a pool can simultaneously use eDTUs up to the limit available when
using performance levels for single databases. The fewer databases that concurrently peak, the lower the pool
eDTU can be set and the more cost-effective the pool becomes. In general, not more than 2/3 (or 67%) of the
databases in the pool should simultaneously peak to their eDTU limit.
Example
To reduce costs for three S3 databases in a 200 eDTU pool, at most two of these databases can
simultaneously peak in their utilization. Otherwise, if more than two of these four S3 databases
simultaneously peak, the pool would have to be sized to more than 200 eDTUs. If the pool is resized to more
than 200 eDTUs, more S3 databases would need to be added to the pool to keep costs lower than
performance levels for single databases.
Note this example does not consider utilization of other databases in the pool. If all databases have some
utilization at any given point in time, then less than 2/3 (or 67%) of the databases can peak simultaneously.
DTU utilization per database
A large difference between the peak and average utilization of a database indicates prolonged periods of low
utilization and short periods of high utilization. This utilization pattern is ideal for sharing resources across
databases. A database should be considered for a pool when its peak utilization is about 1.5 times greater
than its average utilization.
Example
An S3 database that peaks to 100 DTUs and on average uses 67 DTUs or less is a good candidate for sharing
eDTUs in a pool. Alternatively, an S1 database that peaks to 20 DTUs and on average uses 13 DTUs or less is a
good candidate for a pool.

Sizing an elastic pool

The best size for a pool depends on the aggregate eDTUs and storage resources needed for all databases in
the pool. This involves determining the larger of the following:
Maximum DTUs utilized by all databases in the pool.
Maximum storage bytes utilized by all databases in the pool.
For available sizes, see eDTU and storage limits for elastic pools and elastic databases.
SQL Database automatically evaluates the historical resource usage of databases in an existing SQL Database
server and recommends the appropriate pool configuration in the Azure portal. In addition to the
recommendations, a built-in experience estimates the eDTU usage for a custom group of databases on the
server. This enables you to do a "what-if" analysis by interactively adding databases to the pool and removing
them to get resource usage analysis and sizing advice before committing your changes. For a how-to, see
Monitor, manage, and size an elastic pool.
In cases where you can't use tooling, the following step-by-step can help you estimate whether a pool is
more cost-effective than single databases:
1. Estimate the eDTUs needed for the pool as follows:
MAX(<Total number of DBs X average DTU utilization per DB>,
<Number of concurrently peaking DBs X Peak DTU utilization per DB)
2. Estimate the storage space needed for the pool by adding the number of bytes needed for all the
databases in the pool. Then determine the eDTU pool size that provides this amount of storage. For pool
storage limits based on eDTU pool size, see eDTU and storage limits for elastic pools and elastic databases.
3. Take the larger of the eDTU estimates from Step 1 and Step 2.
4. See the SQL Database pricing page and find the smallest eDTU pool size that is greater than the estimate
from Step 3.
5. Compare the pool price from Step 5 to the price of using the appropriate performance levels for single
databases.

eDTU and storage limits for elastic pools

The following table describes the characteristics of Basic, Standard, Premium, and Premium RS elastic pools.
Basic elastic pool limits
POOL SIZE
(EDTUS) 50 100 200 300 400 800 1200 1600

Max data 5 GB 10 GB 20 GB 29 GB 39 GB 78 GB 117 GB 156 GB

storage
per pool*

Max In- N/A N/A N/A N/A N/A N/A N/A N/A
Memory
OLTP
storage
per pool*

Max 100 200 500 500 500 500 500 500

number
DBs per
pool

Max 100 200 400 600 800 1600 2400 3200

concurre
nt
workers
per pool

Max 100 200 400 600 800 1600 2400 3200

concurre
nt logins
per pool

Max 30000 30000 30000 30000 30000 30000 30000 30000

concurre
nt
sessions
per pool

Min {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5}

eDTUs
per
database
POOL SIZE
(EDTUS) 50 100 200 300 400 800 1200 1600

Max {5} {5} {5} {5} {5} {5} {5} {5}

eDTUs
per
database

Standard elastic pool limits

POOL SIZE
(EDTUS) 50 100 200 300 400 800

Max data 50 GB 100 GB 200 GB 300 GB 400 GB 800 GB

storage per
pool*

Max In- N/A N/A N/A N/A N/A N/A

Memory
OLTP storage
per pool*

Max number 100 200 500 500 500 500

DBs per pool

Max 100 200 400 600 800 1600

concurrent
workers per
pool

Max 100 200 400 600 800 1600

concurrent
logins per
pool

Max 30000 30000 30000 30000 30000 30000

concurrent
sessions per
pool

Min eDTUs {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20,

per database 50} 50,100} 50,100} 50,100} 50,100} 50,100}

Max eDTUs {10,20, {10,20, {10,20, {10,20, {10,20, {10,20,

per database 50} 50,100} 50,100} 50,100} 50,100} 50,100}

Standard elastic pool limits (continued)

POOL SIZE
(EDTUS) 1200 1600 2000 2500 3000

Max data 1.2 TB 1.6 TB 2 TB 2.4 TB 2.9 TB

storage per
pool*
POOL SIZE
(EDTUS) 1200 1600 2000 2500 3000

Max In-Memory N/A N/A N/A N/A N/A

OLTP storage
per pool*

Max number 500 500 500 500 500

DBs per pool

Max concurrent 2400 3200 4000 5000 6000

workers per
pool

Max concurrent 2400 3200 4000 5000 6000

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per
pool

Min eDTUs per {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20,

database 50,100} 50,100} 50,100} 50,100} 50,100}

Max eDTUs per {10,20, {10,20, {10,20, {10,20, {10,20,

database 50,100} 50,100} 50,100} 50,100} 50,100}

Premium elastic pool limits

POOL SIZE
(EDTUS) 125 250 500 1000 1500

Max data 250 GB 500 GB 750 GB 750 GB 750 GB

storage per
pool*

Max In-Memory 1 GB 2 GB 4 GB 10 GB 12 GB
OLTP storage
per pool*

Max number 50 100 100 100 100

DBs per pool

Max concurrent 200 400 800 1600 2400

workers per
pool

Max concurrent 200 400 800 1600 2400

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per
pool
POOL SIZE
(EDTUS) 125 250 500 1000 1500

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125} 125,250} 125,250,500} 125,250,500, 125,250,500,
1000} 1000,1500}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125} 125,250} 125,250,500} 125,250,500, 125,250,500,
1000} 1000,1500}

Premium elastic pool limits (continued)

POOL SIZE
(EDTUS) 2000 2500 3000 3500 4000

Max data 750 GB 750 GB 750 GB 750 GB 750 GB

storage per
pool*

Max In-Memory 16 GB 20 GB 24 GB 28 GB 32 GB
OLTP storage
per pool*

Max number 100 100 100 100 100

DBs per pool

Max concurrent 3200 4000 4800 5600 6400

workers per
pool

Max concurrent 3200 4000 4800 5600 6400

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per
pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125,250,500, 125,250,500, 125,250,500, 125,250,500, 125,250,500,
1000,1750} 1000,1750} 1000,1750} 1000,1750} 1000,1750,4000
}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125,250,500, 125,250,500, 125,250,500, 125,250,500, 125,250,500,
1000,1750} 1000,1750} 1000,1750} 1000,1750} 1000,1750,4000
}

Premium RS elastic pool limits

POOL SIZE (EDTUS) 125 250 500 1000

Max data storage 250 GB 500 GB 750 GB 750 GB

per pool*
POOL SIZE (EDTUS) 125 250 500 1000

Max In-Memory 1 GB 2 GB 4 GB 10 GB
OLTP storage per
pool*

Max number DBs 50 100 100 100

per pool

Max concurrent 200 400 800 1600

workers per pool

Max concurrent 200 400 800 1600

logins per pool

Max concurrent 30000 30000 30000 30000

sessions per pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125} 125,250} 125,250,500} 125,250,500,
1000}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125} 125,250} 125,250,500} 125,250,500,
1000}

IMPORTANT
* Pooled databases share pool storage, so data storage in an elastic pool is limited to the smaller of the remaining pool
storage or max storage per database.

If all DTUs of an elastic pool are used, then each database in the pool receives an equal amount of resources
to process queries. The SQL Database service provides resource sharing fairness between databases by
ensuring equal slices of compute time. Elastic pool resource sharing fairness is in addition to any amount of
resource otherwise guaranteed to each database when the DTU min per database is set to a non-zero value.

Database properties for pooled databases

The following table describes the properties for pooled databases.

PROPERTY DESCRIPTION
PROPERTY DESCRIPTION

Max eDTUs per database The maximum number of eDTUs that any database in the
pool may use, if available based on utilization by other
databases in the pool. Max eDTU per database is not a
resource guarantee for a database. This setting is a global
setting that applies to all databases in the pool. Set max
eDTUs per database high enough to handle peaks in
database utilization. Some degree of overcommitting is
expected since the pool generally assumes hot and cold
usage patterns for databases where all databases are not
simultaneously peaking. For example, suppose the peak
utilization per database is 20 eDTUs and only 20% of the
100 databases in the pool are peak at the same time. If the
eDTU max per database is set to 20 eDTUs, then it is
reasonable to overcommit the pool by 5 times, and set the
eDTUs per pool to 400.

Min eDTUs per database The minimum number of eDTUs that any database in the
pool is guaranteed. This setting is a global setting that
applies to all databases in the pool. The min eDTU per
database may be set to 0, and is also the default value. This
property is set to anywhere between 0 and the average
eDTU utilization per database. The product of the number
of databases in the pool and the min eDTUs per database
cannot exceed the eDTUs per pool. For example, if a pool
has 20 databases and the eDTU min per database set to
10 eDTUs, then the eDTUs per pool must be at least as
large as 200 eDTUs.

Max data storage per database The maximum storage for a database in a pool. Pooled
databases share pool storage, so database storage is
limited to the smaller of remaining pool storage and max
storage per database. Max storage per database refers to
the maximum size of the data files and does not include
the space used by log files.

Elastic jobs
With a pool, management tasks are simplified by running scripts in elastic jobs. An elastic job eliminates
most of tedium associated with large numbers of databases. To begin, see Getting started with Elastic jobs.
For more information about other database tools for working with multiple databases, see Scaling out with
Azure SQL Database.

Business continuity features for databases in a pool

Pooled databases generally support the same business continuity features that are available to single
databases.
Point in time restore
Point-in-time-restore uses automatic database backups to recover a database in a pool to a specific point in
time. See Point-In-Time Restore
Geo -Restore
Geo-Restore provides the default recovery option when a database is unavailable because of an incident in
the region where the database is hosted. See Restore an Azure SQL Database or failover to a secondary
Active Geo -Replication
For applications that have more aggressive recovery requirements than Geo-Restore can offer, configure
Active Geo-Replication.

Next steps
You can create and manage an elastic pool using the Azure portal, PowerShell, Transact-SQL, C#, and the
REST API.
For a video, see Microsoft Virtual Academy video course on Azure SQL Database elastic capabilities
To learn more about design patterns for SaaS applications using elastic pools, see Design Patterns for
Multi-tenant SaaS Applications with Azure SQL Database.
SQL Database options and performance:
Understand what's available in each service tier
4/20/2017 • 14 min to read • Edit Online

Azure SQL Database offers four service tiers: Basic, Standard, Premium, and Premium RS. Each service
tier has multiple performance levels to handle different workloads. Higher performance levels provide
additional resources designed to deliver increasingly higher throughput. You can change service tiers and
performance levels dynamically without downtime. Basic, Standard, and Premium service tiers all have an
uptime SLA of 99.99%, flexible business continuity options, security features, and hourly billing. The
Premium RS tier provides the same performance levels, security features and business continuity features as
the Premium tier albeit at a reduced SLA.

IMPORTANT
Premium RS databases run with a lower number of redundant copies than Premium or Standard databases. So, in the
event of a service failure, you may need to recover your database from a backup with up to a 5-minute lag.

You can create single databases with dedicated resources within a service tier at a specific performance level.
You can also create databases within an elastic pool in which the resources are shared across multiple
databases. The resources available for single databases are expressed in terms of Database Transaction Units
(DTUs) and for elastic pools in terms of elastic Database Transaction Units (eDTUs). For more on DTUs and
eDTUs, see What is a DTU?

Choosing a service tier

The following table provides examples of the tiers best suited for different application workloads.

SERVICE TIER TARGET WORKLOADS

Basic Best suited for a small database, supporting typically one

single active operation at a given time. Examples include
databases used for development or testing, or small-scale
infrequently used applications.

Standard The go-to option for cloud applications with low to

medium IO performance requirements, supporting
multiple concurrent queries. Examples include workgroup
or web applications.

Premium Designed for high transactional volume with high IO

performance requirements, supporting many concurrent
users. Examples are databases supporting mission critical
applications.

Premium RS Designed for IO-intensive workloads that do not require

the highest availability guarantees. Examples include
testing high-performance workloads, or an analytical
workload where the database is not the system of record.
First decide if you want to run a single database with a defined amount of dedicated resource or if you want
to share a pool of resources across a group of databases. Review the elastic pool considerations. To decide
on a service tier, start by determining the minimum database features that you need:

SERVICE TIER
FEATURES BASIC STANDARD PREMIUM PREMIUM RS

Maximum single 2 GB 250 GB 4 TB* 500 GB

database size

Maximum database 156 GB 2.9 TB 500 GB 500 GB

size in an elastic
pool

Maximum number 500 500 100 100

of databases per
pool

Database backup 7 days 35 days 35 days 35 days

retention period

IMPORTANT
Individual databases of up to 4 TB is public preview for customers using P11 and P15 performance levels at no
additional charge. Premium pools with more than 750 GB of storage are also currently available. These additional
storage options are currently available in the following regions: US East2, West US, West Europe, South East Asia,
Japan East, Australia East, Canada Central, and Canada East. See Current 4 TB limitations

Once you have determined the minimum service tier, you are ready to determine the performance level for
the database (the number of DTUs). The standard S2 and S3 performance levels are often a good starting
point. For databases with high CPU or IO requirements, the Premium performance levels are the right
starting point. Premium offers more CPU and starts at 10x more IO compared to the highest Standard
performance level.

Single database service tiers and performance levels

For single databases, there are multiple performance levels within each service tier. You have the flexibility
to choose the level that best meets your workload’s demands using the Azure portal, PowerShell, Transact-
SQL, C#, and the REST API.
Regardless of the number of databases hosted, your database gets a guaranteed set of resources and the
expected performance characteristics of your database are not affected.
Basic service tier
PERFORMANCE LEVEL BASIC

Max DTUs 5

Max database size* 2 GB

Max in-memory OLTP storage N/A

PERFORMANCE LEVEL BASIC

Max concurrent workers 30

Max concurrent logins 30

Max concurrent sessions 300

Standard service tier

PERFORMANCE LEVEL S0 S1 S2 S3

Max DTUs 10 20 50 100

Max database size* 250 GB 250 GB 250 GB 250 GB

Max in-memory N/A N/A N/A N/A

OLTP storage

Max concurrent 60 90 120 200

workers

Max concurrent 60 90 120 200

logins

Max concurrent 600 900 1200 2400

sessions

Premium service tier

PERFORMANC
E LEVEL P1 P2 P4 P6 P11 P15

Max DTUs 125 250 500 1000 1750 4000

Max 500 GB 500 GB 500 GB 500 GB 4 TB* 4 TB*

database
size*

Max in- 1 GB 2 GB 4 GB 8 GB 14 GB 32 GB
memory
OLTP
storage

Max 200 400 800 1600 2400 6400

concurrent
workers

Max 200 400 800 1600 2400 6400

concurrent
logins
PERFORMANC
E LEVEL P1 P2 P4 P6 P11 P15

Max 30000 30000 30000 30000 30000 30000

concurrent
sessions

Premium RS service tier

PERFORMANCE LEVEL PRS1 PRS2 PRS4 PRS6

Max DTUs 125 250 500 1000

Max database size* 500 GB 500 GB 500 GB 500 GB

Max in-memory 1 GB 2 GB 4 GB 8 GB
OLTP storage

Max concurrent 200 400 800 1600

workers

Max concurrent 200 400 800 1600

logins

Max concurrent 30000 30000 30000 30000

sessions

* Max database size refers to the maximum size of the data in the database.

NOTE
For a detailed explanation of all other rows in this service tiers table, see Service tier capabilities and limits.

Scaling up or scaling down a single database

After initially picking a service tier and performance level, you can scale a single database up or down
dynamically based on actual experience. If you need to scale up or down, you can easily change the tiers of
your database using the Azure portal, PowerShell, Transact-SQL, C#, and the REST API.

Changing the service tier and/or performance level of a database creates a replica of the original database at
the new performance level, and then switches connections over to the replica. No data is lost during this
process but during the brief moment when we switch over to the replica, connections to the database are
disabled, so some transactions in flight may be rolled back. This window varies, but is on average under 4
seconds, and in more than 99% of cases is less than 30 seconds. If there are large numbers of transactions in
flight at the moment connections are disabled, this window may be longer.
The duration of the entire scale-up process depends on both the size and service tier of the database before
and after the change. For example, a 250 GB database that is changing to, from, or within a Standard service
tier, should complete within 6 hours. For a database of the same size that is changing performance levels
within the Premium service tier, it should complete within 3 hours.
To downgrade a database, the database should be smaller than the maximum allowed size of the target
service tier.
When upgrading a database with Geo-Replication enabled, you must first upgrade its secondary
databases to the desired performance tier before upgrading the primary database.
When downgrading from a Premium service tier, you must first terminate all Geo-Replication
relationships. You can follow the steps described in the Recover from an outage topic to stop the
replication process between the primary and the active secondary databases.
The restore service offerings are different for the various service tiers. If you are downgrading you may
lose the ability to restore to a point in time, or have a lower backup retention period. For more
information, see Azure SQL Database Backup and Restore.
The new properties for the database are not applied until the changes are complete.

Elastic pool service tiers and performance in eDTUs

Pools allow databases to share and consume eDTU resources without needing to assign a specific
performance level to each database in the pool. For example, a single database in a Standard pool can go
from using 0 eDTUs to the maximum database eDTU you set up when you configure the pool. Pools allow
multiple databases with varying workloads to efficiently use eDTU resources available to the entire pool. See
Price and performance considerations for an elastic pool for details.
The following table describes the characteristics of pool service tiers.
Basic elastic pool limits
POOL SIZE
(EDTUS) 50 100 200 300 400 800 1200 1600

Max data 5 GB 10 GB 20 GB 29 GB 39 GB 78 GB 117 GB 156 GB

storage
per pool*

Max In- N/A N/A N/A N/A N/A N/A N/A N/A
Memory
OLTP
storage
per pool*

Max 100 200 500 500 500 500 500 500

number
DBs per
pool

Max 100 200 400 600 800 1600 2400 3200

concurre
nt
workers
per pool

Max 100 200 400 600 800 1600 2400 3200

concurre
nt logins
per pool
POOL SIZE
(EDTUS) 50 100 200 300 400 800 1200 1600

Max 30000 30000 30000 30000 30000 30000 30000 30000

concurre
nt
sessions
per pool

Min {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5}

eDTUs
per
database

Max {5} {5} {5} {5} {5} {5} {5} {5}

eDTUs
per
database

Standard elastic pool limits

POOL SIZE
(EDTUS) 50 100 200 300 400 800

Max data 50 GB 100 GB 200 GB 300 GB 400 GB 800 GB

storage per
pool*

Max In- N/A N/A N/A N/A N/A N/A

Memory
OLTP
storage per
pool*

Max number 100 200 500 500 500 500

DBs per pool

Max 100 200 400 600 800 1600

concurrent
workers per
pool

Max 100 200 400 600 800 1600

concurrent
logins per
pool

Max 30000 30000 30000 30000 30000 30000

concurrent
sessions per
pool
POOL SIZE
(EDTUS) 50 100 200 300 400 800

Min eDTUs {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20,

per database 50} 50,100} 50,100} 50,100} 50,100} 50,100}

Max eDTUs {10,20, {10,20, {10,20, {10,20, {10,20, {10,20,

per database 50} 50,100} 50,100} 50,100} 50,100} 50,100}

Standard elastic pool limits (continued)

POOL SIZE
(EDTUS) 1200 1600 2000 2500 3000

Max data 1.2 TB 1.6 TB 2 TB 2.4 TB 2.9 TB

storage per
pool*

Max In-Memory N/A N/A N/A N/A N/A

OLTP storage
per pool*

Max number 500 500 500 500 500

DBs per pool

Max concurrent 2400 3200 4000 5000 6000

workers per
pool

Max concurrent 2400 3200 4000 5000 6000

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per
pool

Min eDTUs per {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20,

database 50,100} 50,100} 50,100} 50,100} 50,100}

Max eDTUs per {10,20, {10,20, {10,20, {10,20, {10,20,

database 50,100} 50,100} 50,100} 50,100} 50,100}

Premium elastic pool limits

POOL SIZE
(EDTUS) 125 250 500 1000 1500

Max data 250 GB 500 GB 750 GB 750 GB 750 GB

storage per
pool*

Max In-Memory 1 GB 2 GB 4 GB 10 GB 12 GB
OLTP storage
per pool*
POOL SIZE
(EDTUS) 125 250 500 1000 1500

Max number 50 100 100 100 100

DBs per pool

Max concurrent 200 400 800 1600 2400

workers per
pool

Max concurrent 200 400 800 1600 2400

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per
pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125} 125,250} 125,250,500} 125,250,500, 125,250,500,
1000} 1000,1500}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125} 125,250} 125,250,500} 125,250,500, 125,250,500,
1000} 1000,1500}

Premium elastic pool limits (continued)

POOL SIZE
(EDTUS) 2000 2500 3000 3500 4000

Max data 750 GB 750 GB 750 GB 750 GB 750 GB

storage per
pool*

Max In-Memory 16 GB 20 GB 24 GB 28 GB 32 GB
OLTP storage
per pool*

Max number 100 100 100 100 100

DBs per pool

Max concurrent 3200 4000 4800 5600 6400

workers per
pool

Max concurrent 3200 4000 4800 5600 6400

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per
pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125,250,500, 125,250,500, 125,250,500, 125,250,500, 125,250,500,
1000,1750} 1000,1750} 1000,1750} 1000,1750} 1000,1750,400
0}
POOL SIZE
(EDTUS) 2000 2500 3000 3500 4000

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125,250,500, 125,250,500, 125,250,500, 125,250,500, 125,250,500,
1000,1750} 1000,1750} 1000,1750} 1000,1750} 1000,1750,400
0}

Premium RS elastic pool limits

POOL SIZE (EDTUS) 125 250 500 1000

Max data storage 250 GB 500 GB 750 GB 750 GB

per pool*

Max In-Memory 1 GB 2 GB 4 GB 10 GB
OLTP storage per
pool*

Max number DBs 50 100 100 100

per pool

Max concurrent 200 400 800 1600

workers per pool

Max concurrent 200 400 800 1600

logins per pool

Max concurrent 30000 30000 30000 30000

sessions per pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125} 125,250} 125,250,500} 125,250,500,
1000}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125} 125,250} 125,250,500} 125,250,500,
1000}

IMPORTANT
* Pooled databases share pool storage, so data storage in an elastic pool is limited to the smaller of the remaining
pool storage or max storage per database.

Each database within a pool also adheres to the single database characteristics for that tier. For example, the
Basic pool has a limit for max sessions per pool of 4800 - 28800, but an individual database within a Basic
pool has a database limit of 300 sessions.

Scaling up or scaling down an elastic pool

After initially picking a service tier and performance level, you can scale the elastic pool up or down
dynamically based on actual experience.
Changing the min eDTUs per database or max eDTUs per database typically completes in five minutes or
less.
Time to change the pool size (eDTUs) depends on the combined size of all databases in the pool. Changes
average 90 minutes or less per 100 GB. For example, if the total space of all databases in the pool is 200
GB, then the expected latency for changing the pool eDTU per pool is 3 hours or less.
For detailed steps, see Manage an elastic pool in the Azure portal, Manage an elastic pool with Powershell,
Manage an elastic pool with Transact-SQL, or Manage an elastic pool with C#.

Creating or upgrading to 4TB

The following sections discuss implementation details for the 4 TB option.
Creating in the Azure portal
When creating a P11/P15, the default storage option of 1TB is pre-selected. For databases located in one of
the supported regions, you can increase the storage maximum to 4TB. For all other regions, the storage
slider cannot be changed. The price does not change when you select 4 TB of included storage.
Creating using PowerShell or Transact-SQL
When creating a P11/P15 database, you can set the maxsize value to either 1 TB (default) or 4 TB. Values of
‘1024 GB’ and ‘4096 GB’ are also accepted. If you choose the 4 TB maxsize option, the create command will
fail with an error if the database is provisioned in an unsupported region.
Upgrading to 4TB
For existing P11 and P15 databases located in one of the supported regions, you can increase the maxsize
storage to 4 TB. This can be done in the Azure Portal, in PowerShell or with Transact-SQL. The following
example shows the maxsize being changed using the ALTER DATABASE command:

ALTER DATABASE <myDatabaseName>

MODIFY (MAXSIZE = 4096 GB);

Upgrading an existing P11 or P15 database can only be performed by a server-level principal login or by
members of the dbmanager database role. If executed in a supported region the configuration will be
updated immediately. This can be checked using the SELECT DATABASEPROPERTYEX or by inspecting the
database size in the Azure portal. The database will remain online during the upgrade process. However, you
will not be able to utilize the full 4 TB of storage until the actual database files have been upgraded to the
new maxsize. The length of time required depends upon on the size of the database being upgraded.
Error messages
When creating or upgrading an P11/P15 database in an unsupported region, the create or upgrade
operation will fail with the following error message: P11 and P15 database with up to 4TB of storage are
available in US East 2, West US, South East Asia, West Europe, Canada East, Canada Central, Japan
East, and Australia East.

Current limitations of P11 and P15 databases with 4 TB maxsize

When creating or updating a P11 or P15 database, you can only chose between 1 TB and 4 TB maxsize.
Intermediate storage sizes are not currently supported.
The 4 TB database maxsize cannot be changed to 1 TB even if the actual storage used is below 1 TB. Thus,
you cannot downgrade a P11-4TB/P15-4TB to a P11-1TB/P15-1TB or a lower performance tier (e.g., to
P1-P6) until we are providing additional storage options for the rest of the performance tiers. This
restriction also applies to the restore and copy scenarios including point-in-time, geo-restore, long-term-
backup-retention, and database copy. Once a database is configured with the 4 TB option, all restore
operations of this database must be into a P11/P15 with 4 TB maxsize.
For Active Geo-Replication scenarios:
Setting up a geo-replication relationship: If the primary database is P11 or P15, the secondary(ies)
must also be P11 or P15; lower performance tiers will be rejected as secondaries since they are
not capable of supporting 4 TB.
Upgrading the primary database in a geo-replication relationship: Changing the maxsize to 4 TB
on a primary database will trigger the same change on the secondary database. Both upgrades
must be successful for the change on the primary to take effect. Region limitations for the 4TB
option apply (see above). If the secondary is in a region that does not support 4 TB, the primary
will not be upgraded.
Using the Import/Export service for loading P11-4TB/P15-4TB databases is not supported. Use
SqlPackage.exe to import and export data.

Next steps
Learn the details of elastic pools and price and performance considerations for elastic pools.
Learn how to Monitor, manage, and resize elastic pools and Monitor the performance of single
databases.
Now that you know about the SQL Database tiers, try them out with a free account and learn how to
create your first SQL database.
For migration scenarios, use the DTU Calculator to approximate the number of DTUs needed.
Explaining Database Transaction Units (DTUs) and
elastic Database Transaction Units (eDTUs)
4/19/2017 • 5 min to read • Edit Online

This article explains Database Transaction Units (DTUs) and elastic Database Transaction Units (eDTUs) and what
happens when you hit the maximum DTUs or eDTUs.

What are Database Transaction Units (DTUs)

For a single Azure SQL database at a specific performance level within a service tier, Microsoft guarantees a
certain level of resources for that database (independent of any other database in the Azure cloud) and providing
a predictable level of performance. This amount of resources is calculated as a number of Database Transaction
Units or DTUs, and is a blended measure of CPU, memory, I/O (data and transaction log I/O). The ratio amongst
these resources was originally determined by an OLTP benchmark workload designed to be typical of real-world
OLTP workloads. When your workload exceeds the amount of any of these resources, your throughput is throttled
- resulting in slower performance and timeouts. The resources used by your workload do not impact the
resources available to other SQL databases in the Azure cloud, and the resource used by other workloads do not
impact the resources available to your SQL database.

DTUs are most useful for understanding the relative amount of resources between Azure SQL Databases at
different performance levels and service tiers. For example, doubling the DTUs by increasing the performance
level of a database equates to doubling the set of resource available to that database. For example, a Premium
P11 database with 1750 DTUs provides 350x more DTU compute power than a Basic database with 5 DTUs.
To gain deeper insight into the resource (DTU) consumption of your workload, use Azure SQL Database Query
Performance Insight to:
Identify the top queries by CPU/Duration/Execution count that can potentially be tuned for improved
performance. For example, an I/O intensive query might benefit from the use of in-memory optimization
techniques to make better use of the available memory at a certain service tier and performance level.
Drill down into the details of a query, view its text and history of resource utilization.
Access performance tuning recommendations that show actions performed by SQL Database Advisor.
You can change service tiers at any time with minimal downtime to your application (generally averaging under
four seconds). For many businesses and apps, being able to create databases and dial performance up or down on
demand is enough, especially if usage patterns are relatively predictable. But if you have unpredictable usage
patterns, it can make it hard to manage costs and your business model. For this scenario, you use an elastic pool
with a certain number of eDTUs that are shared among multiple database in the pool.

What are elastic Database Transaction Units (eDTUs)

Rather than provide a dedicated set of resources (DTUs) to a SQL Database that is always available regardless of
whether needed not, you can place databases into an elastic pool on a SQL Database server that shares a pool of
resources among those database. The shared resources in an elastic pool measured by elastic Database
Transaction Units or eDTUs. Elastic pools provide a simple cost effective solution to manage the performance
goals for multiple databases that have widely varying and unpredictable usage patterns. In an elastic pool, you can
guarantee that no one database uses all of the resources in the pool and also that a minimum amount of
resources is always available to a database in an elastic pool. See elastic pools and service tiers for more
information.

A pool is given a set number of eDTUs, for a set price. Within the elastic pool, individual databases are given the
flexibility to auto-scale within the configured boundaries. Under heavy load, a database can consume more eDTUs
to meet demand while databases under light loads consume less, up to the point that databases under no load
consume no eDTUs. By provisioning resources for the entire pool, rather than per database, management tasks
are simplified and you have a predictable budget for the pool.
Additional eDTUs can be added to an existing pool with no database downtime and with no impact on the
databases in the pool. Similarly, if extra eDTUs are no longer needed, they can be removed from an existing pool
at any point in time. You can add or subtract databases to the pool, or limit the amount of eDTUs a database can
use under heavy load to reserve eDTUs for other databases. If a database is predictably under-utilizing resources,
you can move it out of the pool and configure it as a single database with predictable amount of resources it
requires.

How can I determine the number of DTUs needed by my workload?

If you are looking to migrate an existing on-premises or SQL Server virtual machine workload to Azure SQL
Database, you can use the DTU Calculator to approximate the number of DTUs needed. For an existing Azure SQL
Database workload, you can use SQL Database Query Performance Insight to understand your database resource
consumption (DTUs) to get deeper insight into how to optimize your workload. You can also use the sys.dm_db_
resource_stats DMV to get the resource consumption information for the last one hour. Alternatively, the catalog
view sys.resource_stats can also be queried to get the same data for the last 14 days, although at a lower fidelity
of five-minute averages.

How do I know if I could benefit from an elastic pool of resources?

Pools are suited for a large number of databases with specific utilization patterns. For a given database, this
pattern is characterized by low average utilization with relatively infrequent utilization spikes. SQL Database
automatically evaluates the historical resource usage of databases in an existing SQL Database server and
recommends the appropriate pool configuration in the Azure portal. For more information, see when should an
elastic pool be used?

What happens when I hit my maximum DTUs

Performance levels are calibrated and governed to provide the needed resources to run your database workload
up to the max limits allowed for your selected service tier/performance level. If your workload is hitting the limits
in one of CPU/Data IO/Log IO limits, you continue to receive the resources at the maximum allowed level, but you
are likely to see increased latencies for your queries. These limits do not result in any errors, but rather a
slowdown in the workload, unless the slowdown becomes so severe that queries start timing out. If you are
hitting limits of maximum allowed concurrent user sessions/requests (worker threads), you see explicit errors. See
Azure SQL Database resource limits for information on limit on resources other than CPU, memory, data I/O, and
transaction log I/O.

Next steps
See Service tier for information on the DTUs and eDTUs available for single databases and for elastic pools.
See Azure SQL Database resource limits for information on limit on resources other than CPU, memory, data
I/O, and transaction log I/O.
See SQL Database Query Performance Insight to understand your (DTUs) consumption.
See SQL Database benchmark overview to understand the methodology behind the OLTP benchmark
workload used to determine the DTU blend.
Azure SQL Database benchmark overview
4/14/2017 • 7 min to read • Edit Online

Overview
Microsoft Azure SQL Database offers three service tiers with multiple performance levels. Each performance level
provides an increasing set of resources, or ‘power’, designed to deliver increasingly higher throughput.
It is important to be able to quantify how the increasing power of each performance level translates into increased
database performance. To do this Microsoft has developed the Azure SQL Database Benchmark (ASDB). The
benchmark exercises a mix of basic operations found in all OLTP workloads. We measure the throughput achieved
for databases running in each performance level.
The resources and power of each service tier and performance level are expressed in terms of Database
Transaction Units (DTUs). DTUs provide a way to describe the relative capacity of a performance level based on a
blended measure of CPU, memory, and read and write rates offered by each performance level. Doubling the DTU
rating of a database equates to doubling the database power. The benchmark allows us to assess the impact on
database performance of the increasing power offered by each performance level by exercising actual database
operations, while scaling database size, number of users, and transaction rates in proportion to the resources
provided to the database.
By expressing the throughput of the Basic service tier using transactions per-hour, the Standard service tier using
transactions per-minute, and the Premium service tier using transactions per-second, it makes it easier to quickly
relate the performance potential of each service tier to the requirements of an application.

Correlating benchmark results to real world database performance

It is important to understand that ASDB, like all benchmarks, is representative and indicative only. The transaction
rates achieved with the benchmark application will not be the same as those that might be achieved with other
applications. The benchmark comprises a collection of different transaction types run against a schema containing
a range of tables and data types. While the benchmark exercises the same basic operations that are common to all
OLTP workloads, it does not represent any specific class of database or application. The goal of the benchmark is to
provide a reasonable guide to the relative performance of a database that might be expected when scaling up or
down between performance levels. In reality, databases are of different sizes and complexity, encounter different
mixes of workloads, and will respond in different ways. For example, an IO-intensive application may hit IO
thresholds sooner, or a CPU-intensive application may hit CPU limits sooner. There is no guarantee that any
particular database will scale in the same way as the benchmark under increasing load.
The benchmark and its methodology are described in more detail below.

Benchmark summary
ASDB measures the performance of a mix of basic database operations which occur most frequently in online
transaction processing (OLTP) workloads. Although the benchmark is designed with cloud computing in mind, the
database schema, data population, and transactions have been designed to be broadly representative of the basic
elements most commonly used in OLTP workloads.

Schema
The schema is designed to have enough variety and complexity to support a broad range of operations. The
benchmark runs against a database comprised of six tables. The tables fall into three categories: fixed-size, scaling,
and growing. There are two fixed-size tables; three scaling tables; and one growing table. Fixed-size tables have a
constant number of rows. Scaling tables have a cardinality that is proportional to database performance, but
doesn’t change during the benchmark. The growing table is sized like a scaling table on initial load, but then the
cardinality changes in the course of running the benchmark as rows are inserted and deleted.
The schema includes a mix of data types, including integer, numeric, character, and date/time. The schema includes
primary and secondary keys, but not any foreign keys - that is, there are no referential integrity constraints
between tables.
A data generation program generates the data for the initial database. Integer and numeric data is generated with
various strategies. In some cases, values are distributed randomly over a range. In other cases, a set of values is
randomly permuted to ensure that a specific distribution is maintained. Text fields are generated from a weighted
list of words to produce realistic looking data.
The database is sized based on a “scale factor.” The scale factor (abbreviated as SF) determines the cardinality of
the scaling and growing tables. As described below in the section Users and Pacing, the database size, number of
users, and maximum performance all scale in proportion to each other.

Transactions
The workload consists of nine transaction types, as shown in the table below. Each transaction is designed to
highlight a particular set of system characteristics in the database engine and system hardware, with high contrast
from the other transactions. This approach makes it easier to assess the impact of different components to overall
performance. For example, the transaction “Read Heavy” produces a significant number of read operations from
disk.

TRANSACTION TYPE DESCRIPTION

Read Lite SELECT; in-memory; read-only

Read Medium SELECT; mostly in-memory; read-only

Read Heavy SELECT; mostly not in-memory; read-only

Update Lite UPDATE; in-memory; read-write

Update Heavy UPDATE; mostly not in-memory; read-write

Insert Lite INSERT; in-memory; read-write

Insert Heavy INSERT; mostly not in-memory; read-write

Delete DELETE; mix of in-memory and not in-memory; read-write

CPU Heavy SELECT; in-memory; relatively heavy CPU load; read-only

Workload mix
Transactions are selected at random from a weighted distribution with the following overall mix. The overall mix
has a read/write ratio of approximately 2:1.
TRANSACTION TYPE % OF MIX

Read Lite 35

Read Medium 20

Read Heavy 5

Update Lite 20

Update Heavy 3

Insert Lite 3

Insert Heavy 2

Delete 2

CPU Heavy 10

Users and pacing

The benchmark workload is driven from a tool that submits transactions across a set of connections to simulate
the behavior of a number of concurrent users. Although all of the connections and transactions are machine
generated, for simplicity we refer to these connections as “users.” Although each user operates independently of all
other users, all users perform the same cycle of steps shown below:
1. Establish a database connection.
2. Repeat until signaled to exit:
Select a transaction at random (from a weighted distribution).
Perform the selected transaction and measure the response time.
Wait for a pacing delay.
3. Close the database connection.
4. Exit.
The pacing delay (in step 2c) is selected at random, but with a distribution that has an average of 1.0 second. Thus
each user can, on average, generate at most one transaction per second.

Scaling rules
The number of users is determined by the database size (in scale-factor units). There is one user for every five
scale-factor units. Because of the pacing delay, one user can generate at most one transaction per second, on
average.
For example, a scale-factor of 500 (SF=500) database will have 100 users and can achieve a maximum rate of 100
TPS. To drive a higher TPS rate requires more users and a larger database.
The table below shows the number of users actually sustained for each service tier and performance level.

SERVICE TIER (PERFORMANCE LEVEL) USERS DATABASE SIZE

Basic 5 720 MB
SERVICE TIER (PERFORMANCE LEVEL) USERS DATABASE SIZE

Standard (S0) 10 1 GB

Standard (S1) 20 2.1 GB

Standard (S2) 50 7.1 GB

Premium (P1) 100 14 GB

Premium (P2) 200 28 GB

Premium (P6/P3) 800 114 GB

Measurement duration
A valid benchmark run requires a steady-state measurement duration of at least one hour.

Metrics
The key metrics in the benchmark are throughput and response time.
Throughput is the essential performance measure in the benchmark. Throughput is reported in transactions per
unit-of-time, counting all transaction types.
Response time is a measure of performance predictability. The response time constraint varies with class of
service, with higher classes of service having a more stringent response time requirement, as shown below.

CLASS OF SERVICE THROUGHPUT MEASURE RESPONSE TIME REQUIREMENT

Premium Transactions per second 95th percentile at 0.5 seconds

Standard Transactions per minute 90th percentile at 1.0 seconds

Basic Transactions per hour 80th percentile at 2.0 seconds

Conclusion
The Azure SQL Database Benchmark measures the relative performance of Azure SQL Database running across the
range of available service tiers and performance levels. The benchmark exercises a mix of basic database
operations which occur most frequently in online transaction processing (OLTP) workloads. By measuring actual
performance, the benchmark provides a more meaningful assessment of the impact on throughput of changing
the performance level than is possible by just listing the resources provided by each level such as CPU speed,
memory size, and IOPS. In the future, we will continue to evolve the benchmark to broaden its scope and expand
the data provided.

Resources
Introduction to SQL Database
Service tiers and performance levels
Performance guidance for single databases
Azure SQL Database resource limits
4/19/2017 • 8 min to read • Edit Online

Overview
Azure SQL Database manages the resources available to a database using two different mechanisms: Resources
Governance and Enforcement of Limits. This topic explains these two main areas of resource management.

Resource governance
One of the design goals of the Basic, Standard, Premium, and Premium RS service tiers is for Azure SQL Database
to behave as if the database is running on its own machine, isolated from other databases. Resource governance
emulates this behavior. If the aggregated resource utilization reaches the maximum available CPU, Memory, Log
I/O, and Data I/O resources assigned to the database, resource governance queues queries in execution and
assign resources to the queued queries as they free up.
As on a dedicated machine, utilizing all available resources results in a longer execution of currently executing
queries, which can result in command timeouts on the client. Applications with aggressive retry logic and
applications that execute queries against the database with a high frequency can encounter errors messages when
trying to execute new queries when the limit of concurrent requests has been reached.
Recommendations:
Monitor the resource utilization and the average response times of queries when nearing the maximum utilization
of a database. When encountering higher query latencies you generally have three options:
1. Reduce the number of incoming requests to the database to prevent timeout and the pile up of requests.
2. Assign a higher performance level to the database.
3. Optimize queries to reduce the resource utilization of each query. For more information, see the Query
Tuning/Hinting section in the Azure SQL Database Performance Guidance article.

Enforcement of limits
Resources other than CPU, Memory, Log I/O, and Data I/O are enforced by denying new requests when limits are
reached. When a database reaches the configured maximum size limit, inserts and updates that increase data size
fail, while selects and deletes continue to work. Clients receive an error message depending on the limit that has
been reached.
For example, the number of connections to a SQL database and the number of concurrent requests that can be
processed are restricted. SQL Database allows the number of connections to the database to be greater than the
number of concurrent requests to support connection pooling. While the number of connections that are available
can easily be controlled by the application, the number of parallel requests is often times harder to estimate and
to control. Especially during peak loads when the application either sends too many requests or the database
reaches its resource limits and starts piling up worker threads due to longer running queries, errors can be
encountered.

Service tiers and performance levels

There are service tiers and performance levels for both single database and elastic pools.
Single databases
For a single database, the limits of a database are defined by the database service tier and performance level. The
following table describes the characteristics of Basic, Standard, Premium, and Premium RS databases at varying
performance levels.
Basic service tier
PERFORMANCE LEVEL BASIC

Max DTUs 5

Max database size* 2 GB

Max in-memory OLTP storage N/A

Max concurrent workers 30

Max concurrent logins 30

Max concurrent sessions 300

Standard service tier

PERFORMANCE LEVEL S0 S1 S2 S3

Max DTUs 10 20 50 100

Max database size* 250 GB 250 GB 250 GB 250 GB

Max in-memory N/A N/A N/A N/A

OLTP storage

Max concurrent 60 90 120 200

workers

Max concurrent 60 90 120 200

logins

Max concurrent 600 900 1200 2400

sessions

Premium service tier

PERFORMANCE
LEVEL P1 P2 P4 P6 P11 P15

Max DTUs 125 250 500 1000 1750 4000

Max database 500 GB 500 GB 500 GB 500 GB 4 TB* 4 TB*

size*

Max in- 1 GB 2 GB 4 GB 8 GB 14 GB 32 GB
memory OLTP
storage
PERFORMANCE
LEVEL P1 P2 P4 P6 P11 P15

Max 200 400 800 1600 2400 6400

concurrent
workers

Max 200 400 800 1600 2400 6400

concurrent
logins

Max 30000 30000 30000 30000 30000 30000

concurrent
sessions

Premium RS service tier

PERFORMANCE LEVEL PRS1 PRS2 PRS4 PRS6

Max DTUs 125 250 500 1000

Max database size* 500 GB 500 GB 500 GB 500 GB

Max in-memory 1 GB 2 GB 4 GB 8 GB
OLTP storage

Max concurrent 200 400 800 1600

workers

Max concurrent 200 400 800 1600

logins

Max concurrent 30000 30000 30000 30000

sessions

* Max database size refers to the maximum size of the data in the database.

IMPORTANT
Customers using P11 and P15 performance levels can use up to 4 TB of included storage at no additional charge. This 4 TB
option is currently available in the following regions: US East2, West US, West Europe, South East Asia, Japan East, Australia
East, Canada Central, and Canada East.

Elastic pools
Elastic pools share resources across databases in the pool. The following table describes the characteristics of
Basic, Standard, Premium, and Premium RS elastic pools.
Basic elastic pool limits
POOL SIZE
(EDTUS) 50 100 200 300 400 800 1200 1600

Max data 5 GB 10 GB 20 GB 29 GB 39 GB 78 GB 117 GB 156 GB

storage
per pool*

Max In- N/A N/A N/A N/A N/A N/A N/A N/A
Memory
OLTP
storage
per pool*

Max 100 200 500 500 500 500 500 500

number
DBs per
pool

Max 100 200 400 600 800 1600 2400 3200

concurren
t workers
per pool

Max 100 200 400 600 800 1600 2400 3200

concurren
t logins
per pool

Max 30000 30000 30000 30000 30000 30000 30000 30000

concurren
t sessions
per pool

Min {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5} {0, 5}

eDTUs
per
database

Max {5} {5} {5} {5} {5} {5} {5} {5}

eDTUs
per
database

Standard elastic pool limits

POOL SIZE
(EDTUS) 50 100 200 300 400 800

Max data 50 GB 100 GB 200 GB 300 GB 400 GB 800 GB

storage per
pool*

Max In- N/A N/A N/A N/A N/A N/A

Memory
OLTP storage
per pool*
POOL SIZE
(EDTUS) 50 100 200 300 400 800

Max number 100 200 500 500 500 500

DBs per pool

Max 100 200 400 600 800 1600

concurrent
workers per
pool

Max 100 200 400 600 800 1600

concurrent
logins per
pool

Max 30000 30000 30000 30000 30000 30000

concurrent
sessions per
pool

Min eDTUs {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20,

per database 50} 50,100} 50,100} 50,100} 50,100} 50,100}

Max eDTUs {10,20, {10,20, {10,20, {10,20, {10,20, {10,20,

per database 50} 50,100} 50,100} 50,100} 50,100} 50,100}

Standard elastic pool limits (continued)

POOL SIZE (EDTUS) 1200 1600 2000 2500 3000

Max data storage 1.2 TB 1.6 TB 2 TB 2.4 TB 2.9 TB

per pool*

Max In-Memory N/A N/A N/A N/A N/A

OLTP storage per
pool*

Max number DBs 500 500 500 500 500

per pool

Max concurrent 2400 3200 4000 5000 6000

workers per pool

Max concurrent 2400 3200 4000 5000 6000

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per pool

Min eDTUs per {0,10,20, {0,10,20, {0,10,20, {0,10,20, {0,10,20,

database 50,100} 50,100} 50,100} 50,100} 50,100}

Max eDTUs per {10,20, {10,20, {10,20, {10,20, {10,20,

database 50,100} 50,100} 50,100} 50,100} 50,100}
POOL SIZE (EDTUS) 1200 1600 2000 2500 3000

Premium elastic pool limits

POOL SIZE (EDTUS) 125 250 500 1000 1500

Max data storage 250 GB 500 GB 750 GB 750 GB 750 GB

per pool*

Max In-Memory 1 GB 2 GB 4 GB 10 GB 12 GB
OLTP storage per
pool*

Max number DBs 50 100 100 100 100

per pool

Max concurrent 200 400 800 1600 2400

workers per pool

Max concurrent 200 400 800 1600 2400

logins per pool

Max concurrent 30000 30000 30000 30000 30000

sessions per pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125} 125,250} 125,250,500} 125,250,500, 125,250,500,
1000} 1000,1500}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125} 125,250} 125,250,500} 125,250,500, 125,250,500,
1000} 1000,1500}

Premium elastic pool limits (continued)

POOL SIZE (EDTUS) 2000 2500 3000 3500 4000

Max data storage 750 GB 750 GB 750 GB 750 GB 750 GB

per pool*

Max In-Memory 16 GB 20 GB 24 GB 28 GB 32 GB
OLTP storage per
pool*

Max number DBs 100 100 100 100 100

per pool

Max concurrent 3200 4000 4800 5600 6400

workers per pool

Max concurrent 3200 4000 4800 5600 6400

logins per pool
POOL SIZE (EDTUS) 2000 2500 3000 3500 4000

Max concurrent 30000 30000 30000 30000 30000

sessions per pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125,250,500, 125,250,500, 125,250,500, 125,250,500, 125,250,500,
1000,1750} 1000,1750} 1000,1750} 1000,1750} 1000,1750,4000}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125,250,500, 125,250,500, 125,250,500, 125,250,500, 125,250,500,
1000,1750} 1000,1750} 1000,1750} 1000,1750} 1000,1750,4000}

Premium RS elastic pool limits

POOL SIZE (EDTUS) 125 250 500 1000

Max data storage per 250 GB 500 GB 750 GB 750 GB

pool*

Max In-Memory 1 GB 2 GB 4 GB 10 GB
OLTP storage per
pool*

Max number DBs per 50 100 100 100

pool

Max concurrent 200 400 800 1600

workers per pool

Max concurrent 200 400 800 1600

logins per pool

Max concurrent 30000 30000 30000 30000

sessions per pool

Min eDTUs per {0,25,50,75, {0,25,50,75, {0,25,50,75, {0,25,50,75,

database 125} 125,250} 125,250,500} 125,250,500,
1000}

Max eDTUs per {25,50,75, {25,50,75, {25,50,75, {25,50,75,

database 125} 125,250} 125,250,500} 125,250,500,
1000}

IMPORTANT
* Pooled databases share pool storage, so data storage in an elastic pool is limited to the smaller of the remaining pool
storage or max storage per database.

For an expanded definition of each resource listed in the previous tables, see the descriptions in Service tier
capabilities and limits. For an overview of service tiers, see Azure SQL Database Service Tiers and Performance
Levels.
Other SQL Database limits
AREA LIMIT DESCRIPTION

Databases using Automated export per 10 Automated export allows you to create
subscription a custom schedule for backing up your
SQL databases. The preview of this
feature will end on March 1, 2017.

Databases per server Up to 5000 Up to 5000 databases are allowed per

server.

DTUs per server 45000 45000 DTUs are allowed per server for
provisioning standalone databases and
elastic pools. The total number of
standalone databases and pools
allowed per server is limited only by the
number of server DTUs.

IMPORTANT
Azure SQL Database Automated Export is now in preview and will be retired on March 1, 2017. Starting December 1st,
2016, you will no longer be able to configure automated export on any SQL database. All your existing automated export
jobs will continue to work until March 1st, 2017. After December 1st, 2016, you can use long-term backup retention or
Azure Automation to archive SQL databases periodically using PowerShell periodically according to a schedule of your
choice. For a sample script, you can download the sample script from GitHub.

Resources
Azure Subscription and Service Limits, Quotas, and Constraints
Azure SQL Database Service Tiers and Performance Levels
Error messages for SQL Database client programs
Choose a cloud SQL Server option: Azure SQL (PaaS)
Database or SQL Server on Azure VMs (IaaS)
4/14/2017 • 13 min to read • Edit Online

Azure has two options for hosting SQL Server workloads in Microsoft Azure:
Azure SQL Database: A SQL database native to the cloud, also known as a platform as a service (PaaS) database
or a database as a service (DBaaS) that is optimized for software-as-a-service (SaaS) app development. It offers
compatibility with most SQL Server features. For more information on PaaS, see What is PaaS.
SQL Server on Azure Virtual Machines: SQL Server installed and hosted in the cloud on Windows Server Virtual
Machines (VMs) running on Azure, also known as an infrastructure as a service (IaaS). SQL Server on Azure
virtual machines is optimized for migrating existing SQL Server applications. All the versions and editions of
SQL Server are available. It offers 100% compatibility with SQL Server, allowing you to host as many databases
as needed and executing cross-database transactions. It offers full control on SQL Server and Windows.
Learn how each option fits into the Microsoft data platform and get help matching the right option to your business
requirements. Whether you prioritize cost savings or minimal administration ahead of everything else, this article
can help you decide which approach delivers against the business requirements you care about most.

Microsoft's data platform

One of the first things to understand in any discussion of Azure versus on-premises SQL Server databases is that
you can use it all. Microsoft’s data platform leverages SQL Server technology and makes it available across physical
on-premises machines, private cloud environments, third-party hosted private cloud environments, and public
cloud. SQL Server on Azure virtual machines enables you to meet unique and diverse business needs through a
combination of on-premises and cloud-hosted deployments, while using the same set of server products,
development tools, and expertise across these environments.

As seen in the diagram, each offering can be characterized by the level of administration you have over the
infrastructure (on the X axis), and by the degree of cost efficiency achieved by database level consolidation and
automation (on the Y axis).
When designing an application, four basic options are available for hosting the SQL Server part of the application:
SQL Server on non-virtualized physical machines
SQL Server in on-premises virtualized machines (private cloud)
SQL Server in Azure Virtual Machine (Microsoft public cloud)
Azure SQL Database (Microsoft public cloud)
In the following sections, you learn about SQL Server in the Microsoft public cloud: Azure SQL Database and SQL
Server on Azure VMs. In addition, you explore common business motivators for determining which option works
best for your application.

A closer look at Azure SQL Database and SQL Server on Azure VMs
Azure SQL Database is a relational database-as-a-service (DBaaS) hosted in the Azure cloud that falls into the
industry categories of Software-as-a-Service (SaaS) and Platform-as-a-Service (PaaS). SQL database is built on
standardized hardware and software that is owned, hosted, and maintained by Microsoft. With SQL Database, you
can develop directly on the service using built-in features and functionality. When using SQL Database, you pay-as-
you-go with options to scale up or out for greater power with no interruption.
SQL Server on Azure Virtual Machines (VMs) falls into the industry category Infrastructure-as-a-Service (IaaS)
and allows you to run SQL Server inside a virtual machine in the cloud. Similar to SQL Database, it is built on
standardized hardware that is owned, hosted, and maintained by Microsoft. When using SQL Server on a VM, you
can either pay-as you-go for a SQL Server license already included in a SQL Server image or easily use an existing
license. You can also easily scale-up/down and pause/resume the VM as needed.
In general, these two SQL options are optimized for different purposes:
Azure SQL Database is optimized to reduce overall costs to the minimum for provisioning and managing
many databases. It reduces ongoing administration costs because you do not have to manage any virtual
machines, operating system or database software. You do not have to manage upgrades, high availability, or
backups. In general, Azure SQL Database can dramatically increase the number of databases managed by a
single IT or development resource.
SQL Server running on Azure VMs is optimized for migrating existing applications to Azure or extending
existing on-premises applications to the cloud in hybrid deployments. In addition, you can use SQL Server in a
virtual machine to develop and test traditional SQL Server applications. With SQL Server on Azure VMs, you
have the full administrative rights over a dedicated SQL Server instance and a cloud-based VM. It is a perfect
choice when an organization already has IT resources available to maintain the virtual machines. These
capabilities allow you to build a highly customized system to address your application’s specific performance
and availability requirements.
The following table summarizes the main characteristics of SQL Database and SQL Server on Azure VMs:

SQL SERVER IN AN AZURE VIRTUAL

BEST FOR: AZURE SQL DATABASE MACHINE

New cloud-designed applications that Existing applications that require fast

have time constraints in development migration to the cloud with minimal
and marketing. changes. Rapid development and test
scenarios when you do not want to buy
on-premises non-production SQL
Server hardware.

Teams that need built-in high Teams that can configure and manage
availability, disaster recovery, and high availability, disaster recovery, and
upgrade for the database. patching for SQL Server. Some provided
automated features dramatically
simplify this.
SQL SERVER IN AN AZURE VIRTUAL
BEST FOR: AZURE SQL DATABASE MACHINE

Teams that do not want to manage the If you need a customized environment
underlying operating system and with full administrative rights.
configuration settings.

Databases of up to 1 TB, or larger SQL Server instances with up to 64 TB

databases that can be horizontally or of storage. The instance can support as
vertically partitioned using a scale-out many databases as needed.
pattern.

Building Software-as-a-Service (SaaS) Migrating and building enterprise and

applications. hybrid applications.

Resources: You do not want to employ IT resources You have some IT resources for
for configuration and management of configuration and management. Some
the underlying infrastructure, but want provided automated features
to focus on the application layer. dramatically simplify this.

Total cost of ownership: Eliminates hardware costs and reduces Eliminates hardware costs.
administrative costs.

Business continuity: In addition to built-in fault tolerance SQL Server on Azure VMs lets you set
infrastructure capabilities, Azure SQL up a high availability and disaster
Database provides features, such as recovery solution for your database’s
automated backups, Point-In-Time specific needs. Therefore, you can have
Restore, Geo-Restore, and Active Geo- a system that is highly optimized for
Replication to increase business your application. You can test and run
continuity. For more information, see failovers by yourself when needed. For
SQL Database business continuity more information, see High Availability
overview. and Disaster Recovery for SQL Server
on Azure Virtual Machines.

Hybrid cloud: Your on-premises application can access With SQL Server on Azure VMs, you can
data in Azure SQL Database. have applications that run partly in the
cloud and partly on-premises. For
example, you can extend your on-
premises network and Active Directory
Domain to the cloud via Azure Virtual
Network. In addition, you can store on-
premises data files in Azure Storage
using SQL Server Data Files in Azure.
For more information, see Introduction
to SQL Server 2014 Hybrid Cloud.

Supports SQL Server transactional Fully supports SQL Server transactional

replication as a subscriber to replicate replication, AlwaysOn Availability
data. Groups, Integration Services, and Log
Shipping to replicate data. Also,
traditional SQL Server backups are fully
supported

Business motivations for choosing Azure SQL Database or SQL Server

on Azure VMs
Cost
Whether you’re a startup that is strapped for cash, or a team in an established company that operates under tight
budget constraints, limited funding is often the primary driver when deciding how to host your databases. In this
section, you learn about the billing and licensing basics in Azure with regards to these two relational database
options: SQL Database and SQL Server on Azure VMs. You also learn about calculating the total application cost.
Billing and licensing basics
SQL Database is sold to customers as a service, not with a license. SQL Server on Azure VMs is sold with an
included license that you pay per-minute. If you have an existing license, you can also use it.
Currently, SQL Database is available in several service tiers, all of which are billed hourly at a fixed rate based on
the service tier and performance level you choose. In addition, you are billed for outgoing Internet traffic at regular
data transfer rates. The Basic, Standard, Premium, and Premium RS service tiers are designed to deliver predictable
performance with multiple performance levels to match your application’s peak requirements. You can change
between service tiers and performance levels to match your application’s varied throughput needs. If your database
has high transactional volume and needs to support many concurrent users, we recommend the Premium service
tier. For the latest information on the current supported service tiers, see Azure SQL Database Service Tiers. You can
also create elastic pools to share performance resources among database instances.
With SQL Database, the database software is automatically configured, patched, and upgraded by Microsoft, which
reduces your administration costs. In addition, its built-in backup capabilities help you achieve significant cost
savings, especially when you have a large number of databases.
With SQL Server on Azure VMs, you can use any of the platform-provided SQL Server images (which includes a
license) or bring your SQL Server license. All the supported SQL Server versions (2008R2, 2012, 2014, 2016) and
editions (Developer, Express, Web, Standard, Enterprise) are available. In addition, Bring-Your-Own-License
versions (BYOL) of the images are available. When using the Azure provided images, the operational cost depends
on the VM size and the edition of SQL Server you choose. Regardless of VM size or SQL Server edition, you pay
per-minute licensing cost of SQL Server and Windows Server, along with the Azure Storage cost for the VM disks.
The per-minute billing option allows you to use SQL Server for as long as you need without buying addition SQL
Server licenses. If you bring your own SQL Server license to Azure, you are charged for Windows Server and
storage costs only. For more information on bring-your-own licensing, see License Mobility through Software
Assurance on Azure.
Calculating the total application cost
When you start using a cloud platform, the cost of running your application includes the development and
administration costs, plus the public cloud platform service costs.
Here is the detailed cost calculation for your application running in SQL Database and SQL Server on Azure VMs:
When using Azure SQL Database:
Total cost of application = Highly minimized administration costs + software development costs + SQL Database
service costs
When using SQL Server on Azure VMs:
Total cost of application = Highly minimized software development cost + administration costs + SQL Server and
Windows Server licensing costs + Azure Storage costs
For more information on pricing, see the following resources:
SQL Database Pricing
Virtual Machine Pricing for SQL and for Windows
Azure Pricing Calculator
NOTE
There is a small subset of features on SQL Server that are not applicable or not available with SQL Database. See SQL
Database Features and SQL Database Transact-SQL information for more information. If you are moving an existing SQL
Server solution to the cloud, see Migrating a SQL Server database to Azure SQL Database. When you migrate an existing on-
premises SQL Server application to SQL Database, consider updating the application to take advantage of the capabilities that
cloud services offer. For example, you may consider using Azure Web App Service or Azure Cloud Services to host your
application layer to increase cost benefits.

Administration
For many businesses, the decision to transition to a cloud service is as much about offloading complexity of
administration as it is cost. With SQL Database, Microsoft administers the underlying hardware. Microsoft
automatically replicates all data to provide high availability, configures and upgrades the database software,
manages load balancing, and does transparent failover if there is a server failure. You can continue to administer
your database, but you no longer need to manage the database engine, server operating system or hardware.
Examples of items you can continue to administer include databases and logins, index and query tuning, and
auditing and security.
With SQL Server on Azure VMs, you have full control over the operating system and SQL Server instance
configuration. With a VM, it’s up to you to decide when to update/upgrade the operating system and database
software and when to install any additional software such as anti-virus. Some automated features are provided to
dramatically simplify patching, backup, and high availability. In addition, you can control the size of the VM, the
number of disks, and their storage configurations. Azure allows you to change the size of a VM as needed. For
information, see Virtual Machine and Cloud Service Sizes for Azure.
Service Level Agreement (SLA )
For many IT departments, meeting up-time obligations of a Service Level Agreement (SLA) is a top priority. In this
section, we look at what SLA applies to each database hosting option.
For SQL Database Basic, Standard, Premium, and Premium RS service tiers Microsoft provides an availability SLA
of 99.99%. For the latest information, see Service Level Agreement. For the latest information on SQL Database
service tiers and the supported business continuity plans, see Service Tiers.
For SQL Server running on Azure VMs, Microsoft provides an availability SLA of 99.95% that covers just the
Virtual Machine. This SLA does not cover the processes (such as SQL Server) running on the VM and requires that
you host at least two VM instances in an availability set. For the latest information, see the VM SLA. For database
high availability (HA) within VMs, you should configure one of the supported high availability options in SQL
Server, such as AlwaysOn Availability Groups. Using a supported high availability option doesn't provide an
additional SLA, but allows you to achieve >99.99% database availability.
Time to market
SQL Database is the right solution for cloud-designed applications when developer productivity and fast time-to-
market are critical. With programmatic DBA-like functionality, it is perfect for cloud architects and developers as it
lowers the need for managing the underlying operating system and database. For example, you can use the REST
API and PowerShell Cmdlets to automate and manage administrative operations for thousands of databases.
Features such as elastic pools allow you to focus on the application layer and deliver your solution to the market
faster.
SQL Server running on Azure VMs is perfect if your existing or new applications require large databases,
interrelated databases, or access to all features in SQL Server or Windows. It is also a good fit when you want to
migrate existing on-premises applications and databases to Azure as-is. Since you do not need to change the
presentation, application, and data layers, you save time and budget on rearchitecting your existing solution.
Instead, you can focus on migrating all your solutions to Azure and in doing some performance optimizations that
may be required by the Azure platform. For more information, see Performance Best Practices for SQL Server on
Azure Virtual Machines.

Summary
This article explored SQL Database and SQL Server on Azure Virtual Machines (VMs) and discussed common
business motivators that might affect your decision. The following is a summary of suggestions for you to consider:
Choose Azure SQL Database if:
You are building new cloud-based applications to take advantage of the cost savings and performance
optimization that cloud services provide. This approach provides the benefits of a fully managed cloud service,
helps lower initial time-to-market, and can provide long-term cost optimization.
You want to have Microsoft perform common management operations on your databases and require stronger
availability SLAs for databases.
Choose SQL Server on Azure VMs if:
You have existing on-premises applications that you want to migrate or extend to the cloud, or if you want to
build enterprise applications larger than 1 TB. This approach provides the benefit of 100% SQL compatibility,
large database capacity, full control over SQL Server and Windows, and secure tunneling to on-premises. This
approach minimizes costs for development and modifications of existing applications.
You have existing IT resources and can ultimately own patching, backups, and database high availability. Notice
that some automated features dramatically simplify these operations.

Next steps
See Your first Azure SQL Database to get started with SQL Database.
See SQL Database pricing.
See Provision a SQL Server virtual machine in Azure to get started with SQL Server on Azure VMs.
Azure SQL Database features
4/14/2017 • 3 min to read • Edit Online

The following tables list the major features of Azure SQL Database and the equivalent features of SQL Server -
and providing information about whether each particular feature is supported and a link to more information
about the feature on each platform. For Transact-SQL differences to consider when migrating an existing
database solution, see Resolving Transact-SQL differences during migration to SQL Database.
We continue to add features to Azure SQL Database. So we encourage you to visit our Service Updates
webpage for Azure, and to use its filters:
Filtered to the SQL Database service.
Filtered to General Availability (GA) announcements for SQL Database features.

FEATURE SQL SERVER AZURE SQL DATABASE

Active Geo-Replication Not supported - see Always On Supported

Availability Groups

Always Encrypted Supported Supported - see Cert store and Key

vault

AlwaysOn Availability Groups Supported Not supported - See Active Geo-

Replication

Attach a database Supported Not supported

Application roles Supported Supported

Auto scale Not supported Supported - see Service tiers

Azure Active Directory Not supported Supported

Azure Data Factory Supported Supported

Auditing Supported Supported

BACPAC file (export) Supported Supported

BACPAC file (import) Supported Supported

BACKUP Supported Not supported

Built-in functions Supported Most - see individual functions

Change data capture Supported Not supported

Change tracking Supported Supported

Collation statements Supported Supported

FEATURE SQL SERVER AZURE SQL DATABASE

Columnstore indexes Supported Premium edition only

Common language runtime (CLR) Supported Not supported

Contained databases Supported Supported

Contained users Supported Supported

Control of flow language keywords Supported Supported

Cross-database queries Supported Partial - see Elastic queries

Cursors Supported Supported

Data compression Supported Supported

Database backups User managed Managed by SQL Database service

Database mail Supported Not supported

Database mirroring Supported Not supported

Database configuration settings Supported Supported

Data Quality Services (DQS) Supported Not supported

Database snapshots Supported Not supported

Data types Supported Supported

DBCC statements Supported Most - see individual statements

DDL statements Supported Most - see Individual statements

DDL triggers Supported Database only

Distributed transactions MS DTC Limited intra-SQL Database scenarios

only

DML statements Supported Most - see Individual statements]

DML triggers Supported Supported

DMVs All Some - see Individual DMVs

Elastic pools Not supported Supported

Elastic jobs Not supported - see SQL Server Agent Supported

FEATURE SQL SERVER AZURE SQL DATABASE

Elastic queries Not supported - see Cross-database Supported

queries

Event notifications Supported Supported

Expressions Supported Supported

Extended events Supported Some - see Individual events

Extended stored procedures Supported Not supported

Files and file groups Supported Primary only

Filestream Supported Not supported

Full-text search Supported Third-party word breakers not

supported

Functions Supported Most - see Individual functions

In-memory optimization Supported Premium edition only

Jobs See SQL Server Agent See Elastic jobs

JSON data support Supported Supported

Language elements Supported Most - See Individual elements

Linked servers Supported Not supported - see Elastic query

Log shipping Supported Not supported - see Active Geo-

Replication

Master Data Services (MDS) Supported Not supported

Minimal logging in bulk import Supported Not supported

Modifying system data Supported Not supported

Online index operations Supported Supported - transaction size limited by

service tier

Operators Supported Most - see Individual operators

Point in time database restore Supported Supported

Polybase Supported Not supported

Policy-based management Supported Not supported

Predicates Supported Most - See Individual predicates

FEATURE SQL SERVER AZURE SQL DATABASE

R Services Supported

Resource governor Supported Not supported

RESTORE statements Supported Not Supported

Restore database from backup Supported From built-in backups only

Row Level Security Supported Supported

Semantic search Supported Not supported

Sequence numbers Supported Supported

Service Broker Supported Not supported

Server configuration settings Supported Not supported - see Database

configuration options

Set statements Supported Most - See Individual statements

Spatial Supported Supported

SQL Server Agent Supported Not supported - See Elastic jobs

SQL Server Analysis Services (SSAS) Supported Not supported - see Azure Analysis
Services

SQL Server Integration Services (SSIS) Supported Not supported - see Azure Data
Factory

SQL Server PowerShell Supported Supported

SQL Server Profiler Supported Not supported - see Extended events

SQL Server Replication Supported Transactional and snapshot replication

subscriber only

SQL Server Reporting Services (SSRS) Supported Not supported

Stored procedures Supported Supported

System stored functions Supported Some - See Individual function

System stored procedures Supported Some - see Individual stored

procedure

System tables Supported Some - See Individual table

System catalog views Supported Some - See Individual views

FEATURE SQL SERVER AZURE SQL DATABASE

Table Partitioning Supported Supported - Primary filegroup only

Temporary tables Local and global Local only

Temporal tables Supported Supported

Transactions Supported Supported

Variables Supported Supported

Transparent data encryption (TDE) Supported Supported

Windows Server Failover Clustering Supported Not supported - See Active Geo-
Replication

XML indexes Supported Supported

Next steps
For information about the Azure SQL Database service, see What is SQL Database?
For information about Transact-SQL support and differences, see Resolving Transact-SQL differences during
migration to SQL Database.
Overview: Tools to manage & develop with Azure
SQL Database
4/14/2017 • 1 min to read • Edit Online

This topic introduces the tools for managing and developing with Azure SQL databases.

IMPORTANT
This documentation set includes QuickStarts, Sample, and How-to guides showing you how to management and develop
with Azure SQL Database using the tools introduced in the following paragraphs. Use the left-hand navigation pane and
filter box to find specific content for the Azure portal, PowerShell, and T-SQL.

Azure portal
The Azure portal is a web-based application where you can create, update, and delete databases and logical
servers and monitor database activity. This tool is great if you're just getting started with Azure, managing a few
databases, or need to do something quickly.

SQL Server Management Studio and Transact-SQL

SQL Server Management Studio (SSMS) is a client tool that runs on your computer for managing your database in
the cloud using Transact-SQL. Many database administrators are familiar with SSMS, which can be used with
Azure SQL databases. Download the latest version of SSMS and always use the latest release when working with
Azure SQL Database.

IMPORTANT
Always use the latest version of SQL Server Management Studio.

SQL Server Data Tools in Visual Studio

SQL Server Data Tools (SSDT) is a client tool that runs on your computer for developing your database in the
cloud. If you're an application developer familiar with Visual Studio or other integrated development
environments (IDEs), try using SSDT in Visual Studio.

IMPORTANT
Always use the latest version of SQL Server Data Tools to remain synchronized with updates to Microsoft Azure and SQL
Database.

PowerShell
You can use PowerShell to manage databases and elastic pools, and to automate Azure resource deployments. Microsoft
recommends this tool for managing a large number of databases and automating deployment and resource changes in a
production environment.

Elastic Database tools

Use the elastic database tools to perform actions such as
Executing a T-SQL script against a set of databases using an elastic job
Moving multi-tenant model databases to a single-tenant model with the split-merge tool
Managing databases in a single-tenant model or a multi-tenant model using the elastic scale client library.

Additional resources
Azure Resource Manager
Azure Automation
Azure Scheduler
Scaling out with Azure SQL Database
4/19/2017 • 6 min to read • Edit Online

You can easily scale out Azure SQL databases using the Elastic Database tools. These tools and features let you
use the virtually unlimited database resources of Azure SQL Database to create solutions for transactional
workloads, and especially Software as a Service (SaaS) applications. Elastic Database features are composed of
the following:
Elastic Database client library: The client library is a feature that allows you to create and maintain sharded
databases. See Get started with Elastic Database tools.
Elastic Database split-merge tool: moves data between sharded databases. This is useful for moving data from
a multi-tenant database to a single-tenant database (or vice-versa). See Elastic database Split-Merge tool
tutorial.
Elastic Database jobs (preview): Use jobs to manage large numbers of Azure SQL databases. Easily perform
administrative operations such as schema changes, credentials management, reference data updates,
performance data collection or tenant (customer) telemetry collection using jobs.
Elastic Database query (preview): Enables you to run a Transact-SQL query that spans multiple databases. This
enables connection to reporting tools such as Excel, PowerBI, Tableau, etc.
Elastic transactions: This feature allows you to run transactions that span several databases in Azure SQL
Database. Elastic database transactions are available for .NET applications using ADO .NET and integrate with
the familiar programming experience using the System.Transaction classes.
The graphic below shows an architecture that includes the Elastic Database features in relation to a collection
of databases.
In this graphic, colors of the database represent schemas. Databases with the same color share the same schema.
1. A set of Azure SQL databases are hosted on Azure using sharding architecture.
2. The Elastic Database client library is used to manage a shard set.
3. A subset of the databases are put into an elastic pool. (See What is a pool?).
4. An Elastic Database job runs scheduled or ad-hoc T-SQL scripts against all databases.
5. The split-merge tool is used to move data from one shard to another.
6. The Elastic Database query allows you to write a query that spans all databases in the shard set.
7. Elastic transactions allows you to run transactions that span several databases.
Why use the tools?
Achieving elasticity and scale for cloud applications has been straightforward for VMs and blob storage--simply
add or subtract units, or increase power. But it has remained a challenge for stateful data processing in relational
databases. Challenges emerged in these scenarios:
Growing and shrinking capacity for the relational database part of your workload.
Managing hotspots that may arise affecting a specific subset of data - such as a particularly busy end-
customer (tenant).
Traditionally, scenarios like these have been addressed by investing in larger-scale database servers to support
the application. However, this option is limited in the cloud where all processing happens on predefined
commodity hardware. Instead, distributing data and processing across many identically-structured databases (a
scale-out pattern known as "sharding") provides an alternative to traditional scale-up approaches both in terms
of cost and elasticity.

Horizontal and vertical scaling

The figure below shows the horizontal and vertical dimensions of scaling, which are the basic ways the elastic
databases can be scaled.
Horizontal scaling refers to adding or removing databases in order to adjust capacity or overall performance. This
is also called “scaling out”. Sharding, in which data is partitioned across a collection of identically structured
databases, is a common way to implement horizontal scaling.
Vertical scaling refers to increasing or decreasing the performance level of an individual database—this is also
known as “scaling up.”
Most cloud-scale database applications will use a combination of these two strategies. For example, a Software as
a Service application may use horizontal scaling to provision new end-customers and vertical scaling to allow
each end-customer’s database to grow or shrink resources as needed by the workload.
Horizontal scaling is managed using the Elastic Database client library.
Vertical scaling is accomplished using Azure PowerShell cmdlets to change the service tier, or by placing
databases in an elastic pool.

Sharding
Sharding is a technique to distribute large amounts of identically-structured data across a number of
independent databases. It is especially popular with cloud developers creating Software as a Service (SAAS)
offerings for end customers or businesses. These end customers are often referred to as “tenants”. Sharding may
be required for any number of reasons:
The total amount of data is too large to fit within the constraints of a single database
The transaction throughput of the overall workload exceeds the capabilities of a single database
Tenants may require physical isolation from each other, so separate databases are needed for each tenant
Different sections of a database may need to reside in different geographies for compliance, performance or
geopolitical reasons.
In other scenarios, such as ingestion of data from distributed devices, sharding can be used to fill a set of
databases that are organized temporally. For example, a separate database can be dedicated to each day or week.
In that case, the sharding key can be an integer representing the date (present in all rows of the sharded tables)
and queries retrieving information for a date range must be routed by the application to the subset of databases
covering the range in question.
Sharding works best when every transaction in an application can be restricted to a single value of a sharding
key. That ensures that all transactions will be local to a specific database.
Multi-tenant and single-tenant
Some applications use the simplest approach of creating a separate database for each tenant. This is the single
tenant sharding pattern that provides isolation, backup/restore ability and resource scaling at the granularity
of the tenant. With single tenant sharding, each database is associated with a specific tenant ID value (or
customer key value), but that key need not always be present in the data itself. It is the application’s responsibility
to route each request to the appropriate database - and the client library can simplify this.

Others scenarios pack multiple tenants together into databases, rather than isolating them into separate
databases. This is a typical multi-tenant sharding pattern - and it may be driven by the fact that an application
manages large numbers of very small tenants. In multi-tenant sharding, the rows in the database tables are all
designed to carry a key identifying the tenant ID or sharding key. Again, the application tier is responsible for
routing a tenant’s request to the appropriate database, and this can be supported by the elastic database client
library. In addition, row-level security can be used to filter which rows each tenant can access - for details, see
Multi-tenant applications with elastic database tools and row-level security. Redistributing data among databases
may be needed with the multi-tenant sharding pattern, and this is facilitated by the elastic database split-merge
tool. To learn more about design patterns for SaaS applications using elastic pools, see Design Patterns for Multi-
tenant SaaS Applications with Azure SQL Database.
Move data from multiple to single -tenancy databases
When creating a SaaS application, it is typical to offer prospective customers a trial version of the software. In this
case, it is cost-effective to use a multi-tenant database for the data. However, when a prospect becomes a
customer, a single-tenant database is better since it provides better performance. If the customer had created
data during the trial period, use the split-merge tool to move the data from the multi-tenant to the new single-
tenant database.

Next steps
For a sample app that demonstrates the client library, see Get started with Elastic Datababase tools.
To convert existing databases to use the tools, see Migrate existing databases to scale-out.
To see the specifics of the elastic pool, see Price and performance considerations for an elastic pool, or create a
new pool with elastic pools.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Building scalable cloud databases
1/17/2017 • 3 min to read • Edit Online

Scaling out databases can be easily accomplished using scalable tools and features for Azure SQL Database. In
particular, you can use the Elastic Database client library to create and manage scaled-out databases. This
feature lets you easily develop sharded applications using hundreds—or even thousands—of Azure SQL
databases. Elastic jobs can then be used to help ease management of these databases.
To install the library, go to Microsoft.Azure.SqlDatabase.ElasticScale.Client.

Documentation
1. Get started with Elastic Database tools
2. Elastic Database features
3. Shard map management
4. Migrate existing databases to scale-out
5. Data dependent routing
6. Multi-shard queries
7. Adding a shard using Elastic Database tools
8. Multi-tenant applications with elastic database tools and row-level security
9. Upgrade client library apps
10. Elastic queries overview
11. Elastic database tools glossary
12. Elastic Database client library with Entity Framework
13. Elastic database client library with Dapper
14. Split-merge tool
15. Performance counters for shard map manager
16. FAQ for Elastic database tools

Client capabilities
Scaling out applications using sharding presents challenges for both the developer as well as the administrator.
The client library simplifies the management tasks by providing tools that let both developers and
administrators manage scaled-out databases. In a typical example, there are many databases, known as "shards,"
to manage. Customers are co-located in the same database, and there is one database per customer (a single-
tenant scheme). The client library includes these features:
Shard Map Management: A special database called the "shard map manager" is created. Shard map
management is the ability for an application to manage metadata about its shards. Developers can use
this functionality to register databases as shards, describe mappings of individual sharding keys or key
ranges to those databases, and maintain this metadata as the number and composition of databases
evolves to reflect capacity changes. Without the elastic database client library, you would need to spend a
lot of time writing the management code when implementing sharding. For details, see Shard map
management.
Data dependent routing: Imagine a request coming into the application. Based on the sharding key
value of the request, the application needs to determine the correct database based on the key value. It
then opens a connection to the database to process the request. Data dependent routing provides the
ability to open connections with a single easy call into the shard map of the application. Data dependent
routing was another area of infrastructure code that is now covered by functionality in the elastic
database client library. For details, see Data dependent routing.
Multi-shard queries (MSQ): Multi-shard querying works when a request involves several (or all) shards. A
multi-shard query executes the same T-SQL code on all shards or a set of shards. The results from the
participating shards are merged into an overall result set using UNION ALL semantics. The functionality as
exposed through the client library handles many tasks, including: connection management, thread
management, fault handling and intermediate results processing. MSQ can query up to hundreds of shards.
For details, see Multi-shard querying.
In general, customers using elastic database tools can expect to get full T-SQL functionality when submitting
shard-local operations as opposed to cross-shard operations that have their own semantics.

Next steps
Try the sample app which demonstrates the client functions.
To install the library, go to Elastic Database Client Library.
For instructions on using the split-merge tool, see the split-merge tool overview.
Elastic database client library is now open sourced!
Use Elastic queries.
The library is available as open source software on GitHub.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Scale out databases with the shard map manager
1/23/2017 • 13 min to read • Edit Online

To easily scale out databases on SQL Azure, use a shard map manager. The shard map manager is a special
database that maintains global mapping information about all shards (databases) in a shard set. The metadata
allows an application to connect to the correct database based upon the value of the sharding key. In addition,
every shard in the set contains maps that track the local shard data (known as shardlets).

Understanding how these maps are constructed is essential to shard map management. This is done using the
ShardMapManager class, found in the Elastic Database client library to manage shard maps.

Shard maps and shard mappings

For each shard, you must select the type of shard map to create. The choice depends on the database
architecture:
1. Single tenant per database
2. Multiple tenants per database (two types):
a. List mapping
b. Range mapping
For a single-tenant model, create a list mapping shard map. The single-tenant model assigns one database per
tenant. This is an effective model for SaaS developers as it simplifies management.
The multi-tenant model assigns several tenants to a single database (and you can distribute groups of tenants
across multiple databases). Use this model when you expect each tenant to have small data needs. In this model,
we assign a range of tenants to a database using range mapping.

Supported .Net types for sharding keys

Elastic Scale support the following .Net Framework types as sharding keys:
integer
long
guid
byte[]
datetime
timespan
datetimeoffset
List and range shard maps
Shard maps can be constructed using lists of individual sharding key values, or they can be constructed
using ranges of sharding key values.
List shard maps
Shards contain shardlets and the mapping of shardlets to shards is maintained by a shard map. A list shard
map is an association between the individual key values that identify the shardlets and the databases that serve
as shards. List mappings are explicit and different key values can be mapped to the same database. For
example, key 1 maps to Database A, and key values 3 and 6 both reference Database B.

KEY SHARD LOCATION

1 Database_A

3 Database_B

4 Database_C

6 Database_B

... ...

Range shard maps

In a range shard map, the key range is described by a pair [Low Value, High Value) where the Low Value is
the minimum key in the range, and the High Value is the first value higher than the range.
For example, [0, 100) includes all integers greater than or equal 0 and less than 100. Note that multiple ranges
can point to the same database, and disjoint ranges are supported (e.g., [100,200) and [400,600) both point to
Database C in the example below.)

KEY SHARD LOCATION

[1,50) Database_A

[50,100) Database_B

[100,200) Database_C

[400,600) Database_C

... ...

Each of the tables shown above is a conceptual example of a ShardMap object. Each row is a simplified example
of an individual PointMapping (for the list shard map) or RangeMapping (for the range shard map) object.

Shard map manager

In the client library, the shard map manager is a collection of shard maps. The data managed by a
ShardMapManager instance is kept in three places:
1. Global Shard Map (GSM): You specify a database to serve as the repository for all of its shard maps and
mappings. Special tables and stored procedures are automatically created to manage the information. This is
typically a small database and lightly accessed, and it should not be used for other needs of the application.
The tables are in a special schema named __ShardManagement.
2. Local Shard Map (LSM): Every database that you specify to be a shard is modified to contain several small
tables and special stored procedures that contain and manage shard map information specific to that shard.
This information is redundant with the information in the GSM, and it allows the application to validate
cached shard map information without placing any load on the GSM; the application uses the LSM to
determine if a cached mapping is still valid. The tables corresponding to the LSM on each shard are also in
the schema __ShardManagement.
3. Application cache: Each application instance accessing a ShardMapManager object maintains a local in-
memory cache of its mappings. It stores routing information that has recently been retrieved.

Constructing a ShardMapManager
A ShardMapManager object is constructed using a factory pattern. The
ShardMapManagerFactory.GetSqlShardMapManager method takes credentials (including the server name
and database name holding the GSM) in the form of a ConnectionString and returns an instance of a
ShardMapManager.
Please Note: The ShardMapManager should be instantiated only once per app domain, within the
initialization code for an application. Creation of additional instances of ShardMapManager in the same
appdomain, will result in increased memory and CPU utilization of the application. A ShardMapManager can
contain any number of shard maps. While a single shard map may be sufficient for many applications, there are
times when different sets of databases are used for different schema or for unique purposes; in those cases
multiple shard maps may be preferable.
In this code, an application tries to open an existing ShardMapManager with the TryGetSqlShardMapManager
method. If objects representing a Global ShardMapManager (GSM) do not yet exist inside the database, the
client library creates them there using the CreateSqlShardMapManager method.
// Try to get a reference to the Shard Map Manager
// via the Shard Map Manager database.
// If it doesn't already exist, then create it.
ShardMapManager shardMapManager;
bool shardMapManagerExists = ShardMapManagerFactory.TryGetSqlShardMapManager(
connectionString,
ShardMapManagerLoadPolicy.Lazy,
out shardMapManager);

if (shardMapManagerExists)
{
Console.WriteLine("Shard Map Manager already exists");
}
else
{
// Create the Shard Map Manager.
ShardMapManagerFactory.CreateSqlShardMapManager(connectionString);
Console.WriteLine("Created SqlShardMapManager");

shardMapManager = ShardMapManagerFactory.GetSqlShardMapManager(
connectionString,
ShardMapManagerLoadPolicy.Lazy);

// The connectionString contains server name, database name, and admin credentials
// for privileges on both the GSM and the shards themselves.
}

As an alternative, you can use Powershell to create a new Shard Map Manager. An example is available here.

Get a RangeShardMap or ListShardMap

After creating a shard map manager, you can get the RangeShardMap or ListShardMap using the
TryGetRangeShardMap, the TryGetListShardMap, or the GetShardMap method.

/// <summary>
/// Creates a new Range Shard Map with the specified name, or gets the Range Shard Map if it already exists.
/// </summary>
public static RangeShardMap<T> CreateOrGetRangeShardMap<T>(ShardMapManager shardMapManager, string shardMapName)
{
// Try to get a reference to the Shard Map.
RangeShardMap<T> shardMap;
bool shardMapExists = shardMapManager.TryGetRangeShardMap(shardMapName, out shardMap);

if (shardMapExists)
{
ConsoleUtils.WriteInfo("Shard Map {0} already exists", shardMap.Name);
}
else
{
// The Shard Map does not exist, so create it
shardMap = shardMapManager.CreateRangeShardMap<T>(shardMapName);
ConsoleUtils.WriteInfo("Created Shard Map {0}", shardMap.Name);
}

return shardMap;
}

Shard map administration credentials

Applications that administer and manipulate shard maps are different from those that use the shard maps to
route connections.
To administer shard maps (add or change shards, shard maps, shard mappings, etc.) you must instantiate the
ShardMapManager using credentials that have read/write privileges on both the GSM database and on
each database that serves as a shard. The credentials must allow for writes against the tables in both the GSM
and LSM as shard map information is entered or changed, as well as for creating LSM tables on new shards.
See Credentials used to access the Elastic Database client library.
Only metadata affected
Methods used for populating or changing the ShardMapManager data do not alter the user data stored in the
shards themselves. For example, methods such as CreateShard, DeleteShard, UpdateMapping, etc. affect the
shard map metadata only. They do not remove, add, or alter user data contained in the shards. Instead, these
methods are designed to be used in conjunction with separate operations you perform to create or remove
actual databases, or that move rows from one shard to another to rebalance a sharded environment. (The split-
merge tool included with elastic database tools makes use of these APIs along with orchestrating actual data
movement between shards.) See Scaling using the Elastic Database split-merge tool.

Populating a shard map example

An example sequence of operations to populate a specific shard map is shown below. The code performs these
steps:
1. A new shard map is created within a shard map manager.
2. The metadata for two different shards is added to the shard map.
3. A variety of key range mappings are added, and the overall contents of the shard map are displayed.
The code is written so that the method can be rerun if an error occurs. Each request tests whether a shard or
mapping already exists, before attempting to create it. The code assumes that databases named
sample_shard_0, sample_shard_1 and sample_shard_2 have already been created in the server referenced by
string shardServer.

public void CreatePopulatedRangeMap(ShardMapManager smm, string mapName)

{
RangeShardMap<long> sm = null;

// check if shardmap exists and if not, create it

if (!smm.TryGetRangeShardMap(mapName, out sm))
{
sm = smm.CreateRangeShardMap<long>(mapName);
}

Shard shard0 = null, shard1=null;

// Check if shard exists and if not,
// create it (Idempotent / tolerant of re-execute)
if (!sm.TryGetShard(new ShardLocation(
shardServer,
"sample_shard_0"),
out shard0))
{
Shard0 = sm.CreateShard(new ShardLocation(
shardServer,
"sample_shard_0"));
}

if (!sm.TryGetShard(new ShardLocation(
shardServer,
"sample_shard_1"),
out shard1))
{
Shard1 = sm.CreateShard(new ShardLocation(
shardServer,
"sample_shard_1"));
}
RangeMapping<long> rmpg=null;

// Check if mapping exists and if not,

// create it (Idempotent / tolerant of re-execute)
if (!sm.TryGetMappingForKey(0, out rmpg))
{
sm.CreateRangeMapping(
new RangeMappingCreationInfo<long>
(new Range<long>(0, 50),
shard0,
MappingStatus.Online));
}

if (!sm.TryGetMappingForKey(50, out rmpg))

{
sm.CreateRangeMapping(
new RangeMappingCreationInfo<long>
(new Range<long>(50, 100),
shard1,
MappingStatus.Online));
}

if (!sm.TryGetMappingForKey(100, out rmpg))

{
sm.CreateRangeMapping(
new RangeMappingCreationInfo<long>
(new Range<long>(100, 150),
shard0,
MappingStatus.Online));
}

if (!sm.TryGetMappingForKey(150, out rmpg))

{
sm.CreateRangeMapping(
new RangeMappingCreationInfo<long>
(new Range<long>(150, 200),
shard1,
MappingStatus.Online));
}

if (!sm.TryGetMappingForKey(200, out rmpg))

{
sm.CreateRangeMapping(
new RangeMappingCreationInfo<long>
(new Range<long>(200, 300),
shard0,
MappingStatus.Online));
}

// List the shards and mappings

foreach (Shard s in sm.GetShards()
.OrderBy(s => s.Location.DataSource)
.ThenBy(s => s.Location.Database))
{
Console.WriteLine("shard: "+ s.Location);
}

foreach (RangeMapping<long> rm in sm.GetMappings())

{
Console.WriteLine("range: [" + rm.Value.Low.ToString() + ":"
+ rm.Value.High.ToString()+ ") ==>" +rm.Shard.Location);
}
}

As an alternative you can use PowerShell scripts to achieve the same result. Some of the sample PowerShell
examples are available here.
Once shard maps have been populated, data access applications can be created or adapted to work with the
maps. Populating or manipulating the maps need not occur again until map layout needs to change.

Data dependent routing

The shard map manager will be most used in applications that require database connections to perform the app-
specific data operations. Those connections must be associated with the correct database. This is known as Data
Dependent Routing. For these applications, instantiate a shard map manager object from the factory using
credentials that have read-only access on the GSM database. Individual requests for later connections supply
credentials necessary for connecting to the appropriate shard database.
Note that these applications (using ShardMapManager opened with read-only credentials) cannot make
changes to the maps or mappings. For those needs, create administrative-specific applications or PowerShell
scripts that supply higher-privileged credentials as discussed earlier. See Credentials used to access the Elastic
Database client library.
For more details, see Data dependent routing.

Modifying a shard map

A shard map can be changed in different ways. All of the following methods modify the metadata describing the
shards and their mappings, but they do not physically modify data within the shards, nor do they create or delete
the actual databases. Some of the operations on the shard map described below may need to be coordinated
with administrative actions that physically move data or that add and remove databases serving as shards.
These methods work together as the building blocks available for modifying the overall distribution of data in
your sharded database environment.
To add or remove shards: use CreateShard and DeleteShard of the Shardmap class.
The server and database representing the target shard must already exist for these operations to execute.
These methods do not have any impact on the databases themselves, only on metadata in the shard map.
To create or remove points or ranges that are mapped to the shards: use CreateRangeMapping,
DeleteMapping of the RangeShardMapping class, and CreatePointMapping of the ListShardMap
Many different points or ranges can be mapped to the same shard. These methods only affect metadata -
they do not affect any data that may already be present in shards. If data needs to be removed from the
database in order to be consistent with DeleteMapping operations, you will need to perform those
operations separately but in conjunction with using these methods.
To split existing ranges into two, or merge adjacent ranges into one: use SplitMapping and
MergeMappings.
Note that split and merge operations do not change the shard to which key values are mapped. A
split breaks an existing range into two parts, but leaves both as mapped to the same shard. A merge
operates on two adjacent ranges that are already mapped to the same shard, coalescing them into a
single range. The movement of points or ranges themselves between shards needs to be coordinated by
using UpdateMapping in conjunction with actual data movement. You can use the Split/Merge service
that is part of elastic database tools to coordinate shard map changes with data movement, when
movement is needed.
To re-map (or move) individual points or ranges to different shards: use UpdateMapping.
Since data may need to be moved from one shard to another in order to be consistent with
UpdateMapping operations, you will need to perform that movement separately but in conjunction with
using these methods.
To take mappings online and offline: use MarkMappingOffline and MarkMappingOnline to control
the online state of a mapping.
Certain operations on shard mappings are only allowed when a mapping is in an “offline” state, including
UpdateMapping and DeleteMapping. When a mapping is offline, a data-dependent request based on a
key included in that mapping will return an error. In addition, when a range is first taken offline, all
connections to the affected shard are automatically killed in order to prevent inconsistent or incomplete
results for queries directed against ranges being changed.
Mappings are immutable objects in .Net. All of the methods above that change mappings also invalidate any
references to them in your code. To make it easier to perform sequences of operations that change a mapping’s
state, all of the methods that change a mapping return a new mapping reference, so operations can be chained.
For example, to delete an existing mapping in shardmap sm that contains the key 25, you can execute the
following:

sm.DeleteMapping(sm.MarkMappingOffline(sm.GetMappingForKey(25)));

Adding a shard
Applications often need to simply add new shards to handle data that is expected from new keys or key ranges,
for a shard map that already exists. For example, an application sharded by Tenant ID may need to provision a
new shard for a new tenant, or data sharded monthly may need a new shard provisioned before the start of each
new month.
If the new range of key values is not already part of an existing mapping and no data movement is necessary, it
is very simple to add the new shard and associate the new key or range to that shard. For details on adding new
shards, see Adding a new shard.
For scenarios that require data movement, however, the split-merge tool is needed to orchestrate the data
movement between shards in combination with the necessary shard map updates. For details on using the split-
merge yool, see Overview of split-merge

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Data dependent routing
4/13/2017 • 5 min to read • Edit Online

Data dependent routing is the ability to use the data in a query to route the request to an appropriate
database. This is a fundamental pattern when working with sharded databases. The request context may also be
used to route the request, especially if the sharding key is not part of the query. Each specific query or transaction
in an application using data dependent routing is restricted to accessing a single database per request. For the
Azure SQL Database Elastic tools, this routing is accomplished with the ShardMapManager class in ADO.NET
applications.
The application does not need to track various connection strings or DB locations associated with different slices
of data in the sharded environment. Instead, the Shard Map Manager opens connections to the correct databases
when needed, based on the data in the shard map and the value of the sharding key that is the target of the
application’s request. The key is typically the customer_id, tenant_id, date_key, or some other specific identifier
that is a fundamental parameter of the database request).
For more information, see Scaling Out SQL Server with Data Dependent Routing.

Download the client library

To get the class, install the Elastic Database Client Library.

Using a ShardMapManager in a data dependent routing application

Applications should instantiate the ShardMapManager during initialization, using the factory call
GetSQLShardMapManager. In this example, both a ShardMapManager and a specific ShardMap that it
contains are initialized. This example shows the GetSqlShardMapManager and GetRangeShardMap methods.

ShardMapManager smm = ShardMapManagerFactory.GetSqlShardMapManager(smmConnnectionString,

ShardMapManagerLoadPolicy.Lazy);
RangeShardMap<int> customerShardMap = smm.GetRangeShardMap<int>("customerMap");

Use lowest privilege credentials possible for getting the shard map
If an application is not manipulating the shard map itself, the credentials used in the factory method should have
just read-only permissions on the Global Shard Map database. These credentials are typically different from
credentials used to open connections to the shard map manager. See also Credentials used to access the Elastic
Database client library.

Call the OpenConnectionForKey method

The ShardMap.OpenConnectionForKey method returns an ADO.Net connection ready for issuing commands
to the appropriate database based on the value of the key parameter. Shard information is cached in the
application by the ShardMapManager, so these requests do not typically involve a database lookup against the
Global Shard Map database.
// Syntax:
public SqlConnection OpenConnectionForKey<TKey>(
TKey key,
string connectionString,
ConnectionOptions options
)

The key parameter is used as a lookup key into the shard map to determine the appropriate database for the
request.
The connectionString is used to pass only the user credentials for the desired connection. No database
name or server name are included in this connectionString since the method will determine the database and
server using the ShardMap.
The connectionOptions should be set to ConnectionOptions.Validate if an environment where shard
maps may change and rows may move to other databases as a result of split or merge operations. This
involves a brief query to the local shard map on the target database (not to the global shard map) before the
connection is delivered to the application.
If the validation against the local shard map fails (indicating that the cache is incorrect), the Shard Map Manager
will query the global shard map to obtain the new correct value for the lookup, update the cache, and obtain and
return the appropriate database connection.
Use ConnectionOptions.None only when shard mapping changes are not expected while an application is
online. In that case, the cached values can be assumed to always be correct, and the extra round-trip validation
call to the target database can be safely skipped. That reduces database traffic. The connectionOptions may
also be set via a value in a configuration file to indicate whether sharding changes are expected or not during a
period of time.
This example uses the value of an integer key CustomerID, using a ShardMap object named
customerShardMap.

int customerId = 12345;

int newPersonId = 4321;

// Connect to the shard for that customer ID. No need to call a SqlConnection
// constructor followed by the Open method.
using (SqlConnection conn = customerShardMap.OpenConnectionForKey(customerId,
Configuration.GetCredentialsConnectionString(), ConnectionOptions.Validate))
{
// Execute a simple command.
SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = @"UPDATE Sales.Customer
SET PersonID = @newPersonID
WHERE CustomerID = @customerID";

cmd.Parameters.AddWithValue("@customerID", customerId);
cmd.Parameters.AddWithValue("@newPersonID", newPersonId);
cmd.ExecuteNonQuery();
}

The OpenConnectionForKey method returns a new already-open connection to the correct database.
Connections utilized in this way still take full advantage of ADO.Net connection pooling. As long as transactions
and requests can be satisfied by one shard at a time, this should be the only modification necessary in an
application already using ADO.Net.
The OpenConnectionForKeyAsync method is also available if your application makes use asynchronous
programming with ADO.Net. Its behavior is the data dependent routing equivalent of ADO.Net's
Connection.OpenAsync method.
Integrating with transient fault handling
A best practice in developing data access applications in the cloud is to ensure that transient faults are caught by
the app, and that the operations are retried several times before throwing an error. Transient fault handling for
cloud applications is discussed at Transient Fault Handling.
Transient fault handling can coexist naturally with the Data Dependent Routing pattern. The key requirement is to
retry the entire data access request including the using block that obtained the data-dependent routing
connection. The example above could be rewritten as follows (note highlighted change).
Example - data dependent routing with transient fault handling

int customerId = 12345;

int newPersonId = 4321;

Configuration.SqlRetryPolicy.ExecuteAction(() =>
{
// Connect to the shard for a customer ID.
using (SqlConnection conn = customerShardMap.OpenConnectionForKey(customerId,
Configuration.GetCredentialsConnectionString(), ConnectionOptions.Validate))
{
// Execute a simple command
SqlCommand cmd = conn.CreateCommand();

cmd.CommandText = @"UPDATE Sales.Customer

SET PersonID = @newPersonID
WHERE CustomerID = @customerID";

cmd.Parameters.AddWithValue("@customerID", customerId);
cmd.Parameters.AddWithValue("@newPersonID", newPersonId);
cmd.ExecuteNonQuery();

Console.WriteLine("Update completed");
}
});

Packages necessary to implement transient fault handling are downloaded automatically when you build the
elastic database sample application. Packages are also available separately at Enterprise Library - Transient Fault
Handling Application Block. Use version 6.0 or later.

Transactional consistency
Transactional properties are guaranteed for all operations local to a shard. For example, transactions submitted
through data-dependent routing execute within the scope of the target shard for the connection. At this time,
there are no capabilities provided for enlisting multiple connections into a transaction, and therefore there are no
transactional guarantees for operations performed across shards.

Next steps
To detach a shard, or to reattach a shard, see Using the RecoveryManager class to fix shard map problems

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Credentials used to access the Elastic Database client
library
1/17/2017 • 3 min to read • Edit Online

The Elastic Database client library uses three different kinds of credentials to access the shard map manager.
Depending on the need, use the credential with the lowest level of access possible.
Management credentials: for creating or manipulating a shard map manager. (See the glossary.)
Access credentials: to access an existing shard map manager to obtain information about shards.
Connection credentials: to connect to shards.
See also Managing databases and logins in Azure SQL Database.

About management credentials

Management credentials are used to create a ShardMapManager object for applications that manipulate shard
maps. (For example, see Adding a shard using Elastic Database tools and Data dependent routing) The user of the
elastic scale client library creates the SQL users and SQL logins and makes sure each is granted the read/write
permissions on the global shard map database and all shard databases as well. These credentials are used to
maintain the global shard map and the local shard maps when changes to the shard map are performed. For
instance, use the management credentials to create the shard map manager object (using
GetSqlShardMapManager:

// Obtain a shard map manager.

ShardMapManager shardMapManager = ShardMapManagerFactory.GetSqlShardMapManager(
smmAdminConnectionString,
ShardMapManagerLoadPolicy.Lazy
);

The variable smmAdminConnectionString is a connection string that contains the management credentials. The
user ID and password provides read/write access to both shard map database and individual shards. The
management connection string also includes the server name and database name to identify the global shard map
database. Here is a typical connection string for that purpose:

"Server=<yourserver>.database.windows.net;Database=<yourdatabase>;User ID=<yourmgmtusername>;Password=
<yourmgmtpassword>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;”

Do not use values in the form of "username@server"—instead just use the "username" value. This is because
credentials must work against both the shard map manager database and individual shards, which may be on
different servers.

Access credentials
When creating a shard map manager in an application that does not administer shard maps, use credentials that
have read-only permissions on the global shard map. The information retrieved from the global shard map under
these credentials are used for data-dependent routing and to populate the shard map cache on the client. The
credentials are provided through the same call pattern to GetSqlShardMapManager as shown above:
// Obtain shard map manager.
ShardMapManager shardMapManager = ShardMapManagerFactory.GetSqlShardMapManager(
smmReadOnlyConnectionString,
ShardMapManagerLoadPolicy.Lazy
);

Note the use of the smmReadOnlyConnectionString to reflect the use of different credentials for this access on
behalf of non-admin users: these credentials should not provide write permissions on the global shard map.

Connection credentials
Additional credentials are needed when using the OpenConnectionForKey method to access a shard associated
with a sharding key. These credentials need to provide permissions for read-only access to the local shard map
tables residing on the shard. This is needed to perform connection validation for data-dependent routing on the
shard. This code snippet allows data access in the context of data dependent routing:

using (SqlConnection conn = rangeMap.OpenConnectionForKey<int>(

targetWarehouse, smmUserConnectionString, ConnectionOptions.Validate))

In this example, smmUserConnectionString holds the connection string for the user credentials. For Azure SQL
DB, here is a typical connection string for user credentials:

"User ID=<yourusername>; Password=<youruserpassword>; Trusted_Connection=False; Encrypt=True; Connection Timeout=30;”

As with the admin credentials, do not values in the form of "username@server". Instead, just use "username". Also
note that the connection string does not contain a server name and database name. That is because the
OpenConnectionForKey call will automatically direct the connection to the correct shard based on the key.
Hence, the database name and server name are not provided.

See also
Managing databases and logins in Azure SQL Database
Securing your SQL Database
Getting started with Elastic Database jobs

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Multi-shard querying
1/23/2017 • 2 min to read • Edit Online

Overview
With the Elastic Database tools, you can create sharded database solutions. Multi-shard querying is used for
tasks such as data collection/reporting that require running a query that stretches across several shards. (Contrast
this to data-dependent routing, which performs all work on a single shard.)
1. Get a RangeShardMap or ListShardMap using the TryGetRangeShardMap, the TryGetListShardMap, or
the GetShardMap method. See Constructing a ShardMapManager and Get a RangeShardMap or
ListShardMap.
2. Create a MultiShardConnection object.
3. Create a MultiShardCommand.
4. Set the CommandText property to a T-SQL command.
5. Execute the command by calling the ExecuteReader method.
6. View the results using the MultiShardDataReader class.

Example
The following code illustrates the usage of multi-shard querying using a given ShardMap named myShardMap.

using (MultiShardConnection conn = new MultiShardConnection(

myShardMap.GetShards(),
myShardConnectionString)
)
{
using (MultiShardCommand cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT c1, c2, c3 FROM ShardedTable";
cmd.CommandType = CommandType.Text;
cmd.ExecutionOptions = MultiShardExecutionOptions.IncludeShardNameColumn;
cmd.ExecutionPolicy = MultiShardExecutionPolicy.PartialResults;

using (MultiShardDataReader sdr = cmd.ExecuteReader())

{
while (sdr.Read())
{
var c1Field = sdr.GetString(0);
var c2Field = sdr.GetFieldValue<int>(1);
var c3Field = sdr.GetFieldValue<Int64>(2);
}
}
}
}

A key difference is the construction of multi-shard connections. Where SqlConnection operates on a single
database, the MultiShardConnection takes a collection of shards as its input. Populate the collection of shards
from a shard map. The query is then executed on the collection of shards using UNION ALL semantics to
assemble a single overall result. Optionally, the name of the shard where the row originates from can be added to
the output using the ExecutionOptions property on command.
Note the call to myShardMap.GetShards(). This method retrieves all shards from the shard map and provides an
easy way to run a query across all relevant databases. The collection of shards for a multi-shard query can be
refined further by performing a LINQ query over the collection returned from the call to
myShardMap.GetShards(). In combination with the partial results policy, the current capability in multi-shard
querying has been designed to work well for tens up to hundreds of shards.
A limitation with multi-shard querying is currently the lack of validation for shards and shardlets that are queried.
While data-dependent routing verifies that a given shard is part of the shard map at the time of querying, multi-
shard queries do not perform this check. This can lead to multi-shard queries running on databases that have been
removed from the shard map.

Multi-shard queries and split-merge operations

Multi-shard queries do not verify whether shardlets on the queried database are participating in ongoing split-
merge operations. (See Scaling using the Elastic Database split-merge tool.) This can lead to inconsistencies where
rows from the same shardlet show for multiple databases in the same multi-shard query. Be aware of these
limitations and consider draining ongoing split-merge operations and changes to the shard map while performing
multi-shard queries.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.

See also
System.Data.SqlClient classes and methods.
Manage shards using the Elastic Database client library. Includes a namespace called
Microsoft.Azure.SqlDatabase.ElasticScale.Query that provides the ability to query multiple shards using a single
query and result. It provides a querying abstraction over a collection of shards. It also provides alternative
execution policies, in particular partial results, to deal with failures when querying over many shards.
Elastic Database tools glossary
1/17/2017 • 2 min to read • Edit Online

The following terms are defined for the Elastic Database tools, a feature of Azure SQL Database. The tools are used
to manage shard maps, and include the client library, the split-merge tool, elastic pools, and queries.
These terms are used in Adding a shard using Elastic Database tools and Using the RecoveryManager class to fix
shard map problems.

Database: An Azure SQL database.

Data dependent routing: The functionality that enables an application to connect to a shard given a specific
sharding key. See Data dependent routing. Compare to Multi-Shard Query.
Global shard map: The map between sharding keys and their respective shards within a shard set. The global
shard map is stored in the shard map manager. Compare to local shard map.
List shard map: A shard map in which sharding keys are mapped individually. Compare to Range Shard Map.
Local shard map: Stored on a shard, the local shard map contains mappings for the shardlets that reside on the
shard.
Multi-shard query: The ability to issue a query against multiple shards; results sets are returned using UNION
ALL semantics (also known as “fan-out query”). Compare to data dependent routing.
Multi-tenant and Single-tenant: This shows a single-tenant database and a multi-tenant database:
Here is a representation of sharded single and multi-tenant databases.

Range shard map: A shard map in which the shard distribution strategy is based on multiple ranges of
contiguous values.
Reference tables: Tables that are not sharded but are replicated across shards. For example, zip codes can be
stored in a reference table.
Shard: An Azure SQL database that stores data from a sharded data set.
Shard elasticity: The ability to perform both horizontal scaling and vertical scaling.
Sharded tables: Tables that are sharded, i.e., whose data is distributed across shards based on their sharding key
values.
Sharding key: A column value that determines how data is distributed across shards. The value type can be one
of the following: int, bigint, varbinary, or uniqueidentifier.
Shard set: The collection of shards that are attributed to the same shard map in the shard map manager.
Shardlet: All of the data associated with a single value of a sharding key on a shard. A shardlet is the smallest unit
of data movement possible when redistributing sharded tables.
Shard map: The set of mappings between sharding keys and their respective shards.
Shard map manager: A management object and data store that contains the shard map(s), shard locations, and
mappings for one or more shard sets.
Verbs
Horizontal scaling: The act of scaling out (or in) a collection of shards by adding or removing shards to a shard
map, as shown below.

Merge: The act of moving shardlets from two shards to one shard and updating the shard map accordingly.
Shardlet move: The act of moving a single shardlet to a different shard.
Shard: The act of horizontally partitioning identically structured data across multiple databases based on a
sharding key.
Split: The act of moving several shardlets from one shard to another (typically new) shard. A sharding key is
provided by the user as the split point.
Vertical Scaling: The act of scaling up (or down) the performance level of an individual shard. For example,
changing a shard from Standard to Premium (which results in more computing resources).

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Moving data between scaled-out cloud databases
1/17/2017 • 18 min to read • Edit Online

If you are a Software as a Service developer, and suddenly your app undergoes tremendous demand, you need
to accommodate the growth. So you add more databases (shards). How do you redistribute the data to the new
databases without disrupting the data integrity? Use the split-merge tool to move data from constrained
databases to the new databases.
The split-merge tool runs as an Azure web service. An administrator or developer uses the tool to move
shardlets (data from a shard) between different databases (shards). The tool uses shard map management to
maintain the service metadata database, and ensure consistent mappings.

Download
Microsoft.Azure.SqlDatabase.ElasticScale.Service.SplitMerge

Documentation
1. Elastic database Split-Merge tool tutorial
2. Split-Merge security configuration
3. Split-merge security considerations
4. Shard map management
5. Migrate existing databases to scale-out
6. Elastic database tools
7. Elastic Database tools glossary

Why use the split-merge tool?

Flexibility
Applications need to stretch flexibly beyond the limits of a single Azure SQL DB database. Use the tool to move
data as needed to new databases while retaining integrity.
Split to grow
You need to increase overall capacity to handle explosive growth. To do so, create additional capacity by
sharding the data and by distributing it across incrementally more databases until capacity needs are fulfilled.
This is a prime example of the ‘split’ feature.
Merge to shrink
Capacity needs shrink due to the seasonal nature of a business. The tool lets you scale down to fewer scale units
when business slows. The ‘merge’ feature in the Elastic Scale split-merge Service covers this requirement.
Manage hotspots by moving shardlets
With multiple tenants per database, the allocation of shardlets to shards can lead to capacity bottlenecks on
some shards. This requires re-allocating shardlets or moving busy shardlets to new or less utilized shards.

Concepts & key features

Customer-hosted services
The split-merge is delivered as a customer-hosted service. You must deploy and host the service in your
Microsoft Azure subscription. The package you download from NuGet contains a configuration template to
complete with the information for your specific deployment. See the split-merge tutorial for details. Since the
service runs in your Azure subscription, you can control and configure most security aspects of the service. The
default template includes the options to configure SSL, certificate-based client authentication, encryption for
stored credentials, DoS guarding and IP restrictions. You can find more information on the security aspects in
the following document split-merge security configuration.
The default deployed service runs with one worker and one web role. Each uses the A1 VM size in Azure Cloud
Services. While you cannot modify these settings when deploying the package, you could change them after a
successful deployment in the running cloud service, (through the Azure portal). Note that the worker role must
not be configured for more than a single instance for technical reasons.
Shard map integration
The split-merge service interacts with the shard map of the application. When using the split-merge service to
split or merge ranges or to move shardlets between shards, the service automatically keeps the shard map up to
date. To do so, the service connects to the shard map manager database of the application and maintains ranges
and mappings as split/merge/move requests progress. This ensures that the shard map always presents an up-
to-date view when split-merge operations are going on. Split, merge and shardlet movement operations are
implemented by moving a batch of shardlets from the source shard to the target shard. During the shardlet
movement operation the shardlets subject to the current batch are marked as offline in the shard map and are
unavailable for data-dependent routing connections using the OpenConnectionForKey API.
Consistent shardlet connections
When data movement starts for a new batch of shardlets, any shard-map provided data-dependent routing
connections to the shard storing the shardlet are killed and subsequent connections from the shard map APIs to
the these shardlets are blocked while the data movement is in progress in order to avoid inconsistencies.
Connections to other shardlets on the same shard will also get killed, but will succeed again immediately on
retry. Once the batch is moved, the shardlets are marked online again for the target shard and the source data is
removed from the source shard. The service goes through these steps for every batch until all shardlets have
been moved. This will lead to several connection kill operations during the course of the complete
split/merge/move operation.
Managing shardlet availability
Limiting the connection killing to the current batch of shardlets as discussed above restricts the scope of
unavailability to one batch of shardlets at a time. This is preferred over an approach where the complete shard
would remain offline for all its shardlets during the course of a split or merge operation. The size of a batch,
defined as the number of distinct shardlets to move at a time, is a configuration parameter. It can be defined for
each split and merge operation depending on the application’s availability and performance needs. Note that
the range that is being locked in the shard map may be larger than the batch size specified. This is because the
service picks the range size such that the actual number of sharding key values in the data approximately
matches the batch size. This is important to remember in particular for sparsely populated sharding keys.
Metadata storage
The split-merge service uses a database to maintain its status and to keep logs during request processing. The
user creates this database in their subscription and provides the connection string for it in the configuration file
for the service deployment. Administrators from the user’s organization can also connect to this database to
review request progress and to investigate detailed information regarding potential failures.
Sharding-awareness
The split-merge service differentiates between (1) sharded tables, (2) reference tables, and (3) normal tables. The
semantics of a split/merge/move operation depend on the type of the table used and are defined as follows:
Sharded tables: Split, merge, and move operations move shardlets from source to target shard. After
successful completion of the overall request, those shardlets are no longer present on the source. Note that
the target tables need to exist on the target shard and must not contain data in the target range prior to
processing of the operation.
Reference tables: For reference tables, the split, merge and move operations copy the data from the source
to the target shard. Note, however, that no changes occur on the target shard for a given table if any row is
already present in this table on the target. The table has to be empty for any reference table copy operation
to get processed.
Other Tables: Other tables can be present on either the source or the target of a split and merge operation.
The split-merge service disregards these tables for any data movement or copy operations. Note, however,
that they can interfere with these operations in case of constraints.
The information on reference vs. sharded tables is provided by the SchemaInfo APIs on the shard map. The
following example illustrates the use of these APIs on a given shard map manager object smm:

// Create the schema annotations

SchemaInfo schemaInfo = new SchemaInfo();

// Reference tables
schemaInfo.Add(new ReferenceTableInfo("dbo", "region"));
schemaInfo.Add(new ReferenceTableInfo("dbo", "nation"));

// Sharded tables
schemaInfo.Add(new ShardedTableInfo("dbo", "customer", "C_CUSTKEY"));
schemaInfo.Add(new ShardedTableInfo("dbo", "orders", "O_CUSTKEY"));

// Publish
smm.GetSchemaInfoCollection().Add(Configuration.ShardMapName, schemaInfo);

The tables ‘region’ and ‘nation’ are defined as reference tables and will be copied with split/merge/move
operations. ‘customer’ and ‘orders’ in turn are defined as sharded tables. C_CUSTKEY and O_CUSTKEY serve as
the sharding key.
Referential Integrity
The split-merge service analyzes dependencies between tables and uses foreign key-primary key relationships
to stage the operations for moving reference tables and shardlets. In general, reference tables are copied first in
dependency order, then shardlets are copied in order of their dependencies within each batch. This is necessary
so that FK-PK constraints on the target shard are honored as the new data arrives.
Shard Map Consistency and Eventual Completion
In the presence of failures, the split-merge service resumes operations after any outage and aims to complete
any in progress requests. However, there may be unrecoverable situations, e.g., when the target shard is lost or
compromised beyond repair. Under those circumstances, some shardlets that were supposed to be moved may
continue to reside on the source shard. The service ensures that shardlet mappings are only updated after the
necessary data has been successfully copied to the target. Shardlets are only deleted on the source once all their
data has been copied to the target and the corresponding mappings have been updated successfully. The
deletion operation happens in the background while the range is already online on the target shard. The split-
merge service always ensures correctness of the mappings stored in the shard map.

The split-merge user interface

The split-merge service package includes a worker role and a web role. The web role is used to submit split-
merge requests in an interactive way. The main components of the user interface are as follows:
Operation Type: The operation type is a radio button that controls the kind of operation performed by the
service for this request. You can choose between the split, merge and move scenarios. You can also cancel a
previously submitted operation. You can use split, merge and move requests for range shard maps. List
shard maps only support move operations.
Shard Map: The next section of request parameters cover information about the shard map and the database
hosting your shard map. In particular, you need to provide the name of the Azure SQL Database server and
database hosting the shardmap, credentials to connect to the shard map database, and finally the name of
the shard map. Currently, the operation only accepts a single set of credentials. These credentials need to
have sufficient permissions to perform changes to the shard map as well as to the user data on the shards.
Source Range (split and merge): A split and merge operation processes a range using its low and high key.
To specify an operation with an unbounded high key value, check the “High key is max” check box and leave
the high key field empty. The range key values that you specify do not need to precisely match a mapping
and its boundaries in your shard map. If you do not specify any range boundaries at all the service will infer
the closest range for you automatically. You can use the GetMappings.ps1 PowerShell script to retrieve the
current mappings in a given shard map.
Split Source Behavior (split): For split operations, define the point to split the source range. You do this by
providing the sharding key where you want the split to occur. Use the radio button specify whether you want
the lower part of the range (excluding the split key) to move, or whether you want the upper part to move
(including the split key).
Source Shardlet (move): Move operations are different from split or merge operations as they do not require
a range to describe the source. A source for move is simply identified by the sharding key value that you plan
to move.
Target Shard (split): Once you have provided the information on the source of your split operation, you need
to define where you want the data to be copied to by providing the Azure SQL Db server and database name
for the target.
Target Range (merge): Merge operations move shardlets to an existing shard. You identify the existing shard
by providing the range boundaries of the existing range that you want to merge with.
Batch Size: The batch size controls the number of shardlets that will go offline at a time during the data
movement. This is an integer value where you can use smaller values when you are sensitive to long periods
of downtime for shardlets. Larger values will increase the time that a given shardlet is offline but may
improve performance.
Operation Id (Cancel): If you have an ongoing operation that is no longer needed, you can cancel the
operation by providing its operation ID in this field. You can retrieve the operation ID from the request status
table (see Section 8.1) or from the output in the web browser where you submitted the request.

Requirements and Limitations

The current implementation of the split-merge service is subject to the following requirements and limitations:
The shards need to exist and be registered in the shard map before a split-merge operation on these shards
can be performed.
The service does not create tables or any other database objects automatically as part of its operations. This
means that the schema for all sharded tables and reference tables need to exist on the target shard prior to
any split/merge/move operation. Sharded tables in particular are required to be empty in the range where
new shardlets are to be added by a split/merge/move operation. Otherwise, the operation will fail the initial
consistency check on the target shard. Also note that reference data is only copied if the reference table is
empty and that there are no consistency guarantees with regard to other concurrent write operations on the
reference tables. We recommend this: when running split/merge operations, no other write operations make
changes to the reference tables.
The service relies on row identity established by a unique index or key that includes the sharding key to
improve performance and reliability for large shardlets. This allows the service to move data at an even finer
granularity than just the sharding key value. This helps to reduce the maximum amount of log space and
locks that are required during the operation. Consider creating a unique index or a primary key including the
sharding key on a given table if you want to use that table with split/merge/move requests. For performance
reasons, the sharding key should be the leading column in the key or the index.
During the course of request processing, some shardlet data may be present both on the source and the
target shard. This is necessary to protect against failures during the shardlet movement. The integration of
split-merge with the shard map ensures that connections through the data dependent routing APIs using the
OpenConnectionForKey method on the shard map do not see any inconsistent intermediate states.
However, when connecting to the source or the target shards without using the OpenConnectionForKey
method, inconsistent intermediate states might be visible when split/merge/move requests are going on.
These connections may show partial or duplicate results depending on the timing or the shard underlying
the connection. This limitation currently includes the connections made by Elastic Scale Multi-Shard-Queries.
The metadata database for the split-merge service must not be shared between different roles. For example,
a role of the split-merge service running in staging needs to point to a different metadata database than the
production role.

Billing
The split-merge service runs as a cloud service in your Microsoft Azure subscription. Therefore charges for
cloud services apply to your instance of the service. Unless you frequently perform split/merge/move
operations, we recommend you delete your split-merge cloud service. That saves costs for running or deployed
cloud service instances. You can re-deploy and start your readily runnable configuration whenever you need to
perform split or merge operations.

Monitoring
Status tables
The split-merge Service provides the RequestStatus table in the metadata store database for monitoring of
completed and ongoing requests. The table lists a row for each split-merge request that has been submitted to
this instance of the split-merge service. It gives the following information for each request:
Timestamp: The time and date when the request was started.
OperationId: A GUID that uniquely identifies the request. This request can also be used to cancel the
operation while it is still ongoing.
Status: The current state of the request. For ongoing requests, it also lists the current phase in which the
request is.
CancelRequest: A flag that indicates whether the request has been cancelled.
Progress: A percentage estimate of completion for the operation. A value of 50 indicates that the operation
is approximately 50% complete.
Details: An XML value that provides a more detailed progress report. The progress report is periodically
updated as sets of rows are copied from source to target. In case of failures or exceptions, this column also
includes more detailed information about the failure.
Azure Diagnostics
The split-merge service uses Azure Diagnostics based on Azure SDK 2.5 for monitoring and diagnostics. You
control the diagnostics configuration as explained here: Enabling Diagnostics in Azure Cloud Services and
Virtual Machines. The download package includes two diagnostics configurations - one for the web role and one
for the worker role. These diagnostics configurations for the service follow the guidance from Cloud Service
Fundamentals in Microsoft Azure. It includes the definitions to log Performance Counters, IIS logs, Windows
Event Logs, and split-merge application event logs.

Deploy Diagnostics
To enable monitoring and diagnostics using the diagnostic configuration for the web and worker roles provided
by the NuGet package, run the following commands using Azure PowerShell:

$storage_name = "<YourAzureStorageAccount>"

$key = "<YourAzureStorageAccountKey"

$storageContext = New-AzureStorageContext -StorageAccountName $storage_name -StorageAccountKey $key

$config_path = "<YourFilePath>\SplitMergeWebContent.diagnostics.xml"

$service_name = "<YourCloudServiceName>"

Set-AzureServiceDiagnosticsExtension -StorageContext $storageContext -DiagnosticsConfigurationPath $config_path -ServiceName

$service_name -Slot Production -Role "SplitMergeWeb"

$config_path = "<YourFilePath>\SplitMergeWorkerContent.diagnostics.xml"

$service_name = "<YourCloudServiceName>"

Set-AzureServiceDiagnosticsExtension -StorageContext $storageContext -DiagnosticsConfigurationPath $config_path -ServiceName

$service_name -Slot Production -Role "SplitMergeWorker"

You can find more information on how to configure and deploy diagnostics settings here: Enabling Diagnostics
in Azure Cloud Services and Virtual Machines.

Retrieve diagnostics
You can easily access your diagnostics from the Visual Studio Server Explorer in the Azure part of the Server
Explorer tree. Open a Visual Studio instance, and in the menu bar click View, and Server Explorer. Click the Azure
icon to connect to your Azure subscription. Then navigate to Azure -> Storage -> -> Tables -> WADLogsTable.
For more information, see Browsing Storage Resources with Server Explorer.
The WADLogsTable highlighted in the figure above contains the detailed events from the split-merge service’s
application log. Note that the default configuration of the downloaded package is geared towards a production
deployment. Therefore the interval at which logs and counters are pulled from the service instances is large (5
minutes). For test and development, lower the interval by adjusting the diagnostics settings of the web or the
worker role to your needs. Right-click on the role in the Visual Studio Server Explorer (see above) and then
adjust the Transfer Period in the dialog for the Diagnostics configuration settings:

Performance
In general, better performance is to be expected from the higher, more performant service tiers in Azure SQL
Database. Higher IO, CPU and memory allocations for the higher service tiers benefit the bulk copy and delete
operations that the split-merge service uses. For that reason, increase the service tier just for those databases
for a defined, limited period of time.
The service also performs validation queries as part of its normal operations. These validation queries check for
unexpected presence of data in the target range and ensure that any split/merge/move operation starts from a
consistent state. These queries all work over sharding key ranges defined by the scope of the operation and the
batch size provided as part of the request definition. These queries perform best when an index is present that
has the sharding key as the leading column.
In addition, a uniqueness property with the sharding key as the leading column will allow the service to use an
optimized approach that limits resource consumption in terms of log space and memory. This uniqueness
property is required to move large data sizes (typically above 1GB).

How to upgrade
1. Follow the steps in Deploy a split-merge service.
2. Change your cloud service configuration file for your split-merge deployment to reflect the new
configuration parameters. A new required parameter is the information about the certificate used for
encryption. An easy way to do this is to compare the new configuration template file from the download
against your existing configuration. Make sure you add the settings for
“DataEncryptionPrimaryCertificateThumbprint” and “DataEncryptionPrimary” for both the web and the
worker role.
3. Before deploying the update to Azure, ensure that all currently running split-merge operations have finished.
You can easily do this by querying the RequestStatus and PendingWorkflows tables in the split-merge
metadata database for ongoing requests.
4. Update your existing cloud service deployment for split-merge in your Azure subscription with the new
package and your updated service configuration file.
You do not need to provision a new metadata database for split-merge to upgrade. The new version will
automatically upgrade your existing metadata database to the new version.

Best practices & troubleshooting

Define a test tenant and exercise your most important split/merge/move operations with the test tenant
across several shards. Ensure that all metadata is defined correctly in your shard map and that the operations
do not violate constraints or foreign keys.
Keep the test tenant data size above the maximum data size of your largest tenant to ensure you are not
encountering data size related issues. This helps you assess an upper bound on the time it takes to move a
single tenant around.
Make sure that your schema allows deletions. The split-merge service requires the ability to remove data
from the source shard once the data has been successfully copied to the target. For example, delete triggers
can prevent the service from deleting the data on the source and may cause operations to fail.
The sharding key should be the leading column in your primary key or unique index definition. That ensures
the best performance for the split or merge validation queries, and for the actual data movement and
deletion operations which always operate on sharding key ranges.
Collocate your split-merge service in the region and data center where your databases reside.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Elastic database tools FAQ
1/17/2017 • 2 min to read • Edit Online

If I have a single-tenant per shard and no sharding key, how do I populate the sharding key for the schema info?
The schema info object is only used to split merge scenarios. If an application is inherently single-tenant, then it
does not require the Split Merge tool and thus there is no need to populate the schema info object.
I’ve provisioned a database and I already have a Shard Map Manager, how do I register this new database as a shard?
Please see Adding a shard to an application using the elastic database client library.
How much do elastic database tools cost?
Using the elastic database client library does not incur any costs. Costs accrue only for the Azure SQL databases
that you use for shards and the Shard Map Manager, as well as the web/worker roles you provision for the Split
Merge tool.
Why are my credentials not working when I add a shard from a different server?
Do not use credentials in the form of “User ID=username@servername”, instead simply use “User ID = username”.
Also, be sure that the “username” login has permissions on the shard.
Do I need to create a Shard Map Manager and populate shards every time I start my applications?
No—the creation of the Shard Map Manager (for example,
ShardMapManagerFactory.CreateSqlShardMapManager) is a one-time operation. Your application should use
the call ShardMapManagerFactory.TryGetSqlShardMapManager() at application start-up time. There should
only one such call per application domain.
I have questions about using elastic database tools, how do I get them answered?
Please reach out to us on the Azure SQL Database forum.
When I get a database connection using a sharding key, I can still query data for other sharding keys on the same shard. Is this by
design?
The Elastic Scale APIs give you a connection to the correct database for your sharding key, but do not provide
sharding key filtering. Add WHERE clauses to your query to restrict the scope to the provided sharding key, if
necessary.
Can I use a different Azure Database edition for each shard in my shard set?
Yes, a shard is an individual database, and thus one shard could be a Premium edition while another be a Standard
edition. Further, the edition of a shard can scale up or down multiple times during the lifetime of the shard.
Does the Split Merge tool provision (or delete) a database during a split or merge operation?
No. For split operations, the target database must exist with the appropriate schema and be registered with the
Shard Map Manager. For merge operations, you must delete the shard from the shard map manager and then
delete the database.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Azure SQL Database elastic query overview
(preview)
2/21/2017 • 9 min to read • Edit Online

The elastic query feature (in preview) enables you to run a Transact-SQL query that spans multiple databases in
Azure SQL Database. It allows you to perform cross-database queries to access remote tables, and to connect
Microsoft and third party tools (Excel, PowerBI, Tableau, etc.) to query across data tiers with multiple databases.
Using this feature, you can scale out queries to large data tiers in SQL Database and visualize the results in
business intelligence (BI) reports.

Why use elastic queries?

Azure SQL Database
Query across Azure SQL databases completely in T-SQL. This allows for read-only querying of remote databases.
This provides an option for current on-premises SQL Server customers to migrate applications using three- and
four-part names or linked server to SQL DB.
Available on standard tier
Elastic query is supported on the Standard performance tier in addition to the Premium performance tier. See
the section on Preview Limitations below on performance limitations for lower performance tiers.
Push to remote databases
Elastic queries can now push SQL parameters to the remote databases for execution.
Stored procedure execution
Execute remote stored procedure calls or remote functions using sp_execute _remote.
Flexibility
External tables with elastic query can now refer to remote tables with a different schema or table name.

Elastic query scenarios

The goal is to facilitate querying scenarios where multiple databases contribute rows into a single overall result.
The query can either be composed by the user or application directly, or indirectly through tools that are
connected to the database. This is especially useful when creating reports, using commercial BI or data
integration tools—or any application that cannot be changed. With an elastic query, you can query across several
databases using the familiar SQL Server connectivity experience in tools such as Excel, PowerBI, Tableau, or
Cognos. An elastic query allows easy access to an entire collection of databases through queries issued by SQL
Server Management Studio or Visual Studio, and facilitates cross-database querying from Entity Framework or
other ORM environments. Figure 1 shows a scenario where an existing cloud application (which uses the elastic
database client library) builds on a scaled-out data tier, and an elastic query is used for cross-database reporting.
Figure 1 Elastic query used on scaled-out data tier
Customer scenarios for elastic query are characterized by the following topologies:
Vertical partitioning - Cross-database queries (Topology 1): The data is partitioned vertically between a
number of databases in a data tier. Typically, different sets of tables reside on different databases. That means
that the schema is different on different databases. For instance, all tables for inventory are on one database
while all accounting-related tables are on a second database. Common use cases with this topology require
one to query across or to compile reports across tables in several databases.
Horizontal Partitioning - Sharding (Topology 2): Data is partitioned horizontally to distribute rows across
a scaled out data tier. With this approach, the schema is identical on all participating databases. This approach
is also called “sharding”. Sharding can be performed and managed using (1) the elastic database tools
libraries or (2) self-sharding. An elastic query is used to query or compile reports across many shards.

NOTE
Elastic query works best for occasional reporting scenarios where most of the processing can be performed on the data
tier. For heavy reporting workloads or data warehousing scenarios with more complex queries, also consider using Azure
SQL Data Warehouse.

Vertical partitioning - cross-database queries

To begin coding, see Getting started with cross-database query (vertical partitioning).
An elastic query can be used to make data located in a SQL database available to other SQL databases. This
allows queries from one database to refer to tables in any other remote SQL database. The first step is to define
an external data source for each remote database. The external data source is defined in the local database from
which you want to gain access to tables located on the remote database. No changes are necessary on the
remote database. For typical vertical partitioning scenarios where different databases have different schemas,
elastic queries can be used to implement common use cases such as access to reference data and cross-database
querying.

IMPORTANT
You must possess ALTER ANY EXTERNAL DATA SOURCE permission. This permission is included with the ALTER
DATABASE permission. ALTER ANY EXTERNAL DATA SOURCE permissions are needed to refer to the underlying data
source.

Reference data: The topology is used for reference data management. In the figure below, two tables (T1 and
T2) with reference data are kept on a dedicated database. Using an elastic query, you can now access tables T1
and T2 remotely from other databases, as shown in the figure. Use topology 1 if reference tables are small or
remote queries into reference table have selective predicates.
Figure 2 Vertical partitioning - Using elastic query to query reference data

Cross-database querying: Elastic queries enable use cases that require querying across several SQL databases.
Figure 3 shows four different databases: CRM, Inventory, HR and Products. Queries performed in one of the
databases also need access to one or all the other databases. Using an elastic query, you can configure your
database for this case by running a few simple DDL statements on each of the four databases. After this one-
time configuration, access to a remote table is as simple as referring to a local table from your T-SQL queries or
from your BI tools. This approach is recommended if the remote queries do not return large results.
Figure 3 Vertical partitioning - Using elastic query to query across various databases
The following steps configure elastic database queries for vertical partitioning scenarios that require access to a
table located on remote SQL databases with the same schema:
CREATE MASTER KEY mymasterkey
CREATE DATABASE SCOPED CREDENTIAL mycredential
CREATE/DROP EXTERNAL DATA SOURCE mydatasource of type RDBMS
CREATE/DROP EXTERNAL TABLE mytable
After running the DDL statements, you can access the remote table “mytable” as though it were a local table.
Azure SQL Database automatically opens a connection to the remote database, processes your request on the
remote database, and returns the results.

Horizontal partitioning - sharding

Using elastic query to perform reporting tasks over a sharded, i.e., horizontally partitioned, data tier requires an
elastic database shard map to represent the databases of the data tier . Typically, only a single shard map is used
in this scenario and a dedicated database with elastic query capabilities serves as the entry point for reporting
queries. Only this dedicated database needs access to the shard map. Figure 4 illustrates this topology and its
configuration with the elastic query database and shard map. The databases in the data tier can be of any Azure
SQL Database version or edition. For more information about the elastic database client library and creating
shard maps, see Shard map management.
Figure 4 Horizontal partitioning - Using elastic query for reporting over sharded data tiers
The following steps configure elastic database queries for horizontal partitioning scenarios that require access to
a set of table that are located on (typically) several remote SQL databases:
CREATE MASTER KEY mymasterkey
CREATE DATABASE SCOPED CREDENTIAL mycredential
Create a shard map representing your data tier using the elastic database client library.
CREATE/DROP EXTERNAL DATA SOURCE mydatasource of type SHARD_MAP_MANAGER
CREATE/DROP EXTERNAL TABLE mytable
Once you have performed these steps, you can access the horizontally partitioned table “mytable” as though it
were a local table. Azure SQL Database automatically opens multiple parallel connections to the remote
databases where the tables are physically stored, processes the requests on the remote databases, and returns
the results. More information on the steps required for the horizontal partitioning scenario can be found in
elastic query for horizontal partitioning.
To begin coding, see Getting started with elastic query for horizontal partitioning (sharding).

T-SQL querying
Once you have defined your external data sources and your external tables, you can use regular SQL Server
connection strings to connect to the databases where you defined your external tables. You can then run T-SQL
statements over your external tables on that connection with the limitations outlined below. You can find more
information and examples of T-SQL queries in the documentation topics for horizontal partitioning and vertical
partitioning.

Connectivity for tools

You can use regular SQL Server connection strings to connect your applications and BI or data integration tools
to databases that have external tables. Make sure that SQL Server is supported as a data source for your tool.
Once connected, refer to the elastic query database and the external tables in that database just like you would
do with any other SQL Server database that you connect to with your tool.

IMPORTANT
Authentication using Azure Active Directory with elastic queries is not currently supported.

Cost
Elastic query is included into the cost of Azure SQL Database databases. Note that topologies where your remote
databases are in a different data center than the elastic query endpoint are supported, but data egress from
remote databases are charged regular Azure rates.

Preview limitations
Running your first elastic query can take up to a few minutes on the Standard performance tier. This time is
necessary to load the elastic query functionality; loading performance improves with higher performance
tiers.
Scripting of external data sources or external tables from SSMS or SSDT is not yet supported.
Import/Export for SQL DB does not yet support external data sources and external tables. If you need to use
Import/Export, drop these objects before exporting and then re-create them after importing.
Elastic query currently only supports read-only access to external tables. You can, however, use full T-SQL
functionality on the database where the external table is defined. This can be useful to, e.g., persist temporary
results using, e.g., SELECT INTO , or to define stored procedures on the elastic query database which refer to
external tables.
Except for nvarchar(max), LOB types are not supported in external table definitions. As a workaround, you can
create a view on the remote database that casts the LOB type into nvarchar(max), define your external table
over the view instead of the base table and then cast it back into the original LOB type in your queries.
Column statistics over external tables are currently not supported. Tables statistics are supported, but need to
be created manually.

Feedback
Please share feedback on your experience with elastic queries with us on Disqus below, the MSDN forums, or on
Stackoverflow. We are interested in all kinds of feedback about the service (defects, rough edges, feature gaps).

Next steps
For a vertical partitioning tutorial, see Getting started with cross-database query (vertical partitioning).
For syntax and sample queries for vertically partitioned data, see Querying vertically partitioned data)
For a horizontal partitioning (sharding) tutorial, see Getting started with elastic query for horizontal
partitioning (sharding).
For syntax and sample queries for horizontally partitioned data, see Querying horizontally partitioned data)
See sp_execute _remote for a stored procedure that executes a Transact-SQL statement on a single remote
Azure SQL Database or set of databases serving as shards in a horizontal partitioning scheme.
Reporting across scaled-out cloud databases
(preview)
2/21/2017 • 7 min to read • Edit Online

Sharded databases distribute rows across a scaled out data tier. The schema is identical on all participating
databases, also known as horizontal partitioning. Using an elastic query, you can create reports that span all
databases in a sharded database.
For a quick start, see Reporting across scaled-out cloud databases.
For non-sharded databases, see Query across cloud databases with different schemas.

Prerequisites
Create a shard map using the elastic database client library. see Shard map management. Or use the sample
app in Get started with elastic database tools.
Alternatively, see Migrate existing databases to scaled-out databases.
The user must possess ALTER ANY EXTERNAL DATA SOURCE permission. This permission is included with the
ALTER DATABASE permission.
ALTER ANY EXTERNAL DATA SOURCE permissions are needed to refer to the underlying data source.

Overview
These statements create the metadata representation of your sharded data tier in the elastic query database.
1. CREATE MASTER KEY
2. CREATE DATABASE SCOPED CREDENTIAL
3. CREATE EXTERNAL DATA SOURCE
4. CREATE EXTERNAL TABLE

1.1 Create database scoped master key and credentials

The credential is used by the elastic query to connect to your remote databases.
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password';
CREATE DATABASE SCOPED CREDENTIAL <credential_name> WITH IDENTITY = '<username>',
SECRET = '<password>'
[;]

NOTE
Make sure that the "<username>" does not include any "@servername" suffix.

1.2 Create external data sources

Syntax:

<External_Data_Source> ::=
CREATE EXTERNAL DATA SOURCE <data_source_name> WITH
(TYPE = SHARD_MAP_MANAGER,
LOCATION = '<fully_qualified_server_name>',
DATABASE_NAME = ‘<shardmap_database_name>',
CREDENTIAL = <credential_name>,
SHARD_MAP_NAME = ‘<shardmapname>’
) [;]

Example

CREATE EXTERNAL DATA SOURCE MyExtSrc

WITH
(
TYPE=SHARD_MAP_MANAGER,
LOCATION='myserver.database.windows.net',
DATABASE_NAME='ShardMapDatabase',
CREDENTIAL= SMMUser,
SHARD_MAP_NAME='ShardMap'
);

Retrieve the list of current external data sources:

select * from sys.external_data_sources;

The external data source references your shard map. An elastic query then uses the external data source and the
underlying shard map to enumerate the databases that participate in the data tier. The same credentials are used
to read the shard map and to access the data on the shards during the processing of an elastic query.

1.3 Create external tables

Syntax:
CREATE EXTERNAL TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name
( { <column_definition> } [ ,...n ])
{ WITH ( <sharded_external_table_options> ) }
) [;]

<sharded_external_table_options> ::=
DATA_SOURCE = <External_Data_Source>,
[ SCHEMA_NAME = N'nonescaped_schema_name',]
[ OBJECT_NAME = N'nonescaped_object_name',]
DISTRIBUTION = SHARDED(<sharding_column_name>) | REPLICATED |ROUND_ROBIN

Example

CREATE EXTERNAL TABLE [dbo].[order_line](

[ol_o_id] int NOT NULL,
[ol_d_id] tinyint NOT NULL,
[ol_w_id] int NOT NULL,
[ol_number] tinyint NOT NULL,
[ol_i_id] int NOT NULL,
[ol_delivery_d] datetime NOT NULL,
[ol_amount] smallmoney NOT NULL,
[ol_supply_w_id] int NOT NULL,
[ol_quantity] smallint NOT NULL,
[ol_dist_info] char(24) NOT NULL
)

WITH
(
DATA_SOURCE = MyExtSrc,
SCHEMA_NAME = 'orders',
OBJECT_NAME = 'order_details',
DISTRIBUTION=SHARDED(ol_w_id)
);

Retrieve the list of external tables from the current database:

SELECT * from sys.external_tables;

To drop external tables:

DROP EXTERNAL TABLE [ database_name . [ schema_name ] . | schema_name. ] table_name[;]

Remarks
The DATA_SOURCE clause defines the external data source (a shard map) that is used for the external table.
The SCHEMA_NAME and OBJECT_NAME clauses map the external table definition to a table in a different schema.
If omitted, the schema of the remote object is assumed to be “dbo” and its name is assumed to be identical to the
external table name being defined. This is useful if the name of your remote table is already taken in the database
where you want to create the external table. For example, you want to define an external table to get an aggregate
view of catalog views or DMVs on your scaled out data tier. Since catalog views and DMVs already exist locally,
you cannot use their names for the external table definition. Instead, use a different name and use the catalog
view’s or the DMV’s name in the SCHEMA_NAME and/or OBJECT_NAME clauses. (See the example below.)
The DISTRIBUTION clause specifies the data distribution used for this table. The query processor utilizes the
information provided in the DISTRIBUTION clause to build the most efficient query plans.
1. SHARDED means data is horizontally partitioned across the databases. The partitioning key for the data
distribution is the parameter.
2. REPLICATED means that identical copies of the table are present on each database. It is your responsibility to
ensure that the replicas are identical across the databases.
3. ROUND_ROBIN means that the table is horizontally partitioned using an application-dependent distribution
method.
Data tier reference: The external table DDL refers to an external data source. The external data source specifies a
shard map which provides the external table with the information necessary to locate all the databases in your
data tier.
Security considerations
Users with access to the external table automatically gain access to the underlying remote tables under the
credential given in the external data source definition. Avoid undesired elevation of privileges through the
credential of the external data source. Use GRANT or REVOKE for an external table just as though it were a regular
table.
Once you have defined your external data source and your external tables, you can now use full T-SQL over your
external tables.

Example: querying horizontal partitioned databases

The following query performs a three-way join between warehouses, orders and order lines and uses several
aggregates and a selective filter. It assumes (1) horizontal partitioning (sharding) and (2) that warehouses, orders
and order lines are sharded by the warehouse id column, and that the elastic query can co-locate the joins on the
shards and process the expensive part of the query on the shards in parallel.

select
w_id as warehouse,
o_c_id as customer,
count(*) as cnt_orderline,
max(ol_quantity) as max_quantity,
avg(ol_amount) as avg_amount,
min(ol_delivery_d) as min_deliv_date
from warehouse
join orders
on w_id = o_w_id
join order_line
on o_id = ol_o_id and o_w_id = ol_w_id
where w_id > 100 and w_id < 200
group by w_id, o_c_id

Stored procedure for remote T-SQL execution: sp_execute_remote

Elastic query also introduces a stored procedure that provides direct access to the shards. The stored procedure is
called sp_execute _remote and can be used to execute remote stored procedures or T-SQL code on the remote
databases. It takes the following parameters:
Data source name (nvarchar): The name of the external data source of type RDBMS.
Query (nvarchar): The T-SQL query to be executed on each shard.
Parameter declaration (nvarchar) - optional: String with data type definitions for the parameters used in the
Query parameter (like sp_executesql).
Parameter value list - optional: Comma-separated list of parameter values (like sp_executesql).
The sp_execute_remote uses the external data source provided in the invocation parameters to execute the given
T-SQL statement on the remote databases. It uses the credential of the external data source to connect to the
shardmap manager database and the remote databases.
Example:

EXEC sp_execute_remote
N'MyExtSrc',
N'select count(w_id) as foo from warehouse'

Connectivity for tools

Use regular SQL Server connection strings to connect your application, your BI and data integration tools to the
database with your external table definitions. Make sure that SQL Server is supported as a data source for your
tool. Then reference the elastic query database like any other SQL Server database connected to the tool, and use
external tables from your tool or application as if they were local tables.

Best practices
Ensure that the elastic query endpoint database has been given access to the shardmap database and all
shards through the SQL DB firewalls.
Validate or enforce the data distribution defined by the external table. If your actual data distribution is
different from the distribution specified in your table definition, your queries may yield unexpected results.
Elastic query currently does not perform shard elimination when predicates over the sharding key would allow
it to safely exclude certain shards from processing.
Elastic query works best for queries where most of the computation can be done on the shards. You typically
get the best query performance with selective filter predicates that can be evaluated on the shards or joins
over the partitioning keys that can be performed in a partition-aligned way on all shards. Other query patterns
may need to load large amounts of data from the shards to the head node and may perform poorly

Next steps
For an overview of elastic query, see Elastic query overview.
For a vertical partitioning tutorial, see Getting started with cross-database query (vertical partitioning).
For syntax and sample queries for vertically partitioned data, see Querying vertically partitioned data)
For a horizontal partitioning (sharding) tutorial, see Getting started with elastic query for horizontal
partitioning (sharding).
See sp_execute _remote for a stored procedure that executes a Transact-SQL statement on a single remote
Azure SQL Database or set of databases serving as shards in a horizontal partitioning scheme.
Query across cloud databases with different schemas
(preview)
2/21/2017 • 6 min to read • Edit Online

Vertically-partitioned databases use different sets of tables on different databases. That means that the schema is
different on different databases. For instance, all tables for inventory are on one database while all accounting-
related tables are on a second database.

Prerequisites
The user must possess ALTER ANY EXTERNAL DATA SOURCE permission. This permission is included with the
ALTER DATABASE permission.
ALTER ANY EXTERNAL DATA SOURCE permissions are needed to refer to the underlying data source.

Overview
NOTE
Unlike with horizontal partitioning, these DDL statements do not depend on defining a data tier with a shard map through
the elastic database client library.

1. CREATE MASTER KEY

2. CREATE DATABASE SCOPED CREDENTIAL
3. CREATE EXTERNAL DATA SOURCE
4. CREATE EXTERNAL TABLE
Create database scoped master key and credentials
The credential is used by the elastic query to connect to your remote databases.

CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password';

CREATE DATABASE SCOPED CREDENTIAL <credential_name> WITH IDENTITY = '<username>',
SECRET = '<password>'
[;]

NOTE
Ensure that the <username> does not include any "@servername" suffix.

Create external data sources

Syntax:

<External_Data_Source> ::=
CREATE EXTERNAL DATA SOURCE <data_source_name> WITH
(TYPE = RDBMS,
LOCATION = ’<fully_qualified_server_name>’,
DATABASE_NAME = ‘<remote_database_name>’,
CREDENTIAL = <credential_name>
) [;]

IMPORTANT
The TYPE parameter must be set to RDBMS.

Example
The following example illustrates the use of the CREATE statement for external data sources.

CREATE EXTERNAL DATA SOURCE RemoteReferenceData

WITH
(
TYPE=RDBMS,
LOCATION='myserver.database.windows.net',
DATABASE_NAME='ReferenceData',
CREDENTIAL= SqlUser
);

To retrieve the list of current external data sources:

select * from sys.external_data_sources;

External Tables
Syntax:
CREATE EXTERNAL TABLE [ database_name . [ schema_name ] . | schema_name . ] table_name
( { <column_definition> } [ ,...n ])
{ WITH ( <rdbms_external_table_options> ) }
)[;]

<rdbms_external_table_options> ::=
DATA_SOURCE = <External_Data_Source>,
[ SCHEMA_NAME = N'nonescaped_schema_name',]
[ OBJECT_NAME = N'nonescaped_object_name',]

Example

CREATE EXTERNAL TABLE [dbo].[customer](

[c_id] int NOT NULL,
[c_firstname] nvarchar(256) NULL,
[c_lastname] nvarchar(256) NOT NULL,
[street] nvarchar(256) NOT NULL,
[city] nvarchar(256) NOT NULL,
[state] nvarchar(20) NULL,
[country] nvarchar(50) NOT NULL,
)
WITH
(
DATA_SOURCE = RemoteReferenceData
);

The following example shows how to retrieve the list of external tables from the current database:

select * from sys.external_tables;

Remarks
Elastic query extends the existing external table syntax to define external tables that use external data sources of
type RDBMS. An external table definition for vertical partitioning covers the following aspects:
Schema: The external table DDL defines a schema that your queries can use. The schema provided in your
external table definition needs to match the schema of the tables in the remote database where the actual data
is stored.
Remote database reference: The external table DDL refers to an external data source. The external data
source specifies the logical server name and database name of the remote database where the actual table
data is stored.
Using an external data source as outlined in the previous section, the syntax to create external tables is as follows:
The DATA_SOURCE clause defines the external data source (i.e. the remote database in case of vertical
partitioning) that is used for the external table.
The SCHEMA_NAME and OBJECT_NAME clauses provide the ability to map the external table definition to a table
in a different schema on the remote database, or to a table with a different name, respectively. This is useful if you
want to define an external table to a catalog view or DMV on your remote database - or any other situation where
the remote table name is already taken locally.
The following DDL statement drops an existing external table definition from the local catalog. It does not impact
the remote database.

DROP EXTERNAL TABLE [ [ schema_name ] . | schema_name. ] table_name[;]

Permissions for CREATE/DROP EXTERNAL TABLE: ALTER ANY EXTERNAL DATA SOURCE permissions are
needed for external table DDL which is also needed to refer to the underlying data source.

Security considerations
Users with access to the external table automatically gain access to the underlying remote tables under the
credential given in the external data source definition. You should carefully manage access to the external table in
order to avoid undesired elevation of privileges through the credential of the external data source. Regular SQL
permissions can be used to GRANT or REVOKE access to an external table just as though it were a regular table.

Example: querying vertically partitioned databases

The following query performs a three-way join between the two local tables for orders and order lines and the
remote table for customers. This is an example of the reference data use case for elastic query:

SELECT
c_id as customer,
c_lastname as customer_name,
count(*) as cnt_orderline,
max(ol_quantity) as max_quantity,
avg(ol_amount) as avg_amount,
min(ol_delivery_d) as min_deliv_date
FROM customer
JOIN orders
ON c_id = o_c_id
JOIN order_line
ON o_id = ol_o_id and o_c_id = ol_c_id
WHERE c_id = 100

Stored procedure for remote T-SQL execution: sp_execute_remote

EXEC sp_execute_remote
N'MyExtSrc',
N'select count(w_id) as foo from warehouse'

Connectivity for tools

You can use regular SQL Server connection strings to connect your BI and data integration tools to databases on
the SQL DB server that has elastic query enabled and external tables defined. Make sure that SQL Server is
supported as a data source for your tool. Then refer to the elastic query database and its external tables just like
any other SQL Server database that you would connect to with your tool.

Best practices
Ensure that the elastic query endpoint database has been given access to the remote database by enabling
access for Azure Services in its SQL DB firewall configuration. Also ensure that the credential provided in the
external data source definition can successfully log into the remote database and has the permissions to access
the remote table.
Elastic query works best for queries where most of the computation can be done on the remote databases. You
typically get the best query performance with selective filter predicates that can be evaluated on the remote
databases or joins that can be performed completely on the remote database. Other query patterns may need
to load large amounts of data from the remote database and may perform poorly.

Next steps
For an overview of elastic query, see Elastic query overview.
For a vertical partitioning tutorial, see Getting started with cross-database query (vertical partitioning).
For a horizontal partitioning (sharding) tutorial, see Getting started with elastic query for horizontal
partitioning (sharding).
For syntax and sample queries for horizontally partitioned data, see Querying horizontally partitioned data)
See sp_execute _remote for a stored procedure that executes a Transact-SQL statement on a single remote
Azure SQL Database or set of databases serving as shards in a horizontal partitioning scheme.
Distributed transactions across cloud databases
4/10/2017 • 7 min to read • Edit Online

Elastic database transactions for Azure SQL Database (SQL DB) allow you to run transactions that span several
databases in SQL DB. Elastic database transactions for SQL DB are available for .NET applications using ADO .NET
and integrate with the familiar programming experience using the System.Transaction classes. To get the library,
see .NET Framework 4.6.1 (Web Installer).
On premises, such a scenario usually required running Microsoft Distributed Transaction Coordinator (MSDTC).
Since MSDTC is not available for Platform-as-a-Service application in Azure, the ability to coordinate distributed
transactions has now been directly integrated into SQL DB. Applications can connect to any SQL Database to
launch distributed transactions, and one of the databases will transparently coordinate the distributed transaction,
as shown in the following figure.

Common scenarios
Elastic database transactions for SQL DB enable applications to make atomic changes to data stored in several
different SQL Databases. The preview focuses on client-side development experiences in C# and .NET. A server-
side experience using T-SQL is planned for a later time.
Elastic database transactions targets the following scenarios:
Multi-database applications in Azure: With this scenario, data is vertically partitioned across several databases
in SQL DB such that different kinds of data reside on different databases. Some operations require changes to
data which is kept in two or more databases. The application uses elastic database transactions to coordinate
the changes across databases and ensure atomicity.
Sharded database applications in Azure: With this scenario, the data tier uses the Elastic Database client library
or self-sharding to horizontally partition the data across many databases in SQL DB. One prominent use case is
the need to perform atomic changes for a sharded multi-tenant application when changes span tenants. Think
for instance of a transfer from one tenant to another, both residing on different databases. A second case is
fine-grained sharding to accommodate capacity needs for a large tenant which in turn typically implies that
some atomic operations needs to stretch across several databases used for the same tenant. A third case is
atomic updates to reference data that are replicated across databases. Atomic, transacted, operations along
these lines can now be coordinated across several databases using the preview. Elastic database transactions
use two-phase commit to ensure transaction atomicity across databases. It is a good fit for transactions that
involve less than 100 databases at a time within a single transaction. These limits are not enforced, but one
should expect performance and success rates for elastic database transactions to suffer when exceeding these
limits.

Installation and migration

The capabilities for elastic database transactions in SQL DB are provided through updates to the .NET libraries
System.Data.dll and System.Transactions.dll. The DLLs ensure that two-phase commit is used where necessary to
ensure atomicity. To start developing applications using elastic database transactions, install .NET Framework 4.6.1
or a later version. When running on an earlier version of the .NET framework, transactions will fail to promote to a
distributed transaction and an exception will be raised.
After installation, you can use the distributed transaction APIs in System.Transactions with connections to SQL DB.
If you have existing MSDTC applications using these APIs, simply rebuild your existing applications for .NET 4.6
after installing the 4.6.1 Framework. If your projects target .NET 4.6, they will automatically use the updated DLLs
from the new Framework version and distributed transaction API calls in combination with connections to SQL DB
will now succeed.
Remember that elastic database transactions do not require installing MSDTC. Instead, elastic database
transactions are directly managed by and within SQL DB. This significantly simplifies cloud scenarios since a
deployment of MSDTC is not necessary to use distributed transactions with SQL DB. Section 4 explains in more
detail how to deploy elastic database transactions and the required .NET framework together with your cloud
applications to Azure.

Development experience
Multi-database applications
The following sample code uses the familiar programming experience with .NET System.Transactions. The
TransactionScope class establishes an ambient transaction in .NET. (An “ambient transaction” is one that lives in
the current thread.) All connections opened within the TransactionScope participate in the transaction. If different
databases participate, the transaction is automatically elevated to a distributed transaction. The outcome of the
transaction is controlled by setting the scope to complete to indicate a commit.

using (var scope = new TransactionScope())

{
using (var conn1 = new SqlConnection(connStrDb1))
{
conn1.Open();
SqlCommand cmd1 = conn1.CreateCommand();
cmd1.CommandText = string.Format("insert into T1 values(1)");
cmd1.ExecuteNonQuery();
}

using (var conn2 = new SqlConnection(connStrDb2))

{
conn2.Open();
var cmd2 = conn2.CreateCommand();
cmd2.CommandText = string.Format("insert into T2 values(2)");
cmd2.ExecuteNonQuery();
}

scope.Complete();
}
Sharded database applications
Elastic database transactions for SQL DB also support coordinating distributed transactions where you use the
OpenConnectionForKey method of the elastic database client library to open connections for a scaled out data tier.
Consider cases where you need to guarantee transactional consistency for changes across several different
sharding key values. Connections to the shards hosting the different sharding key values are brokered using
OpenConnectionForKey. In the general case, the connections can be to different shards such that ensuring
transactional guarantees requires a distributed transaction. The following code sample illustrates this approach. It
assumes that a variable called shardmap is used to represent a shard map from the elastic database client library:

using (var scope = new TransactionScope())

{
using (var conn1 = shardmap.OpenConnectionForKey(tenantId1, credentialsStr))
{
conn1.Open();
SqlCommand cmd1 = conn1.CreateCommand();
cmd1.CommandText = string.Format("insert into T1 values(1)");
cmd1.ExecuteNonQuery();
}

using (var conn2 = shardmap.OpenConnectionForKey(tenantId2, credentialsStr))

{
conn2.Open();
var cmd2 = conn2.CreateCommand();
cmd2.CommandText = string.Format("insert into T1 values(2)");
cmd2.ExecuteNonQuery();
}

scope.Complete();
}

.NET installation for Azure Cloud Services

Azure provides several offerings to host .NET applications. A comparison of the different offerings is available in
Azure App Service, Cloud Services, and Virtual Machines comparison. If the guest OS of the offering is smaller than
.NET 4.6.1 required for elastic transactions, you need to upgrade the guest OS to 4.6.1.
For Azure App Services, upgrades to the guest OS are currently not supported. For Azure Virtual Machines, simply
log into the VM and run the installer for the latest .NET framework. For Azure Cloud Services, you need to include
the installation of a newer .NET version into the startup tasks of your deployment. The concepts and steps are
documented in Install .NET on a Cloud Service Role.
Note that the installer for .NET 4.6.1 may require more temporary storage during the bootstrapping process on
Azure cloud services than the installer for .NET 4.6. To ensure a successful installation, you need to increase
temporary storage for your Azure cloud service in your ServiceDefinition.csdef file in the LocalResources section
and the environment settings of your startup task, as shown in the following sample:
<LocalResources>
...
<LocalStorage name="TEMP" sizeInMB="5000" cleanOnRoleRecycle="false" />
<LocalStorage name="TMP" sizeInMB="5000" cleanOnRoleRecycle="false" />
</LocalResources>
<Startup>
<Task commandLine="install.cmd" executionContext="elevated" taskType="simple">
<Environment>
...
<Variable name="TEMP">
<RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/LocalResources/LocalResource[@name='TEMP']/@path" />
</Variable>
<Variable name="TMP">
<RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/LocalResources/LocalResource[@name='TMP']/@path" />
</Variable>
</Environment>
</Task>
</Startup>

Transactions across multiple servers

Elastic database transactions are supported across different logical servers in Azure SQL Database. When
transactions cross logical server boundaries, the participating servers first need to be entered into a mutual
communication relationship. Once the communication relationship has been established, any database in any of
the two servers can participate in elastic transactions with databases from the other server. With transactions
spanning more than two logical servers, a communication relationship needs to be in place for any pair of logical
servers.
Use the following PowerShell cmdlets to manage cross-server communication relationships for elastic database
transactions:
New-AzureRmSqlServerCommunicationLink: Use this cmdlet to create a new communication relationship
between two logical servers in Azure SQL DB. The relationship is symmetric which means both servers can
initiate transactions with the other server.
Get-AzureRmSqlServerCommunicationLink: Use this cmdlet to retrieve existing communication
relationships and their properties.
Remove-AzureRmSqlServerCommunicationLink: Use this cmdlet to remove an existing communication
relationship.

Monitoring transaction status

Use Dynamic Management Views (DMVs) in SQL DB to monitor status and progress of your ongoing elastic
database transactions. All DMVs related to transactions are relevant for distributed transactions in SQL DB. You can
find the corresponding list of DMVs here: Transaction Related Dynamic Management Views and Functions
(Transact-SQL).
These DMVs are particularly useful:
sys.dm_tran_active_transactions: Lists currently active transactions and their status. The UOW (Unit Of Work)
column can identify the different child transactions that belong to the same distributed transaction. All
transactions within the same distributed transaction carry the same UOW value. See the DMV documentation
for more details.
sys.dm_tran_database_transactions: Provides additional information about transactions, such as placement
of the transaction in the log. See the DMV documentation for more details.
sys.dm_tran_locks: Provides information about the locks that are currently held by ongoing transactions. See
the DMV documentation for more details.
Limitations
The following limitations currently apply to elastic database transactions in SQL DB:
Only transactions across databases in SQL DB are supported. Other X/Open XA resource providers and
databases outside of SQL DB cannot participate in elastic database transactions. That means that elastic
database transactions cannot stretch across on premises SQL Server and Azure SQL Databases. For distributed
transactions on premises, continue to use MSDTC.
Only client-coordinated transactions from a .NET application are supported. Server-side support for T-SQL such
as BEGIN DISTRIBUTED TRANSACTION is planned, but not yet available.
Transactions across WCF services are not supported. For example, you have a WCF service method that
executes a transaction. Enclosing the call within a transaction scope will fail as a
System.ServiceModel.ProtocolException.

Next steps
Not yet using elastic database capabilities for your Azure applications? Check out our Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL
Database feedback forum.
Managing scaled-out cloud databases
1/17/2017 • 7 min to read • Edit Online

To manage scaled-out sharded databases, the Elastic Database jobs feature (preview) enables you to reliably
execute a Transact-SQL (T-SQL) script across a group of databases, including:
a custom-defined collection of databases (explained below)
all databases in an elastic pool
a shard set (created using Elastic Database client library).

Documentation
Install the Elastic Database job components.
Get started with Elastic Database jobs.
Create and manage jobs using PowerShell.
Create and manage scaled out Azure SQL Databases
Elastic Database jobs is currently a customer-hosted Azure Cloud Service that enables the execution of ad-hoc
and scheduled administrative tasks, which are called jobs. With jobs, you can easily and reliably manage large
groups of Azure SQL Databases by running Transact-SQL scripts to perform administrative operations.

Why use jobs?

Manage
Easily do schema changes, credentials management, reference data updates, performance data collection or
tenant (customer) telemetry collection.
Reports
Aggregate data from a collection of Azure SQL Databases into a single destination table.
Reduce overhead
Normally, you must connect to each database independently in order to run Transact-SQL statements or perform
other administrative tasks. A job handles the task of logging in to each database in the target group. You also
define, maintain and persist Transact-SQL scripts to be executed across a group of Azure SQL Databases.
Accounting
Jobs run the script and log the status of execution for each database. You also get automatic retry when failures
occur.
Flexibility
Define custom groups of Azure SQL Databases, and define schedules for running a job.

NOTE
In the Azure portal, only a reduced set of functions limited to SQL Azure elastic pools is available. Use the PowerShell APIs
to access the full set of current functionality.

Applications
Perform administrative tasks, such as deploying a new schema.
Update reference data-product information common across all databases. Or schedules automatic updates
every weekday, after hours.
Rebuild indexes to improve query performance. The rebuilding can be configured to execute across a
collection of databases on a recurring basis, such as during off-peak hours.
Collect query results from a set of databases into a central table on an on-going basis. Performance queries
can be continually executed and configured to trigger additional tasks to be executed.
Execute longer running data processing queries across a large set of databases, for example the collection of
customer telemetry. Results are collected into a single destination table for further analysis.

Elastic Database jobs: end-to-end

1. Install the Elastic Database jobs components. For more information, see Installing Elastic Database jobs. If
the installation fails, see how to uninstall.
2. Use the PowerShell APIs to access more functionality, for example creating custom-defined database
collections, adding schedules and/or gathering results sets. Use the portal for simple installation and
creation/monitoring of jobs limited to execution against a elastic pool.
3. Create encrypted credentials for job execution and add the user (or role) to each database in the group.
4. Create an idempotent T-SQL script that can be run against every database in the group.
5. Follow these steps to create jobs using the Azure portal: Creating and managing Elastic Database jobs.
6. Or use PowerShell scripts: Create and manage a SQL Database elastic database jobs using PowerShell
(preview).

Idempotent scripts
The scripts must be idempotent. In simple terms, "idempotent" means that if the script succeeds, and it is run
again, the same result occurs. A script may fail due to transient network issues. In that case, the job will
automatically retry running the script a preset number of times before desisting. An idempotent script has the
same result even if has been successfully run twice.
A simple tactic is to test for the existence of an object before creating it.
IF NOT EXIST (some_object)
-- Create the object
-- If it exists, drop the object before recreating it.

Similarly, a script must be able to execute successfully by logically testing for and countering any conditions it
finds.

Failures and logs

If a script fails after multiple attempts, the job logs the error and continues. After a job ends (meaning a run
against all databases in the group), you can check its list of failed attempts. The logs provide details to debug
faulty scripts.

Group types and creation

There are two kinds of groups:
1. Shard sets
2. Custom groups
Shard set groups are created using the Elastic Database tools. When you create a shard set group, databases are
added or removed from the group automatically. For example, a new shard will be automatically in the group
when you add it to the shard map. A job can then be run against the group.
Custom groups, on the other hand, are rigidly defined. You must explicitly add or remove databases from custom
groups. If a database in the group is dropped, the job will attempt to run the script against the database resulting
in an eventual failure. Groups created using the Azure portal currently are custom groups.

Components and pricing

The following components work together to create an Azure Cloud service that enables ad-hoc execution of
administrative jobs. The components are installed and configured automatically during setup, in your
subscription. You can identify the services as they all have the same auto-generated name. The name is unique,
and consists of the prefix "edj" followed by 21 randomly generated characters.
Azure Cloud Service: elastic database jobs (preview) is delivered as a customer-hosted Azure Cloud service
to perform execution of the requested tasks. From the portal, the service is deployed and hosted in your
Microsoft Azure subscription. The default deployed service runs with the minimum of two worker roles for
high availability. The default size of each worker role (ElasticDatabaseJobWorker) runs on an A0 instance. For
pricing, see Cloud services pricing.
Azure SQL Database: The service uses an Azure SQL Database known as the control database to store all
of the job metadata. The default service tier is a S0. For pricing, see SQL Database Pricing.
Azure Service Bus: An Azure Service Bus is for coordination of the work within the Azure Cloud Service. See
Service Bus Pricing.
Azure Storage: An Azure Storage account is used to store diagnostic output logging in the event that an
issue requires further debugging (see Enabling Diagnostics in Azure Cloud Services and Virtual Machines). For
pricing, see Azure Storage Pricing.

How Elastic Database jobs work

1. An Azure SQL Database is designated a control database which stores all meta-data and state data.
2. The control database is accessed by the job service to launch and track jobs to execute.
3. Two different roles communicate with the control database:
Controller: Determines which jobs require tasks to perform the requested job, and retries failed jobs by
creating new job tasks.
Job Task Execution: Carries out the job tasks.
Job task types
There are multiple types of job tasks that carry out execution of jobs:
ShardMapRefresh: Queries the shard map to determine all the databases used as shards
ScriptSplit: Splits the script across ‘GO’ statements into batches
ExpandJob: Creates child jobs for each database from a job that targets a group of databases
ScriptExecution: Executes a script against a particular database using defined credentials
Dacpac: Applies a DACPAC to a particular database using particular credentials

End-to-end job execution work-flow

1. Using either the Portal or the PowerShell API, a job is inserted into the control database. The job requests
execution of a Transact-SQL script against a group of databases using specific credentials.
2. The controller identifies the new job. Job tasks are created and executed to split the script and to refresh the
group’s databases. Lastly, a new job is created and executed to expand the job and create new child jobs
where each child job is specified to execute the Transact-SQL script against an individual database in the
group.
3. The controller identifies the created child jobs. For each job, the controller creates and triggers a job task to
execute the script against a database.
4. After all job tasks have completed, the controller updates the jobs to a completed state. At any point during
job execution, the PowerShell API can be used to view the current state of job execution. All times returned by
the PowerShell APIs are represented in UTC. If desired, a cancellation request can be initiated to stop a job.

Next steps
Install the components, then create and add a log in to each database in the group of databases. To further
understand job creation and management, see creating and managing elastic database jobs. See also Getting
started with Elastic Database jobs.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Securing your SQL Database
4/14/2017 • 5 min to read • Edit Online

This article walks through the basics of securing the data tier of an application using Azure SQL Database. In
particular, this article will get you started with resources for protecting data, controlling access, and proactive
monitoring.
For a complete overview of security features available on all flavors of SQL, see the Security Center for SQL
Server Database Engine and Azure SQL Database. Additional information is also available in the Security and
Azure SQL Database technical white paper (PDF).

Protect data
SQL Database secures you data by providing encryption for data in motion using Transport Layer Security, for
data at rest using Transparent Data Encryption, and for data in use using Always Encrypted.

IMPORTANT
All connections to Azure SQL Database require encryption (SSL/TLS) at all times while data is "in transit" to and from the
database. In your application's connection string, you must specify parameters to encrypt the connection and not to trust
the server certificate (this is done for you if you copy your connection string out of the Azure Classic Portal), otherwise the
connection will not verify the identity of the server and will be susceptible to "man-in-the-middle" attacks. For the
ADO.NET driver, for instance, these connection string parameters are Encrypt=True and TrustServerCertificate=False.

For other ways to encrypt your data, consider:

Cell-level encryption to encrypt specific columns or even cells of data with different encryption keys.
If you need a Hardware Security Module or central management of your encryption key hierarchy, consider
using Azure Key Vault with SQL Server in an Azure VM.

Control access
SQL Database secures your data by limiting access to your database using firewall rules, authentication
mechanisms requiring users to prove their identity, and authorization to data through role-based memberships
and permissions, as well as through row-level security and dynamic data masking. For a discussion of the use of
access control features in SQL Database, see Control access.

IMPORTANT
Managing databases and logical servers within Azure is controlled by your portal user account's role assignments. For more
information on this topic, see Role-based access control in Azure Portal.

Firewall and firewall rules

To help protect your data, firewalls prevent all access to your database server until you specify which computers
have permission using firewall rules. The firewall grants access to databases based on the originating IP address
of each request.
Authentication
SQL database authentication refers to how you prove your identity when connecting to the database. SQL
Database supports two types of authentication:
SQL Authentication, which uses a username and password. When you created the logical server for your
database, you specified a "server admin" login with a username and password. Using these credentials, you
can authenticate to any database on that server as the database owner, or "dbo."
Azure Active Directory Authentication, which uses identities managed by Azure Active Directory and is
supported for managed and integrated domains. Use Active Directory authentication (integrated security)
whenever possible. If you want to use Azure Active Directory Authentication, you must create another server
admin called the "Azure AD admin," which is allowed to administer Azure AD users and groups. This admin
can also perform all operations that a regular server admin can. See Connecting to SQL Database By Using
Azure Active Directory Authentication for a walkthrough of how to create an Azure AD admin to enable Azure
Active Directory Authentication.
Authorization
Authorization refers to what a user can do within an Azure SQL Database, and this is controlled by your user
account's database role memberships and object-level permissions. As a best practice, you should grant users the
least privileges necessary. The server admin account you are connecting with is a member of db_owner, which
has authority to do anything within the database. Save this account for deploying schema upgrades and other
management operations. Use the "ApplicationUser" account with more limited permissions to connect from your
application to the database with the least privileges needed by your application.
Row-level security
Row-Level Security enables customers to control access to rows in a database table based on the characteristics
of the user executing a query (e.g., group membership or execution context). For more information, see Row-
Level security.
Data masking
SQL Database Dynamic Data Masking limits sensitive data exposure by masking it to non-privileged users.
Dynamic Data Masking automatically discovers potentially sensitive data in Azure SQL Database and provides
actionable recommendations to mask these fields, with minimal impact on the application layer. It works by
obfuscating the sensitive data in the result set of a query over designated database fields, while the data in the
database is not changed. For more information, see Get started with SQL Database Dynamic Data Masking can be
used to limit exposure of sensitive data.

Proactive monitoring
SQL Database secures your data by providing auditing and threat detection capabilities.
Auditing
SQL Database Auditing tracks database activities and helps you to maintain regulatory compliance, by recording
database events to an audit log in your Azure Storage account. Auditing enables you to understand ongoing
database activities, as well as analyze and investigate historical activity to identify potential threats or suspected
abuse and security violations. For additional information, see Get started with SQL Database Auditing.
Threat detection
Threat Detection complements auditing, by providing an additional layer of security intelligence built into the
Azure SQL Database service. It works around the clock to learn, profile and detect anomalous database activities.
You will be alerted about suspicious activities, potential vulnerabilities, SQL injection attacks and anomalous
database access patterns. You can respond to alerts by following the provided informative and actionable
instructions. For more information, see Get started with SQL Database Threat Detection.
Data Masking
SQL Database Dynamic Data Masking limits sensitive data exposure by masking it to non-privileged users.
Dynamic Data Masking automatically discovers potentially sensitive data in Azure SQL Database and provides
actionable recommendations to mask these fields, with minimal impact on the application layer. It works by
obfuscating the sensitive data in the result set of a query over designated database fields, while the data in the
database is not changed. For more information, see Get started with SQL Database Dynamic Data Masking

Compliance
In addition to the above features and functionality that can help your application meet various security
compliance requirements, Azure SQL Database also participates in regular audits and has been certified against a
number of compliance standards. For more information, see the Microsoft Azure Trust Center, where you can find
the most current list of SQL Database compliance certifications.

Next steps
For a discussion of the use of access control features in SQL Database, see Control access.
For a discussion of database auditing, see SQL Database auditing.
For a discussion of threat detection, see SQL Database threat detection.
Azure SQL Database access control
4/14/2017 • 4 min to read • Edit Online

To provide security, SQL Database controls access with firewall rules limiting connectivity by IP address,
authentication mechanisms requiring users to prove their identity, and authorization mechanisms limiting users
to specific actions and data.

IMPORTANT
For an overview of the SQL Database security features, see SQL security overview. For a tutorial using SQL Server
authentication, see SQL authentication and authorization. For a tutorial using Azure Active Directory authentication, see
Azure AD authentication and authorization.

Firewall and firewall rules

Microsoft Azure SQL Database provides a relational database service for Azure and other Internet-based
applications. To help protect your data, firewalls prevent all access to your database server until you specify which
computers have permission. The firewall grants access to databases based on the originating IP address of each
request. For more information, see Overview of Azure SQL Database firewall rules
The Azure SQL Database service is only available through TCP port 1433. To access a SQL Database from your
computer, ensure that your client computer firewall allows outgoing TCP communication on TCP port 1433. If not
needed for other applications, block inbound connections on TCP port 1433.
As part of the connection process, connections from Azure virtual machines are redirected to a different IP address
and port, unique for each worker role. The port number is in the range from 11000 to 11999. For more
information about TCP ports, see Ports beyond 1433 for ADO.NET 4.5 and SQL Database2.

Authentication
SQL Database supports two types of authentication:
SQL Authentication, which uses a username and password. When you created the logical server for your
database, you specified a "server admin" login with a username and password. Using these credentials, you can
authenticate to any database on that server as the database owner, or "dbo."
Azure Active Directory Authentication, which uses identities managed by Azure Active Directory and is
supported for managed and integrated domains. Use Active Directory authentication (integrated security)
whenever possible. If you want to use Azure Active Directory Authentication, you must create another server
admin called the "Azure AD admin," which is allowed to administer Azure AD users and groups. This admin can
also perform all operations that a regular server admin can. See Connecting to SQL Database By Using Azure
Active Directory Authentication for a walkthrough of how to create an Azure AD admin to enable Azure Active
Directory Authentication.
The Database Engine closes connections that remain idle for more than 30 minutes. The connection must login
again before it can be used. Continuously active connections to SQL Database require reauthorization (performed
by the database engine) at least every 10 hours. The database engine attempts reauthorization using the originally
submitted password and no user input is required. For performance reasons, when a password is reset in SQL
Database, the connection is not be reauthenticated, even if the connection is reset due to connection pooling. This
is different from the behavior of on-premises SQL Server. If the password has been changed since the connection
was initially authorized, the connection must be terminated and a new connection made using the new password.
A user with the KILL DATABASE CONNECTION permission can explicitly terminate a connection to SQL Database by
using the KILL command.
User accounts can be created in the master database and can be granted permissions in all databases on the
server, or they can be created in the database itself (called contained users). For information on creating and
managing logins, see Manage logins. To enhance portability and scalabilty, use contained database users to
enhance scalability. For more information on contained users, see Contained Database Users - Making Your
Database Portable, CREATE USER (Transact-SQL), and Contained Databases.
As a best practice your application should use a dedicated account to authenticate -- this way you can limit the
permissions granted to the application and reduce the risks of malicious activity in case your application code is
vulnerable to a SQL injection attack. The recommended approach is to create a contained database user, which
allows your app to authenticate directly to the database.

Authorization
Authorization refers to what a user can do within an Azure SQL Database, and this is controlled by your user
account's database role memberships and object-level permissions. As a best practice, you should grant users the
least privileges necessary. The server admin account you are connecting with is a member of db_owner, which has
authority to do anything within the database. Save this account for deploying schema upgrades and other
management operations. Use the "ApplicationUser" account with more limited permissions to connect from your
application to the database with the least privileges needed by your application. For more information, see
Manage logins.
Typically, only administrators need access to the master database. Routine access to each user database should be
through non-administrator contained database users created in each database. When you use contained database
users, you do not need to create logins in the master database. For more information, see Contained Database
Users - Making Your Database Portable.
You should familiarize yourself with the following features that can be used to limit or elevate permissions:
Impersonation and module-signing can be used to securely elevate permissions temporarily.
Row-Level Security can be used limit which rows a user can access.
Data Masking can be used to limit exposure of sensitive data.
Stored procedures can be used to limit the actions that can be taken on the database.

Next steps
For an overview of the SQL Database security features, see SQL security overview.
To learn more about firewall rules, see Firewall rules.
To learn about users and logins, see Manage logins.
For a discussion of proactive monitoring, see Database Auditing and SQL Database Threat Detection.
For a tutorial using SQL Server authentication, see SQL authentication and authorization.
For a tutorial using Azure Active Directory authentication, see Azure AD authentication and authorization.
Azure SQL Database server-level and database-
level firewall rules
4/14/2017 • 11 min to read • Edit Online

Overview
Initially, all Transact-SQL access to your Azure SQL server is blocked by the firewall. To begin using your Azure
SQL server, you must specify one or more server-level firewall rules that enable access to your Azure SQL
server. Use the firewall rules to specify which IP address ranges from the Internet are allowed, and whether
Azure applications can attempt to connect to your Azure SQL server.
To selectively grant access to just one of the databases in your Azure SQL server, you must create a database-
level rule for the required database. Specify an IP address range for the database firewall rule that is beyond
the IP address range specified in the server-level firewall rule, and ensure that the IP address of the client falls
in the range specified in the database-level rule.
Connection attempts from the Internet and Azure must first pass through the firewall before they can reach
your Azure SQL server or SQL Database, as shown in the following diagram:
Server-level firewall rules: These rules enable clients to access your entire Azure SQL server, that is, all the
databases within the same logical server. These rules are stored in the master database. Server-level
firewall rules can be configured by using the portal or by using Transact-SQL statements. To create server-
level firewall rules using the Azure portal or PowerShell, you must be the subscription owner or a
subscription contributor. To create a server-level firewall rule using Transact-SQL, you must connect to the
SQL Database instance as the server-level principal login or the Azure Active Directory administrator (which
means that a server-level firewall rule must first be created by a user with Azure-level permissions).
Database-level firewall rules: These rules enable clients to access certain (secure) databases within the
same logical server. You can create these rules for each database (including the master database0) and they
are stored in the individual databases. Database-level firewall rules can only be configured by using
Transact-SQL statements and only after you have configured the first server-level firewall. If you specify an
IP address range in the database-level firewall rule that is outside the range specified in the server-level
firewall rule, only those clients that have IP addresses in the database-level range can access the database.
You can have a maximum of 128 database-level firewall rules for a database. Database-level firewall rules
for master and user databases can only be created and managed through Transact-SQL. For more
information on configuring database-level firewall rules, see the example later in this article and see
sp_set_database_firewall_rule (Azure SQL Databases).
Recommendation: Microsoft recommends using database-level firewall rules whenever possible to enhance
security and to make your database more portable. Use server-level firewall rules for administrators and when
you have many databases that have the same access requirements and you don't want to spend time
configuring each database individually.

NOTE
For information about portable databases in the context of business continuity, see Authentication requirements for
disaster recovery.

Connecting from the Internet

When a computer attempts to connect to your database server from the Internet, the firewall first checks the
originating IP address of the request against the database-level firewall rules, for the database that the
connection is requesting:
If the IP address of the request is within one of the ranges specified in the database-level firewall rules, the
connection is granted to the SQL Database that contains the rule.
If the IP address of the request is not within one of the ranges specified in the database-level firewall rule,
the server-level firewall rules are checked. If the IP address of the request is within one of the ranges
specified in the server-level firewall rules, the connection is granted. Server-level firewall rules apply to all
SQL databases on the Azure SQL server.
If the IP address of the request is not within the ranges specified in any of the database-level or server-level
firewall rules, the connection request fails.

NOTE
To access Azure SQL Database from your local computer, ensure the firewall on your network and local computer allows
outgoing communication on TCP port 1433.

Connecting from Azure

To allow applications from Azure to connect to your Azure SQL server, Azure connections must be enabled.
When an application from Azure attempts to connect to your database server, the firewall verifies that Azure
connections are allowed. A firewall setting with starting and ending address equal to 0.0.0.0 indicates these
connections are allowed. If the connection attempt is not allowed, the request does not reach the Azure SQL
Database server.

IMPORTANT
This option configures the firewall to allow all connections from Azure including connections from the subscriptions of
other customers. When selecting this option, make sure your login and user permissions limit access to only authorized
users.
Creating and managing firewall rules
The first server-level firewall setting can be created using the Azure portal or programmatically using Azure
PowerShell, Azure CLI, or the REST API. Subsequent server-level firewall rules can be created and managed
using these methods, and through Transact-SQL.

IMPORTANT
Database-level firewall rules can only be created and managed using Transact-SQL.

To improve performance, server-level firewall rules are temporarily cached at the database level. To refresh the
cache, see DBCC FLUSHAUTHCACHE.
Azure portal
To set a server-level firewall rule in the Azure portal, you can either go to the Overview page for your Azure
SQL database or the Overview page for your Azure Database logical server.

TIP
For a tutorial, see Create a DB using the Azure portal.

From database overview page

1. To set a server-level firewall rule from the database overview page, click Set server firewall on the
toolbar as shown in the following image: The Firewall settings page for the SQL Database server
opens.

2. Click Add client IP on the toolbar to add the IP address of the computer you are currently using and
then click Save. A server-level firewall rule is created for your current IP address.
From server overview page
The overview page for your server opens, showing you the fully qualified server name (such as
mynewserver20170403.database.windows.net) and provides options for further configuration.
1. To set a server-level rule from server overview page, click Firewall in the left-hand menu under Settings
as showed in the following image:

sys.firewall_rules Server Displays the current server-level

firewall rules

sp_set_firewall_rule Server Creates or updates server-level firewall

rules

sp_delete_firewall_rule Server Removes server-level firewall rules

sys.database_firewall_rules Database Displays the current database-level

firewall rules

sp_set_database_firewall_rule Database Creates or updates the database-level

firewall rules

sp_delete_database_firewall_rule Databases Removes database-level firewall rules

The following examples review the existing rules, enable a range of IP addresses on the server Contoso, and
deletes a firewall rule:

SELECT * FROM sys.firewall_rules ORDER BY name;

Next, add a firewall rule.

EXECUTE sp_set_firewall_rule @name = N'ContosoFirewallRule',

@start_ip_address = '192.168.1.1', @end_ip_address = '192.168.1.10'

To delete a server-level firewall rule, execute the sp_delete_firewall_rule stored procedure. The following
example deletes the rule named ContosoFirewallRule:
EXECUTE sp_delete_firewall_rule @name = N'ContosoFirewallRule'

Azure PowerShell
CMDLET LEVEL DESCRIPTION

Get- Server Returns the current server-level

AzureSqlDatabaseServerFirewallRule firewall rules

New- Server Creates a new server-level firewall rule

AzureSqlDatabaseServerFirewallRule

Set- Server Updates the properties of an existing

AzureSqlDatabaseServerFirewallRule server-level firewall rule

Remove- Server Removes server-level firewall rules

AzureSqlDatabaseServerFirewallRule

The following example sets a server-level firewall rule using PowerShell:

New-AzureRmSqlServerFirewallRule -ResourceGroupName "myResourceGroup" `

-ServerName $servername `
-FirewallRuleName "AllowSome" -StartIpAddress "0.0.0.0" -EndIpAddress "0.0.0.1"

TIP
For PowerShell examples in the context of a quick start, see Create DB - PowerShell and Create a single database and
configure a firewall rule using PowerShell

Azure CLI
CMDLET LEVEL DESCRIPTION

az sql server firewall create Creates a firewall rule to allow access

to all SQL Databases on the server
from the entered IP address range.

az sql server firewall delete Deletes a firewall rule.

az sql server firewall list Lists the firewall rules.

az sql server firewall rule show Shows the details of a firewall rule.

ax sql server firewall rule update Updates a firewall rule.

The following example sets a server-level firewall rule using the Azure CLI:

az sql server firewall-rule create --resource-group myResourceGroup --server $servername \

-n AllowYourIp --start-ip-address 0.0.0.0 --end-ip-address 0.0.0.1
TIP
For an Azure CLI example in the context of a quick start, see Create DDB - Azure CLI and Create a single database and
configure a firewall rule using the Azure CLI

REST API
API LEVEL DESCRIPTION

List Firewall Rules Server Displays the current server-level

firewall rules

Create Firewall Rule Server Creates or updates server-level firewall

rules

Set Firewall Rule Server Updates the properties of an existing

server-level firewall rule

Delete Firewall Rule Server Removes server-level firewall rules

Server-level firewall rule versus a database-level firewall rule

Q. Should users of one database be fully isolated from another database?
If yes, grant access using database-level firewall rules. This avoids using server-level firewall rules, which
permit access through the firewall to all databases, reducing the depth of your defenses.
Q. Do users at the IP address’s need access to all databases?
Use server-level firewall rules to reduce the number of times you must configure firewall rules.
Q. Does the person or team configuring the firewall rules only have access through the Azure portal,
PowerShell, or the REST API?
You must use server-level firewall rules. Database-level firewall rules can only be configured using Transact-
SQL.
Q. Is the person or team configuring the firewall rules prohibited from having high-level permission at the
database level?
Use server-level firewall rules. Configuring database-level firewall rules using Transact-SQL, requires at least
CONTROL DATABASE permission at the database level.

Q. Is the person or team configuring or auditing the firewall rules, centrally managing firewall rules for many
(perhaps 100s) of databases?
This selection depends upon your needs and environment. Server-level firewall rules might be easier to
configure, but scripting can configure rules at the database-level. And even if you use server-level firewall rules,
you might need to audit the database-firewall rules, to see if users with CONTROL permission on the database
have created database-level firewall rules.
Q. Can I use a mix of both server-level and database-level firewall rules?
Yes. Some users, such as administrators might need server-level firewall rules. Other users, such as users of a
database application, might need database-level firewall rules.

Troubleshooting the database firewall

Consider the following points when access to the Microsoft Azure SQL Database service does not behave as
you expect:
Local firewall configuration: Before your computer can access Azure SQL Database, you may need to
create a firewall exception on your computer for TCP port 1433. If you are making connections inside the
Azure cloud boundary, you may have to open additional ports. For more information, see the SQL
Database: Outside vs inside section of Ports beyond 1433 for ADO.NET 4.5 and SQL Database.
Network address translation (NAT): Due to NAT, the IP address used by your computer to connect to
Azure SQL Database may be different than the IP address shown in your computer IP configuration settings.
To view the IP address your computer is using to connect to Azure, log in to the portal and navigate to the
Configure tab on the server that hosts your database. Under the Allowed IP Addresses section, the
Current Client IP Address is displayed. Click Add to the Allowed IP Addresses to allow this computer to
access the server.
Changes to the allow list have not taken effect yet: There may be as much as a five-minute delay for
changes to the Azure SQL Database firewall configuration to take effect.
The login is not authorized or an incorrect password was used: If a login does not have permissions
on the Azure SQL Database server or the password used is incorrect, the connection to the Azure SQL
Database server is denied. Creating a firewall setting only provides clients with an opportunity to attempt
connecting to your server; each client must provide the necessary security credentials. For more information
about preparing logins, see Managing Databases, Logins, and Users in Azure SQL Database.
Dynamic IP address: If you have an Internet connection with dynamic IP addressing and you are
having trouble getting through the firewall, you could try one of the following solutions:
Ask your Internet Service Provider (ISP) for the IP address range assigned to your client computers
that access the Azure SQL Database server, and then add the IP address range as a firewall rule.
Get static IP addressing instead for your client computers, and then add the IP addresses as firewall
rules.

Next steps
For a quick start on creating a database and a server-level firewall rule, see Create an Azure SQL database.
For help in connecting to an Azure SQL database from open source or third-party applications, see Client
quick-start code samples to SQL Database.
For information on additional ports that you may need to open, see the SQL Database: Outside vs inside
section of Ports beyond 1433 for ADO.NET 4.5 and SQL Database
For an overview of Azure SQL Database security, see Securing your database
Use Azure Active Directory Authentication for
authentication with SQL Database or SQL Data
Warehouse
4/14/2017 • 7 min to read • Edit Online

Azure Active Directory authentication is a mechanism of connecting to Microsoft Azure SQL Database and SQL
Data Warehouse by using identities in Azure Active Directory (Azure AD). With Azure AD authentication, you can
centrally manage the identities of database users and other Microsoft services in one central location. Central ID
management provides a single place to manage database users and simplifies permission management. Benefits
include the following:
It provides an alternative to SQL Server authentication.
Helps stop the proliferation of user identities across database servers.
Allows password rotation in a single place
Customers can manage database permissions using external (AAD) groups.
It can eliminate storing passwords by enabling integrated Windows authentication and other forms of
authentication supported by Azure Active Directory.
Azure AD authentication uses contained database users to authenticate identities at the database level.
Azure AD supports token-based authentication for applications connecting to SQL Database.
Azure AD authentication supports ADFS (domain federation) or native user/password authentication for a
local Azure Active Directory without domain synchronization.
Azure AD supports connections from SQL Server Management Studio that use Active Directory Universal
Authentication, which includes Multi-Factor Authentication (MFA). MFA includes strong authentication with a
range of easy verification options — phone call, text message, smart cards with pin, or mobile app
notification. For more information, see SSMS support for Azure AD MFA with SQL Database and SQL Data
Warehouse.
The configuration steps include the following procedures to configure and use Azure Active Directory
authentication.
1. Create and populate Azure AD.
2. Optional: Associate or change the active directory that is currently associated with your Azure Subscription.
3. Create an Azure Active Directory administrator for Azure SQL server or Azure SQL Data Warehouse.
4. Configure your client computers.
5. Create contained database users in your database mapped to Azure AD identities.
6. Connect to your database by using Azure AD identities.

NOTE
To learn how to create and populate Azure AD, and then configure Azure AD with Azure SQL Database and SQL Data
Warehouse, see Configure Azure AD with Azure SQL Database.

Trust architecture
The following high-level diagram summarizes the solution architecture of using Azure AD authentication with
Azure SQL Database. The same concepts apply to SQL Data Warehouse. To support Azure AD native user
password, only the Cloud portion and Azure AD/Azure SQL Database is considered. To support Federated
authentication (or user/password for Windows credentials), the communication with ADFS block is required. The
arrows indicate communication pathways.

The following diagram indicates the federation, trust, and hosting relationships that allow a client to connect to a
database by submitting a token. The token is authenticated by an Azure AD, and is trusted by the database.
Customer 1 can represent an Azure Active Directory with native users or an Azure AD with federated users.
Customer 2 represents a possible solution including imported users; in this example coming from a federated
Azure Active Directory with ADFS being synchronized with Azure Active Directory. It's important to understand
that access to a database using Azure AD authentication requires that the hosting subscription is associated to
the Azure AD. The same subscription must be used to create the SQL Server hosting the Azure SQL Database or
SQL Data Warehouse.

Administrator structure
When using Azure AD authentication, there are two Administrator accounts for the SQL Database server; the
original SQL Server administrator and the Azure AD administrator. The same concepts apply to SQL Data
Warehouse. Only the administrator based on an Azure AD account can create the first Azure AD contained
database user in a user database. The Azure AD administrator login can be an Azure AD user or an Azure AD
group. When the administrator is a group account, it can be used by any group member, enabling multiple Azure
AD administrators for the SQL Server instance. Using group account as an administrator enhances manageability
by allowing you to centrally add and remove group members in Azure AD without changing the users or
permissions in SQL Database. Only one Azure AD administrator (a user or group) can be configured at any time.

Permissions
To create new users, you must have the ALTER ANY USER permission in the database. The ALTER ANY USER
permission can be granted to any database user. The ALTER ANY USER permission is also held by the server
administrator accounts, and database users with the CONTROL ON DATABASE or ALTER ON DATABASE permission
for that database, and by members of the db_owner database role.
To create a contained database user in Azure SQL Database or SQL Data Warehouse, you must connect to the
database using an Azure AD identity. To create the first contained database user, you must connect to the
database by using an Azure AD administrator (who is the owner of the database). This is demonstrated in steps 4
and 5 below. Any Azure AD authentication is only possible if the Azure AD admin was created for Azure SQL
Database or SQL Data Warehouse server. If the Azure Active Directory admin was removed from the server,
existing Azure Active Directory users created previously inside SQL Server can no longer connect to the database
using their Azure Active Directory credentials.

Azure AD features and limitations

The following members of Azure AD can be provisioned in Azure SQL server or SQL Data Warehouse:
Native members: A member created in Azure AD in the managed domain or in a customer domain. For more
information, see Add your own domain name to Azure AD.
Federated domain members: A member created in Azure AD with a federated domain. For more information,
see Microsoft Azure now supports federation with Windows Server Active Directory.
Imported members from other Azure AD's who are native or federated domain members.
Active Directory groups created as security groups.
Microsoft accounts (for example outlook.com, hotmail.com, live.com) or other guest accounts (for example
gmail.com, yahoo.com) are not supported. If you can log in to https://login.live.com using the account and
password, then you are using a Microsoft account, which is not supported for Azure AD authentication for Azure
SQL Database or Azure SQL Data Warehouse.

Connecting using Azure AD identities

Azure Active Directory authentication supports the following methods of connecting to a database using Azure
AD identities:
Using integrated Windows authentication
Using an Azure AD principal name and a password
Using Application token authentication
Additional considerations
To enhance manageability, we recommended you provision a dedicated Azure AD group as an administrator.
Only one Azure AD administrator (a user or group) can be configured for an Azure SQL server or Azure SQL
Data Warehouse at any time.
Only an Azure AD administrator for SQL Server can initially connect to the Azure SQL server or Azure SQL
Data Warehouse using an Azure Active Directory account. The Active Directory administrator can configure
subsequent Azure AD database users.
We recommend setting the connection timeout to 30 seconds.
SQL Server 2016 Management Studio and SQL Server Data Tools for Visual Studio 2015 (version
14.0.60311.1April 2016 or later) support Azure Active Directory authentication. (Azure AD authentication is
supported by the .NET Framework Data Provider for SqlServer; at least version .NET Framework 4.6).
Therefore the newest versions of these tools and data-tier applications (DAC and .bacpac) can use Azure AD
authentication.
ODBC version 13.1 supports Azure Active Directory authentication however bcp.exe cannot connect using
Azure Active Directory authentication because it uses an older ODBC provider.
sqlcmd supports Azure Active Directory authentication beginning with version 13.1 available from the
Download Center.
SQL Server Data Tools for Visual Studio 2015 requires at least the April 2016 version of the Data Tools
(version 14.0.60311.1). Currently Azure AD users are not shown in SSDT Object Explorer. As a workaround,
view the users in sys.database_principals.
Microsoft JDBC Driver 6.0 for SQL Server supports Azure AD authentication. Also, see Setting the Connection
Properties.
PolyBase cannot authenticate by using Azure AD authentication.
Azure AD authentication is supported for SQL Database by the Azure portal Import Database and Export
Database blades. Import and export using Azure AD authentication is also supported from the PowerShell
command.

Next steps
To learn how to create and populate Azure AD, and then configure Azure AD with Azure SQL Database, see
Configure Azure AD with Azure SQL Database.
For an overview of access and control in SQL Database, see SQL Database access and control.
For an overview of logins, users, and database roles in SQL Database, see Logins, users, and database roles.
For more information about database principals, see Principals.
For more information about database roles, see Database roles.
For more information about firewall rules in SQL Database, see SQL Database firewall rules.
For a tutorial using SQL Server authentication, see SQL authentication and authorization.
For a tutorial using Azure Active Directory authentication, see Azure AD authentication and authorization.
Controlling and granting database access
4/14/2017 • 11 min to read • Edit Online

When firewall rules have been configured, people can connect to a SQL Database as one of the administrator
accounts, as the database owner, or as a database user in the database.

NOTE
This topic applies to Azure SQL server, and to both SQL Database and SQL Data Warehouse databases that are created
on the Azure SQL server. For simplicity, SQL Database is used when referring to both SQL Database and SQL Data
Warehouse.

TIP
For a tutorial using SQL Server authentication, see Tutorial: SQL Server authentication. For a tutorial using Azure Active
Directory authentication, see Tutorial: AAD authentication.

Unrestricted administrative accounts

There are two administrative accounts (Server admin and Active Directory admin) that act as
administrators. To identify these administrator accounts for your SQL server, open the Azure portal, and
navigate to the properties of your SQL server.

Server admin
When you create an Azure SQL server, you must designate a Server admin login. SQL server creates that
account as a login in the master database. This account connects using SQL Server authentication (user
name and password). Only one of these accounts can exist.
Azure Active Directory admin
One Azure Active Directory account, either an individual or security group account, can also be configured
as an administrator. It is optional to configure an Azure AD administrator, but an Azure AD administrator
must be configured if you want to use Azure AD accounts to connect to SQL Database. For more
information about configuring Azure Active Directory access, see Connecting to SQL Database or SQL Data
Warehouse By Using Azure Active Directory Authentication and SSMS support for Azure AD MFA with SQL
Database and SQL Data Warehouse.
The Server admin and Azure AD admin accounts has the following characteristics:
These are the only accounts that can automatically connect to any SQL Database on the server. (To connect
to a user database, other accounts must either be the owner of the database, or have a user account in the
user database.)
These accounts enter user databases as the dbo user and they have all the permissions in the user
databases. (The owner of a user database also enters the database as the dbo user.)
These accounts do not enter the master database as the dbo user and they have limited permissions in
master.
These accounts are not members of the standard SQL Server sysadmin fixed server role, which is not
available in SQL database.
These accounts can create, alter, and drop databases, logins, users in master, and server-level firewall rules.
These accounts can add and remove members to the dbmanager and loginmanager roles.
These accounts can view the sys.sql_logins system table.
Configuring the firewall
When the server-level firewall is configured for an individual IP address or range, the SQL server admin and
the Azure Active Directory admin can connect to the master database and all the user databases. The initial
server-level firewall can be configured through the Azure portal, using PowerShell or using the REST API. Once
a connection is made, additional server-level firewall rules can also be configured by using Transact-SQL.
Administrator access path
When the server-level firewall is properly configured, the SQL server admin and the Azure Active Directory
admin can connect using client tools such as SQL Server Management Studio or SQL Server Data Tools. Only
the latest tools provide all the features and capabilities. The following diagram shows a typical configuration
for the two administrator accounts.

When using an open port in the server-level firewall, administrators can connect to any SQL Database.
Connecting to a database by using SQL Server Management Studio
For a walk-through of creating a server, a database, server-level firewall rules, and using SQL Server
Management Studio to query a database, see Get started with Azure SQL Database servers, databases, and
firewall rules by using the Azure portal and SQL Server Management Studio.
IMPORTANT
It is recommended that you always use the latest version of Management Studio to remain synchronized with updates
to Microsoft Azure and SQL Database. Update SQL Server Management Studio.

Additional server-level administrative roles

In addition to the server-level administrative roles discussed previously, SQL Database provides two restricted
administrative roles in the master database to which user accounts can be added that grant permissions to
either create databases or manage logins.
Database creators
One of these administrative roles is the dbmanager role. Members of this role can create new databases. To
use this role, you create a user in the master database and then add the user to the dbmanager database role.
To create a database, the user must be a user based on a SQL Server login in the master database or contained
database user based on an Azure Active Directory user.
1. Using an administrator account, connect to the master database.
2. Optional step: Create a SQL Server authentication login, using the CREATE LOGIN statement. Sample
statement:

CREATE LOGIN Mary WITH PASSWORD = '<strong_password>';

NOTE
Use a strong password when creating a login or contained database user. For more information, see Strong
Passwords.

To improve performance, logins (server-level principals) are temporarily cached at the database level.
To refresh the authentication cache, see DBCC FLUSHAUTHCACHE.
3. In the master database, create a user by using the CREATE USER statement. The user can be an Azure
Active Directory authentication contained database user (if you have configured your environment for
Azure AD authentication), or a SQL Server authentication contained database user, or a SQL Server
authentication user based on a SQL Server authentication login (created in the previous step.) Sample
statements:

CREATE USER [mike@contoso.com] FROM EXTERNAL PROVIDER;

CREATE USER Tran WITH PASSWORD = '<strong_password>';
CREATE USER Mary FROM LOGIN Mary;

4. Add the new user, to the dbmanager database role by using the ALTER ROLE statement. Sample
statements:

ALTER ROLE dbmanager ADD MEMBER Mary;

ALTER ROLE dbmanager ADD MEMBER [mike@contoso.com];
NOTE
The dbmanager is a database role in master database so you can only add a database user to the dbmanager
role. You cannot add a server-level login to database-level role.

5. If necessary, configure a firewall rule to allow the new user to connect. (The new user might be covered
by an existing firewall rule.)
Now the user can connect to the master database and can create new databases. The account creating the
database becomes the owner of the database.
Login managers
The other administrative role is the login manager role. Members of this role can create new logins in the
master database. If you wish, you can complete the same steps (create a login and user, and add a user to the
loginmanager role) to enable a user to create new logins in the master. Usually logins are not necessary as
Microsoft recommends using contained database users, which authenticate at the database-level instead of
using users based on logins. For more information, see Contained Database Users - Making Your Database
Portable.

Non-administrator users
Generally, non-administrator accounts do not need access to the master database. Create contained database
users at the database level using the CREATE USER (Transact-SQL) statement. The user can be an Azure Active
Directory authentication contained database user (if you have configured your environment for Azure AD
authentication), or a SQL Server authentication contained database user, or a SQL Server authentication user
based on a SQL Server authentication login (created in the previous step.) For more information, see
Contained Database Users - Making Your Database Portable.
To create users, connect to the database, and execute statements similar to the following examples:

CREATE USER Mary FROM LOGIN Mary;

CREATE USER [mike@contoso.com] FROM EXTERNAL PROVIDER;

Initially, only one of the administrators or the owner of the database can create users. To authorize additional
users to create new users, grant that selected user the ALTER ANY USER permission, by using a statement such
as:

GRANT ALTER ANY USER TO Mary;

To give additional users full control of the database, make them a member of the db_owner fixed database
role using the ALTER ROLE statement.

NOTE
The most common reason to create database users based on logins, is when you have SQL Server authentication users
that need access to multiple databases. Users based on logins are tied to the login, and only one password that is
maintained for that login. Contained database users in individual databases are each individual entities and each
maintains its own password. This can confuse contained database users if they do not maintain their passwords as
identical.

Configuring the database -level firewall

As a best practice, non-administrator users should only have access through the firewall to the databases that
they use. Instead of authorizing their IP addresses through the server-level firewall and giving them access to
all databases, use the sp_set_database_firewall_rule statement to configure the database-level firewall. The
database-level firewall cannot be configured by using the portal.
Non-administrator access path
When the database-level firewall is properly configured, the database users can connect using client tools such
as SQL Server Management Studio or SQL Server Data Tools. Only the latest tools provide all the features and
capabilities. The following diagram shows a typical non-administrator access path.

Groups and roles

Efficient access management uses permissions assigned to groups and roles instead of individual users.
When using Azure Active Directory authentication, put Azure Active Directory users into an Azure Active
Directory group. Create a contained database user for the group. Place one or more database users into
a database role and then assign permissions to the database role.
When using SQL Server authentication, create contained database users in the database. Place one or
more database users into a database role and then assign permissions to the database role.
The database roles can be the built-in roles such as db_owner, db_ddladmin, db_datawriter,
db_datareader, db_denydatawriter, and db_denydatareader. db_owner is commonly used to grant full
permission to only a few users. The other fixed database roles are useful for getting a simple database in
development quickly, but are not recommended for most production databases. For example, the
db_datareader fixed database role grants read access to every table in the database, which is usually more
than is strictly necessary. It is far better to use the CREATE ROLE statement to create your own user-defined
database roles and carefully grant each role the least permissions necessary for the business need. When a
user is a member of multiple roles, they aggregate the permissions of them all.

Permissions
There are over 100 permissions that can be individually granted or denied in SQL Database. Many of these
permissions are nested. For example, the UPDATE permission on a schema includes the UPDATE permission
on each table within that schema. As in most permission systems, the denial of a permission overrides a grant.
Because of the nested nature and the number of permissions, it can take careful study to design an
appropriate permission system to properly protect your database. Start with the list of permissions at
Permissions (Database Engine) and review the poster size graphic of the permissions.
Considerations and restrictions
When managing logins and users in SQL Database, consider the following:
You must be connected to the master database when executing the CREATE/ALTER/DROP DATABASE
statements.
The database user corresponding to the Server admin login cannot be altered or dropped.
US-English is the default language of the Server admin login.
Only the administrators (Server admin login or Azure AD administrator) and the members of the
dbmanager database role in the master database have permission to execute the CREATE DATABASE and
DROP DATABASE statements.
You must be connected to the master database when executing the CREATE/ALTER/DROP LOGIN statements.
However using logins is discouraged. Use contained database users instead.
To connect to a user database, you must provide the name of the database in the connection string.
Only the server-level principal login and the members of the loginmanager database role in the master
database have permission to execute the CREATE LOGIN , ALTER LOGIN , and DROP LOGIN statements.
When executing the CREATE/ALTER/DROP LOGIN and CREATE/ALTER/DROP DATABASE statements in an
ADO.NET application, using parameterized commands is not allowed. For more information, see
Commands and Parameters.
When executing the CREATE/ALTER/DROP DATABASE and CREATE/ALTER/DROP LOGIN statements, each of
these statements must be the only statement in a Transact-SQL batch. Otherwise, an error occurs. For
example, the following Transact-SQL checks whether the database exists. If it exists, a DROP DATABASE
statement is called to remove the database. Because the DROP DATABASE statement is not the only
statement in the batch, executing the following Transact-SQL statement results in an error.

IF EXISTS (SELECT [name]

FROM [sys].[databases]
WHERE [name] = N'database_name')
DROP DATABASE [database_name];
GO

When executing the CREATE USER statement with the FOR/FROM LOGIN option, it must be the only
statement in a Transact-SQL batch.
When executing the ALTER USER statement with the WITH LOGIN option, it must be the only statement in a
Transact-SQL batch.
To CREATE/ALTER/DROP a user requires the ALTER ANY USER permission on the database.
When the owner of a database role tries to add or remove another database user to or from that database
role, the following error may occur: User or role 'Name' does not exist in this database. This error
occurs because the user is not visible to the owner. To resolve this issue, grant the role owner the
VIEW DEFINITION permission on the user.

Next steps
To learn more about firewall rules, see Azure SQL Database Firewall.
For an overview of all the SQL Database security features, see SQL security overview.
For a tutorial, see Get started with SQL security
For information about views and stored procedures, see Creating views and stored procedures
For information about granting access to a database object, see Granting Access to a Database Object
For a tutorial using SQL Server authentication, see Tutorial: SQL Server authentication.
For a tutorial using Azure Active Directory authentication, see Tutorial: AAD authentication.
Universal Authentication with SQL Database and SQL
Data Warehouse (SSMS support for MFA)
4/14/2017 • 2 min to read • Edit Online

Azure SQL Database and Azure SQL Data Warehouse support connections from SQL Server Management Studio
(SSMS) using Active Directory Universal Authentication.
Active Directory Universal Authentication supports the two non-interactive authentication methods (Active
Directory Password Authentication and Active Directory Integrated Authentication). Non-interactive Active
Directory Password and Active Directory Integrated Authentication methods can be used in many different
applications (ADO.NET, JDBC, ODBC, etc.). These two methods never result in pop-up dialog boxes.
Active Directory Universal Authentication is an interactive method that also supports Azure Multi-Factor
Authentication (MFA). Azure MFA helps safeguard access to data and applications while meeting user
demand for a simple sign-in process. It delivers strong authentication with a range of easy verification
options—phone call, text message, smart cards with pin, or mobile app notification—allowing users to
choose the method they prefer. Interactive MFA with Azure AD can result in a pop-up dialog box for
validation.
For a description of Multi-Factor Authentication, see Multi-Factor Authentication. For configuration steps, see
Configure Azure SQL Database multi-factor authentication for SQL Server Management Studio.
Azure AD domain name or tenant ID parameter
Beginning with SSMS version 17, users that are imported into the current Active Directory from other Azure Active
Directories, can provide the Azure AD domain name, or tenant ID when they connect. This allows Active Directory
Universal Authentication to identify the correct authenticating authority. This option is also required to support
Microsoft accounts (MSA) such as outlook.com, hotmail.com or live.com. All these users who want to be
authenticated using Universal Authentication must enter their Azure AD domain name or tenant ID. This parameter
represents the current Azure AD domain name/tenant ID the Azure Server is linked with. For example, if Azure
Server is associated with Azure AD domain contosotest.onmicrosoft.com where user joe@contosodev.onmicrosoft.com is
hosted as an imported user from Azure AD domain contosodev.onmicrosoft.com , the domain name required to
authenticate this user is contosotest.onmicrosoft.com . When the user is a native user of the Azure AD linked to Azure
Server, and is not an MSA account, no domain name or tenant ID is required. To enter the parameter (beginning
with SSMS version 17), in the Connect to Database dialog box, complete the dialog box, selecting Active
Directory Universal Authentication, click Options, click the Connection Properties tab, check the AD
domain name or tenant ID box, and provide authenticating authority, such as the domain name
(contosotest.onmicrosoft.com) or the guid of the tenant ID.

Universal Authentication limitations for SQL Database and SQL Data

Warehouse
SSMS is the only tool currently enabled for MFA through Active Directory Universal Authentication.
Only a single Azure Active Directory account can log in for an instance of SSMS using Universal Authentication.
To log in as another Azure AD account, you must use another instance of SSMS. (This restriction is limited to
Active Directory Universal Authentication; you can log in to different servers using Active Directory Password
Authentication, Active Directory Integrated Authentication, or SQL Server Authentication).
SSMS supports Active Directory Universal Authentication for Object Explorer, Query Editor, and Query Store
visualization.
Neither DacFx nor the Schema Designer support Universal Authentication.
There are no additional software requirements for Active Directory Universal Authentication except that you
must use a supported version of SSMS.

Next steps
For configuration steps, see Configure Azure SQL Database multi-factor authentication for SQL Server
Management Studio.
Grant others access to your database: SQL Database Authentication and Authorization: Granting Access
Make sure others can connect through the firewall: Configure an Azure SQL Database server-level firewall rule
using the Azure portal
Get started with SQL database auditing
4/14/2017 • 11 min to read • Edit Online

Azure SQL Database Auditing tracks database events and writes them to an audit log in your Azure Storage
account.
Auditing can help you maintain regulatory compliance, understand database activity, and gain insight into
discrepancies and anomalies that could indicate business concerns or suspected security violations.
Auditing enables and facilitates adherence to compliance standards but doesn't guarantee compliance. For more
information about Azure programs that support standards compliance, see the Azure Trust Center.
Azure SQL Database Auditing overview
Set up auditing for your database
Analyze audit logs and reports

Azure SQL Database auditing overview

SQL Database Auditing allows you to:
Retain an audit trail of selected events. You can define categories of database actions to be audited.
Report on database activity. You can use preconfigured reports and a dashboard to get started quickly with
activity and event reporting.
Analyze reports. You can find suspicious events, unusual activity, and trends.
There are two Auditing methods:
Blob Auditing - logs are written to Azure Blob Storage. This is a newer auditing method, which provides
higher performance, supports higher granularity object-level auditing, and is more cost effective. Blob
Auditing will be replacing Table Auditing.
Table Auditing (deprecated) - logs are written to Azure Table Storage.

IMPORTANT
The introduction of the new Blob Auditing brings a major change to the way server auditing policy is being inherited by the
database. See Blob/Table differences in server auditing policy inheritance section for additional details.

You can configure auditing for different types of event categories, as explained in the Set up auditing for your
database section.
An auditing policy can be defined for a specific database or as a default server policy. A default server auditing
policy applies to all existing and newly created databases on a server.

Set up auditing for your database

The following section describes the configuration of auditing using the Azure portal.
1. Launch the Azure portal at https://portal.azure.com.
Bl ob Audi ti ng

2. Navigate to the settings blade of the SQL Database / SQL Server you want to audit. In the Settings blade,
select Auditing & Threat detection.
3. In the database auditing configuration blade, you can check the Inherit settings from server checkbox to
designate that this database will be audited according to its server's settings. If this option is checked, you
will see a View server auditing settings link that allows you to view or modify the server auditing
settings from this context.

4. If you prefer to enable Blob Auditing on the database-level (in addition or instead of server-level auditing),
uncheck the Inherit Auditing settings from server option, turn ON Auditing, and choose the Blob
Auditing Type.

If server Blob auditing is enabled, the database configured audit will exist side by side with the server
Blob audit.

6. If you want to customize the audited events, you can do this via PowerShell or REST API - see the Automation
(PowerShell / REST API) section for more details.
7. Once you've configured your auditing settings, you can turn on the new Threat Detection (preview) feature,
and configure the emails to receive security alerts. Threat Detection allows you to receive proactive alerts on
anomalous database activities that may indicate potential security threats. See Getting Started with Threat
Detection for more details.
8. Click Save.
(deprecated)
Tabl e Audi ti ng

Before setting up Table Auditing, check if you are using a "Downlevel Client". Also, if you have strict firewall
settings, please note that the IP endpoint of your database will change when enabling Table Auditing.

1. Launch the Azure portal at https://portal.azure.com.

2. Navigate to the settings blade of the SQL Database / SQL Server you want to audit. In the Settings blade, select
Auditing & Threat detection (see screenshot in Blob Auditing section).
3. In the database auditing configuration blade, you can check the Inherit settings from server checkbox to
designate that this database will be audited according to its server's settings. If this option is checked, you
will see a View server auditing settings link that allows you to view or modify the server auditing
settings from this context.

4. If you prefer not to inherit Auditing settings from server, uncheck the Inherit Auditing settings from
server option, turn ON Auditing, and choose Table Auditing Type.

5. Select Storage Details to open the Audit Logs Storage Blade. Select the Azure storage account where logs will
be saved, and the retention period, after which the old logs will be deleted. Tip: Use the same storage account
for all audited databases to get the most out of the auditing reports templates (see screenshot in Blob Auditing
section).
6. Click on Audited Events to customize which events to audit. In the Logging by event blade, click
Success and Failure to log all events, or choose individual event categories.

Customizing audited events can also be done via PowerShell or REST API - see the Automation
(PowerShell / REST API) section for more details.

7. Once you've configured your auditing settings, you can turn on the new Threat Detection (preview) feature,
and configure the emails to receive security alerts. Threat Detection allows you to receive proactive alerts on
anomalous database activities that may indicate potential security threats. See Getting Started with Threat
Detection for more details.
8. Click Save.
Blob/table differences in server auditing policy inheritance
1. If Server Blob auditing is enabled, it always applies to the database (i.e. the database will be audited),
Bl ob audi ti ng

regardless of:
The database auditing settings.
Whether or not the "Inherit settings from server" checkbox is checked in the database blade.
2. Enabling Blob auditing on the database, in addition to enabling it on the server, will not override or change
any of the settings of the server Blob auditing - both audits will exist side by side. In other words, the
database will be audited twice in parallel (once by the Server policy and once by the Database policy).

NOTE
You should avoid enabling both server Blob auditing and database Blob auditing together, unless:
You need to use a different storage account or retention period for a specific database.
You want to audit different event types or categories for a specific database than are being audited for the rest
of the databases on this server (e.g. if table inserts need to be audited only for a specific database).

Otherwise, it is recommended to only enable server-level Blob Auditing and leave the database-level
auditing disabled for all databases.

(deprecated)
Tabl e audi ti ng

If server-level Table auditing is enabled, it only applies to the database if the "Inherit settings from server"
checkbox is checked in the database blade (this is checked by default for all existing and newly created databases).
If the checkbox is checked, any changes made to the auditing settings in database override the server
settings for this database.
If the checkbox is unchecked and the database-level auditing is disabled, the database will not be audited.

Analyze audit logs and reports

Audit logs are aggregated in the Azure storage account you chose during setup.
You can explore audit logs using a tool such as Azure Storage Explorer.
See below specifics for analysis of both Blob and Table audit logs.
Blob Auditing logs are saved as a collection of blob files within a container named "sqldbauditlogs".
Bl ob Audi ti ng

For further details about the Blob audit logs storage folder hierarchy, blob naming convention, and log format,
see the Blob Audit Log Format Reference (doc file download).
There are several methods to view Blob Auditing logs:
1. Through the Azure portal - open the relevant database. At the top of the database's Auditing & Threat
detection blade, click on View audit logs.

An Audit records blade will open, where you'll be able to view the logs.
You can choose to view specific dates by clicking on Filter at the top area of the Audit records blade
You can toggle between audit records that were created by server policy or database policy audit

2. Download log files from your Azure Storage Blob container via the portal or by using a tool such as Azure
Storage Explorer.
Once you have downloaded the log file locally, you can double-click the file to open, view and analyze the
logs in SSMS.
Additional methods:
You can download multiple files simultaneously via Azure Storage Explorer - right-click on a
specific subfolder (e.g. a subfolder that includes all log files for a specific date) and choose "Save as"
to save in a local folder.
After downloading several files (or an entire day, as described above), you can merge them locally
as follows:
Open SSMS -> File -> Open -> Merge Extended Events -> Choose all files to merge
Programmatically:
Extended Events Reader C# library (more info here)
Querying Extended Events Files Using PowerShell (more info here)
3. We have created a sample application that runs in Azure and utilizes OMS public APIs to push SQL audit
logs into OMS for consumption via the OMS dashboard (more info here).
(deprecated)
Tabl e audi ti ng

Table Auditing logs are saved as a collection of Azure Storage Tables with a SQLDBAuditLogs prefix.
For further details about the Table audit log format, see the Table Audit Log Format Reference (doc file download).
There are several methods to view Table Auditing logs:
1. Through the Azure portal - open the relevant database. At the top of the database's Auditing & Threat
detection blade, click on View audit logs.

An Audit records blade will open, where you'll be able to view the logs.
You can choose to view specific dates by clicking on Filter at the top area of the Audit records blade
You can download and view the audit logs in Excel format by clicking on Open in Excel at the top
area of the Audit records blade
2. Alternatively, a preconfigured report template is available as a downloadable Excel spreadsheet to help you
quickly analyze log data. To use the template on your audit logs, you need Excel 2013 or later and Power
Query, which you can download here.
You can also import your audit logs into the Excel template directly from your Azure storage account using
Power Query. You can then explore your audit records and create dashboards and reports on top of the log
data.

Practices for usage in production

When using Geo-replicated databases, it is possible to set up Auditing on either the Primary database, the
Audi ti ng geo-repl i cated databases

Secondary database, or both, depending on the Audit type.

Table Auditing - you can configure a separate policy, on either the database or the server level, for each of the
two databases (Primary and Secondary) as described in Set up auditing for your database section.
Blob auditing - follow these instructions:
1. Primary database - turn on Blob Auditing either on the server or the database itself, as described in Set up
auditing for your database section.
2. Secondary database - Blob Auditing can only be turned on/off from the Primary database auditing
settings.
Turn on Blob Auditing on the Primary database, as described in Set up auditing for your database
section. Blob Auditing must be enabled on the primary database itself, not the server.
Once Blob Auditing is enabled on the Primary database, it will also become enabled on the
Secondary database.
IMPORTANT
By default, the storage settings for the Secondary database will be identical to those of the Primary
database, causing cross-regional traffic. You can avoid this by enabling Blob Auditing on the Secondary
server and configuring a local storage in the Secondary server storage settings (this will override the
storage location for the Secondary database and result in each database saving the Audit logs to a local
storage).

In production, you are likely to refresh your storage keys periodically. When refreshing your keys, you need to re-
Storage key regenerati on

save the auditing policy. The process is as follows:

1. In the storage details blade switch the Storage Access Key from Primary to Secondary, and then click OK
at the bottom. Then click SAVE at the top of the auditing configuration blade.

2. Go to the storage configuration blade and regenerate the Primary Access Key.

3. Go back to the auditing configuration blade, switch the Storage Access Key from Secondary to Primary, and
then click OK at the bottom. Then click SAVE at the top of the auditing configuration blade.
4. Go back to the storage configuration blade and regenerate the Secondary Access Key (in preparation for the
next keys refresh cycle).
Automation (PowerShell / REST API)
You can also configure Auditing in Azure SQL Database using the following automation tools:
1. PowerShell cmdlets
Get-AzureRMSqlDatabaseAuditingPolicy
Get-AzureRMSqlServerAuditingPolicy
Remove-AzureRMSqlDatabaseAuditing
Remove-AzureRMSqlServerAuditing
Set-AzureRMSqlDatabaseAuditingPolicy
Set-AzureRMSqlServerAuditingPolicy
Use-AzureRMSqlServerAuditingPolicy
For an script example, see Configure auditing and threat detectoin using PowerShell.
2. REST API - Blob Auditing
Create or Update Database Blob Auditing Policy
Create or Update Server Blob Auditing Policy
Get Database Blob Auditing Policy
Get Server Blob Auditing Policy
Get Server Blob Auditing Operation Result
3. REST API - Table Auditing (deprecated)
Create or Update Database Auditing Policy
Create or Update Server Auditing Policy
Get Database Auditing Policy
Get Server Auditing Policy
SQL Database - Downlevel clients support and IP
endpoint changes for Auditing
4/14/2017 • 1 min to read • Edit Online

Database Auditing works automatically with SQL clients that support TDS redirection. Note that redirection does
not apply when using the Blob Auditing method.

Downlevel clients support

Any client which implements TDS 7.4 should also support redirection. Exceptions to this include JDBC 4.0 in which
the redirection feature is not fully supported and Tedious for Node.JS in which redirection was not implemented.
For "Downlevel clients", i.e. which support TDS version 7.3 and below - the server FQDN in the connection string
should be modified:
Original server FQDN in the connection string: <server name>.database.windows.net
Modified server FQDN in the connection string: <server name>.database.secure.windows.net
A partial list of "Downlevel clients" includes:
.NET 4.0 and below,
ODBC 10.0 and below.
JDBC (while JDBC does support TDS 7.4, the TDS redirection feature is not fully supported)
Tedious (for Node.JS)
Remark: The above server FDQN modification may be useful also for applying a SQL Server Level Auditing policy
without a need for a configuration step in each database (Temporary mitigation).

IP endpoint changes when enabling Auditing

Please note that when you enable Table Auditing, the IP endpoint of your database will change. If you have strict
firewall settings, please update those firewall settings accordingly.
The new database IP endpoint will depend on the database region:

DATABASE REGION POSSIBLE IP ENDPOINTS

China North 139.217.29.176, 139.217.28.254

China East 42.159.245.65, 42.159.246.245

Australia East 104.210.91.32, 40.126.244.159, 191.239.64.60,

40.126.255.94

Australia Southeast 191.239.184.223, 40.127.85.81, 191.239.161.83,

40.127.81.130

Brazil South 104.41.44.161, 104.41.62.230, 23.97.99.54, 104.41.59.191

Central US 104.43.255.70, 40.83.14.7, 23.99.128.244, 40.83.15.176

DATABASE REGION POSSIBLE IP ENDPOINTS

Central US EUAP 52.180.178.16, 52.180.176.190

East Asia 23.99.125.133, 13.75.40.42, 23.97.71.138, 13.94.43.245

East US 2 104.209.141.31, 104.208.238.177, 191.237.131.51,

104.208.235.50

East US 23.96.107.223, 104.41.150.122, 23.96.38.170, 104.41.146.44

East US EUAP 52.225.190.86, 52.225.191.187

Central India 104.211.98.219, 104.211.103.71

South India 104.211.227.102, 104.211.225.157

West India 104.211.161.152, 104.211.162.21

Japan East 104.41.179.1, 40.115.253.81, 23.102.64.207, 40.115.250.196

Japan West 104.214.140.140, 104.214.146.31, 191.233.32.34,

104.214.146.198

North Central US 191.236.155.178, 23.96.192.130, 23.96.177.169,

23.96.193.231

North Europe 104.41.209.221, 40.85.139.245, 137.116.251.66,

40.85.142.176

South Central US 191.238.184.128, 40.84.190.84, 23.102.160.153,

40.84.186.66

Southeast Asia 104.215.198.156, 13.76.252.200, 23.97.51.109,

13.76.252.113

West Europe 104.40.230.120, 13.80.23.64, 137.117.171.161, 13.80.8.37,

104.47.167.215, 40.118.56.193, 104.40.176.73, 40.118.56.20

West US 191.236.123.146, 138.91.163.240, 168.62.194.148,

23.99.6.91

West US 2 13.66.224.156, 13.66.227.8

West Central US 52.161.29.186, 52.161.27.213

Canada Central 13.88.248.106, 13.88.248.110

Canada East 40.86.227.82, 40.86.225.194

UK North 13.87.101.18, 13.87.100.232

UK South 2 13.87.32.202, 13.87.32.226

SQL Database Threat Detection
4/14/2017 • 2 min to read • Edit Online

Threat Detection detects anomalous database activities indicating potential security threats to the database. Threat
Detection is currently in preview.

Overview
Threat Detection provides a new layer of security, which enables customers to detect and respond to potential
threats as they occur by providing security alerts on anomalous activities. Users can explore the suspicious events
using SQL Database Auditing to determine if they result from an attempt to access, breach or exploit data in the
database. Threat Detection makes it simple to address potential threats to the database without the need to be a
security expert or manage advanced security monitoring systems.
For example, Threat Detection detects certain anomalous database activities indicating potential SQL injection
attempts. SQL injection is one of the common Web application security issues on the Internet, used to attack data-
driven applications. Attackers take advantage of application vulnerabilities to inject malicious SQL statements into
application entry fields, for breaching or modifying data in the database.

Set up threat detection for your database in the Azure portal

1. Launch the Azure portal at https://portal.azure.com.
2. Navigate to the configuration blade of the SQL Database you want to monitor. In the Settings blade, select
Auditing & Threat Detection.
3. In the Auditing & Threat Detection configuration blade turn ON auditing, which will display the threat
detection settings.
4. Turn ON threat detection.
5. Configure the list of emails that will receive security alerts upon detection of anomalous database activities.
6. Click Save in the Auditing & Threat detection blade to save the new or updated auditing and threat
detection policy.
Set up threat detection using PowerShell
For an script example, see Configure auditing and threat detection using PowerShell.

Explore anomalous database activities upon detection of a suspicious

event
1. You will receive an email notification upon detection of anomalous database activities.
The email will provide information on the suspicious security event including the nature of the anomalous
activities, database name, server name and the event time. In addition, it will provide information on
possible causes and recommended actions to investigate and mitigate the potential threat to the database.
2. In the email, click on the Azure SQL Auditing Log link, which will launch the Azure portal and show the
relevant Auditing records around the time of the suspicious event.
3. Click on the audit records to view more details on the suspicious database activities such as SQL statement,
failure reason and client IP.

4. In the Auditing Records blade, click Open in Excel to open a pre-configured excel template to import and
run deeper analysis of the audit log around the time of the suspicious event.
Note: In Excel 2010 or later, Power Query and the Fast Combine setting is required.

5. To configure the Fast Combine setting - In the POWER QUERY ribbon tab, select Options to display the
Options dialog. Select the Privacy section and choose the second option - 'Ignore the Privacy Levels and
potentially improve performance':
6. To load SQL audit logs, ensure that the parameters in the settings tab are set correctly and then select the
'Data' ribbon and click the 'Refresh All' button.

7. The results appear in the SQL Audit Logs sheet which enables you to run deeper analysis of the anomalous
activities that were detected, and mitigate the impact of the security event in your application.

Next steps
For an overview of SQL Database auditing, see Database auditing.
For a PowerShell script example, see Configure auditing and threat detection using PowerShell.
SQL Database dynamic data masking
4/14/2017 • 3 min to read • Edit Online

SQL Database dynamic data masking limits sensitive data exposure by masking it to non-privileged users.
Dynamic data masking helps prevent unauthorized access to sensitive data by enabling customers to designate
how much of the sensitive data to reveal with minimal impact on the application layer. It’s a policy-based security
feature that hides the sensitive data in the result set of a query over designated database fields, while the data in
the database is not changed.
For example, a service representative at a call center may identify callers by several digits of their credit card
number, but those data items should not be fully exposed to the service representative. A masking rule can be
defined that masks all but the last four digits of any credit card number in the result set of any query. As another
example, an appropriate data mask can be defined to protect personally identifiable information (PII) data, so that
a developer can query production environments for troubleshooting purposes without violating compliance
regulations.

SQL Database dynamic data masking basics

You set up a dynamic data masking policy in the Azure portal by selecting the dynamic data masking operation in
your SQL Database configuration blade or settings blade.
Dynamic data masking permissions
Dynamic data masking can be configured by the Azure Database admin, server admin, or security officer roles.
Dynamic data masking policy
SQL users excluded from masking - A set of SQL users or AAD identities that get unmasked data in the SQL
query results. Users with administrator privileges are always excluded from masking, and see the original data
without any mask.
Masking rules - A set of rules that define the designated fields to be masked and the masking function that is
used. The designated fields can be defined using a database schema name, table name, and column name.
Masking functions - A set of methods that control the exposure of data for different scenarios.

MASKING FUNCTION MASKING LOGIC

Default Full masking according to the data types of the

designated fields

• Use XXXX or fewer Xs if the size of the field is less than 4

characters for string data types (nchar, ntext, nvarchar).
• Use a zero value for numeric data types (bigint, bit, decimal,
int, money, numeric, smallint, smallmoney, tinyint, float, real).
• Use 01-01-1900 for date/time data types (date, datetime2,
datetime, datetimeoffset, smalldatetime, time).
• For SQL variant, the default value of the current type is
used.
• For XML the document is used.
• Use an empty value for special data types (timestamp table,
hierarchyid, GUID, binary, image, varbinary spatial types).
MASKING FUNCTION MASKING LOGIC

Credit card Masking method, which exposes the last four digits of
the designated fields and adds a constant string as a prefix
in the form of a credit card.

XXXX-XXXX-XXXX-1234

Email Masking method, which exposes the first letter and

replaces the domain with XXX.com using a constant string
prefix in the form of an email address.

aXX@XXXX.com

Random number Masking method, which generates a random number

according to the selected boundaries and actual data types. If
the designated boundaries are equal, then the masking
function is a constant number.

Custom text Masking method, which exposes the first and last
characters and adds a custom padding string in the middle.
If the original string is shorter than the exposed prefix and
suffix, only the padding string is used.
prefix[padding]suffix

Recommended fields to mask

The DDM recommendations engine, flags certain fields from your database as potentially sensitive fields, which
may be good candidates for masking. In the Dynamic Data Masking blade in the portal, you will see the
recommended columns for your database. All you need to do is click Add Mask for one or more columns and
then Save to apply a mask for these fields.

Set up dynamic data masking for your database using Powershell

cmdlets
See Azure SQL Database Cmdlets.

Set up dynamic data masking for your database using REST API
See Operations for Azure SQL Databases.
Overview of business continuity with Azure SQL
Database
4/13/2017 • 11 min to read • Edit Online

This overview describes the capabilities that Azure SQL Database provides for business continuity and disaster
recovery. It provides options, recommendations, and tutorials for recovering from disruptive events that could
cause data loss or cause your database and application to become unavailable. The discussion includes what to
do when a user or application error affects data integrity, an Azure region has an outage, or your application
requires maintenance.

SQL Database features that you can use to provide business

continuity
SQL Database provides several business continuity features, including automated backups and optional
database replication. Each has different characteristics for estimated recovery time (ERT) and potential data loss
for recent transactions. Once you understand these options, you can choose among them - and, in most
scenarios, use them together for different scenarios. As you develop your business continuity plan, you need to
understand the maximum acceptable time before the application fully recovers after the disruptive event - this
is your recovery time objective (RTO). You also need to understand the maximum amount of recent data
updates (time interval) the application can tolerate losing when recovering after the disruptive event - the
recovery point objective (RPO).
The following table compares the ERT and RPO for the three most common scenarios.

CAPABILITY BASIC TIER STANDARD TIER PREMIUM TIER

Point in Time Restore from Any restore point within 7 Any restore point within 35 Any restore point within 35
backup days days days

Geo-Restore from geo- ERT < 12h, RPO < 1h ERT < 12h, RPO < 1h ERT < 12h, RPO < 1h
replicated backups

Restore from Azure Backup ERT < 12h, RPO < 1 wk ERT < 12h, RPO < 1 wk ERT < 12h, RPO < 1 wk
Vault

Active Geo-Replication ERT < 30s, RPO < 5s ERT < 30s, RPO < 5s ERT < 30s, RPO < 5s

Use database backups to recover a database

SQL Database automatically performs a combination of full database backups weekly, differential database
backups hourly, and transaction log backups every five minutes to protect your business from data loss. These
backups are stored in geo-redundant storage for 35 days for databases in the Standard and Premium service
tiers and seven days for databases in the Basic service tier - see service tiers for more details on service tiers. If
the retention period for your service tier does not meet your business requirements, you can increase the
retention period by changing the service tier. The full and differential database backups are also replicated to a
paired data center for protection against a data center outage. See automatic database backups for more details.
If the built-in retention period is not sufficient for your application, you can extend it by configuring the Long-
term retention policy for your database(s). For more information, see Long-term retention.
You can use these automatic database backups to recover a database from various disruptive events, both
within your data center and to another data center. Using automatic database backups, the estimated time of
recovery depends on several factors including the total number of databases recovering in the same region at
the same time, the database size, the transaction log size, and network bandwidth. In most cases, the recovery
time is less than 12 hours. When recovering to another data region, the potential data loss is limited to 1 hour
by the geo-redundant storage of hourly differential database backups.

IMPORTANT
To recover using automated backups, you must be a member of the SQL Server Contributor role or the subscription
owner - see RBAC: Built-in roles. You can recover using the Azure portal, PowerShell, or the REST API. You cannot use
Transact-SQL.

Use automated backups as your business continuity and recovery mechanism if your application:
Is not considered mission critical.
Doesn't have a binding SLA therefore the downtime of 24 hours or longer will not result in financial liability.
Has a low rate of data change (low transactions per hour) and losing up to an hour of change is an
acceptable data loss.
Is cost sensitive.
If you need faster recovery, use Active Geo-Replication (discussed next). If you need to be able to recover data
from a period older than 35 days, consider archiving your database regularly to a BACPAC file (a compressed
file containing your database schema and associated data) stored either in Azure blob storage or in another
location of your choice. For more information on how to create a transactionally consistent database archive,
see create a database copy and export the database copy.
Use Active Geo -Replication and auto -failover groups to reduce recovery time and limit data loss associated
with a recovery
In addition to using database backups for database recovery in the event of a business disruption, you can use
Active Geo-Replication to configure a database to have up to four readable secondary databases in the regions
of your choice. These secondary databases are kept synchronized with the primary database using an
asynchronous replication mechanism. This feature is used to protect against business disruption in the event of
a data center outage or during an application upgrade. Active Geo-Replication can also be used to provide
better query performance for read-only queries to geographically dispersed users.
To enable automated and transparent failover you should organize your geo-replicated databases into groups
using the auto-failover group feature of SQL Database.
If the primary database goes offline unexpectedly or you need to take it offline for maintenance activities, you
can quickly promote a secondary to become the primary (also called a failover) and configure applications to
connect to the newly promoted primary. If your application is connecting to the databases using the failover
group listener, you don’t need to change the SQL connection string configuration after failover. With a planned
failover, there is no data loss. With an unplanned failover, there may be some small amount of data loss for very
recent transactions due to the nature of asynchronous replication. Using auto-failover groups, you can
customize the failover policy to minimize the potential data loss. After a failover, you can later failback - either
according to a plan or when the data center comes back online. In all cases, users experience a small amount of
downtime and need to reconnect.
IMPORTANT
To use Active Geo-Replication and auto-failover groups, you must either be the subscription owner or have
administrative permissions in SQL Server. You can configure and failover using the Azure portal, PowerShell, or the REST
API using permissions on the subscription or using Transact-SQL using permissions within SQL Server.

Use Active Geo-Replication if your application meets any of these criteria:

Is mission critical.
Has a service level agreement (SLA) that does not allow for 24 hours or more of downtime.
Downtime will result in financial liability.
Has a high rate of data change is high and losing an hour of data is not acceptable.
The additional cost of active geo-replication is lower than the potential financial liability and associated loss
of business.
>

Recover a database after a user or application error

*No one is perfect! A user might accidentally delete some data, inadvertently drop an important table, or even
drop an entire database. Or, an application might accidentally overwrite good data with bad data due to an
application defect.
In this scenario, these are your recovery options.
Perform a point-in-time restore
You can use the automated backups to recover a copy of your database to a known good point in time,
provided that time is within the database retention period. After the database is restored, you can either replace
the original database with the restored database or copy the needed data from the restored data into the
original database. If the database uses Active Geo-Replication, we recommend copying the required data from
the restored copy into the original database. If you replace the original database with the restored database, you
will need to reconfigure and resynchronize Active Geo-Replication (which can take quite some time for a large
database).
For more information and for detailed steps for restoring a database to a point in time using the Azure portal or
using PowerShell, see point-in-time restore. You cannot recover using Transact-SQL.
Restore a deleted database
If the database is deleted but the logical server has not been deleted, you can restore the deleted database to
the point at which it was deleted. This restores a database backup to the same logical SQL server from which it
was deleted. You can restore it using the original name or provide a new name or the restored database.
For more information and for detailed steps for restoring a deleted database using the Azure portal or using
PowerShell, see restore a deleted database. You cannot restore using Transact-SQL.
IMPORTANT
If the logical server is deleted, you cannot recover a deleted database.

Restore from Azure Backup Vault

If the data loss occurred outside the current retention period for automated backups and your database is
configured for long term retention, you can restore from a weekly backup in Azure Backup Vault to a new
database. At this point, you can either replace the original database with the restored database or copy the
needed data from the restored database into the original database. If you need to retrieve an old version of your
database prior to a major application upgrade, satisfy a request from auditors or a legal order, you can create a
new database using a full backup saved in the Azure Backup Vault. For more information, see Long-term
retention.

Recover a database to another region from an Azure regional data

center outage
Although rare, an Azure data center can have an outage. When an outage occurs, it causes a business disruption
that might only last a few minutes or might last for hours.
One option is to wait for your database to come back online when the data center outage is over. This works
for applications that can afford to have the database offline. For example, a development project or free trial
you don't need to work on constantly. When a data center has an outage, you won't know how long the
outage will last, so this option only works if you don't need your database for a while.
Another option is to either failover to another data region if you are using Active Geo-Replication or the
recover using geo-redundant database backups (Geo-Restore). Failover takes only a few seconds, while
recovery from backups takes hours.
When you take action, how long it takes you to recover, and how much data loss you incur in the event of a data
center outage depends upon how you decide to use the business continuity features discussed above in your
application. Indeed, you may choose to use a combination of database backups and Active Geo-Replication
depending upon your application requirements. For a discussion of application design considerations for stand-
alone databases and for elastic pools using these business continuity features, see Design an application for
cloud disaster recovery and Elastic Pool disaster recovery strategies.
The sections below provide an overview of the steps to recover using either database backups or Active Geo-
Replication. For detailed steps including planning requirements, post recovery steps and information about how
to simulate an outage to perform a disaster recovery drill, see Recover a SQL Database from an outage.
Prepare for an outage
Regardless of the business continuity feature you use, you must:
Identify and prepare the target server, including server-level firewall rules, logins, and master database level
permissions.
Determine how to redirect clients and client applications to the new server
Document other dependencies, such as auditing settings and alerts
If you do not plan and prepare properly, bringing your applications online after a failover or a recovery takes
additional time and likely also require troubleshooting at a time of stress - a bad combination.
Failover to a geo -replicated secondary database
If you are using Active Geo-Replication and auto-failover groups as your recovery mechanism, you can
configure an automatic failover policy or use manual failover. Once initiated, the failover causes the secondary
to become the new primary and ready to record new transactions and respond to queries - with minimal data
loss for the data that had not yet been replicated. For information on designing the failover process, see Design
an application for cloud disaster recovery.

NOTE
When the data center comes back online the old primaries automatically re-connect to the new primary and become
secondary databases. If you need to relocate the primary back to the original region you can initiate a planned failover
manually (failback).

Perform a Geo -Restore

If you are using automated backups with geo-redundant storage replication as your recovery mechanism,
initiate a database recovery using Geo-Restore. Recovery usually takes place within 12 hours - with data loss of
up to one hour determined by when the last hourly differential backup with taken and replicated. Until the
recovery completes, the database is unable to record any transactions or respond to any queries.

NOTE
If the data center comes back online before you switch your application over to the recovered database, you can simply
cancel the recovery.

Perform post failover / recovery tasks

After recovery from either recovery mechanism, you must perform the following additional tasks before your
users and applications are back up and running:
Redirect clients and client applications to the new server and restored database
Ensure appropriate server-level firewall rules are in place for users to connect (or use database-level
firewalls)
Ensure appropriate logins and master database level permissions are in place (or use contained users)
Configure auditing, as appropriate
Configure alerts, as appropriate

Upgrade an application with minimal downtime

Sometimes an application needs to be taken offline because of planned maintenance such as an application
upgrade. Manage application upgrades describes how to use Active Geo-Replication to enable rolling upgrades
of your cloud application to minimize downtime during upgrades and provide a recovery path in the event
something goes wrong. This article looks at two different methods of orchestrating the upgrade process and
discusses the benefits and trade-offs of each option.

Next steps
For a discussion of application design considerations for stand-alone databases and for elastic pools, see Design
an application for cloud disaster recovery and Elastic Pool disaster recovery strategies.
Learn about SQL Database backups
4/14/2017 • 5 min to read • Edit Online

SQL Database automatically creates a database backups and uses Azure read-access geo-redundant storage
(RA-GRS) to provide geo-redundancy. These backups are created automatically and at no additional charge. You
don't need to do anything to make them happen. Database backups are an essential part of any business
continuity and disaster recovery strategy because they protect your data from accidental corruption or deletion.
If you want to keep backups in your own storage container you can configure a long-term backup retention
policy. For more information, see Long-term retention.

What is a SQL Database backup?

SQL Database uses SQL Server technology to create full, differential, and transaction log backups. The
transaction log backups generally happen every 5 - 10 minutes, with the frequency based on the performance
level and amount of database activity. Transaction log backups, with full and differential backups, allow you to
restore a database to a specific point-in-time to the same server that hosts the database. When you restore a
database, the service figures out which full, differential, and transaction log backups need to be restored.
You can use these backups to:
Restore a database to a point-in-time within the retention period. This operation will create a new database
in the same server as the original database.
Restore a deleted database to the time it was deleted or any time within the retention period. The deleted
database can only be restored in the same server where the original database was created.
Restore a database to another geographical region. This allows you to recover from a geographic disaster
when you cannot access your server and database. It creates a new database in any existing server anywhere
in the world.
Restore a database from a specific backup stored in your Azure Recovery Services vault. This allows you to
restore an old version of the database to satisfy a compliance request or to run an old version of the
application. See Long-term retention.
To perform a restore, see restore database from backups.

NOTE
In Azure storage, the term replication refers to copying files from one location to another. SQL's database replication
refers to keeping to multiple secondary databases synchronized with a primary database.

How much backup storage is included at no cost?

SQL Database provides up to 200% of your maximum provisioned database storage as backup storage at no
additional cost. For example, if you have a Standard DB instance with a provisioned DB size of 250 GB, you have
500 GB of backup storage at no additional charge. If your database exceeds the provided backup storage, you
can choose to reduce the retention period by contacting Azure Support. Another option is to pay for extra
backup storage that is billed at the standard Read-Access Geographically Redundant Storage (RA-GRS) rate.

How often do backups happen?

Full database backups happen weekly, differential database backups generally happen every few hours, and
transaction log backups generally happen every 5 - 10 minutes. The first full backup is scheduled immediately
after a database is created. It usually completes within 30 minutes, but it can take longer when the database is of
a significant size. For example, the initial backup can take longer on a restored database or a database copy.
After the first full backup, all further backups are scheduled automatically and managed silently in the
background. The exact timing of all database backups is determined by the SQL Database service as it balances
the overall system workload.
The backup storage geo-replication occurs based on the Azure Storage replication schedule.

How long do you keep my backups?

Each SQL Database backup has a retention period that is based on the service-tier of the database. The retention
period for a database in the:
Basic service tier is 7 days.
Standard service tier is 35 days.
Premium service tier is 35 days.
If you downgrade your database from the Standard or Premium service tiers to Basic, the backups are saved for
seven days. All existing backups older than seven days are no longer available.
If you upgrade your database from the Basic service tier to Standard or Premium, SQL Database keeps existing
backups until they are 35 days old. It keeps new backups as they occur for 35 days.
If you delete a database, SQL Database keeps the backups in the same way it would for an online database. For
example, suppose you delete a Basic database that has a retention period of seven days. A backup that is four
days old is saved for three more days.

IMPORTANT
If you delete the Azure SQL server that hosts SQL Databases, all databases that belong to the server are also deleted and
cannot be recovered. You cannot restore a deleted server.

How to extend the backup retention period?

If your application requires that the backups are available for longer period of time you can extend the built-in
retention period by configuring the Long-term backup retention policy for individual databases (LTR policy).
This allows you to extend the built-it retention period from 35 days to up to 10 years. For more information, see
Long-term retention.
Once you add the LTR policy to a database using Azure portal or API, the weekly full database backups will be
automatically copied to your own Azure Backup Service Vault. If your database is encrypted with TDE the
backups are automatically encrypted at rest. The Services Vault will automatically delete your expired backups
based on their timestamp and the LTR policy. So you don't need to manage the backup schedule or worry about
the cleanup of the old files. The restore API supports backups stored in the vault as long as the vault is in the
same subscription as your SQL database. You can use the Azure portal or PowerShell to access these backups.

TIP
For a How-to guide, see Configure and restore from Azure SQL Database long-term backup retention

Next steps
Database backups are an essential part of any business continuity and disaster recovery strategy because
they protect your data from accidental corruption or deletion. To learn about the other Azure SQL Database
business continuity solutions, see Business continuity overview.
To restore to a point in time using the Azure portal, see restore database to a point in time using the Azure
portal.
To restore to a point in time using PowerShell, see restore database to a point in time using PowerShell.
To configure, manage, and restore from long-term retention of automated backups in an Azure Recovery
Services vault using the Azure portal, see Manage long-term backup retention using the Azure portal.
To configure, manage, and restore from long-term retention of automated backups in an Azure Recovery
Services vault using PowerShell, see Manage long-term backup retention using PowerShell.
Storing Azure SQL Database Backups for up to 10
years
4/19/2017 • 6 min to read • Edit Online

Many applications have regulatory, compliance, or other business purposes that require you to retain the
automatic full database backups beyond the 7-35 days provided by SQL Database's automatic backups. The
Long-Term Backup Retention feature enables you to store your Azure SQL Database backups in an Azure
Recovery Services vault for up to 10 years. You can store up to 1000 databases per vault. You can select any
backup in the vault to restore it as a new database.

IMPORTANT
Long-Term Backup Retention is currently in preview and available in the following regions: Australia East, Australia
Southeast, Brazil South, Central US, East Asia, East US, East US 2, India Central, India South, Japan East, Japan West, North
Central US, North Europe, South Central US, Southeast Asia, West Europe, and West US.

NOTE
You can enable up to 200 databases per vault during a 24-hour period. Therefore, we recommend that you use a
separate vault for each server to minimize the impact of this limit.

How does SQL Database long-term backup retention work?

Long-term backup retention of backups allows you to associate an Azure SQL Database server with an Azure
Recovery Services vault.
The vault must be created in the same Azure subscription that created the SQL server and in the same
geographic region and resource group.
You then configure a retention policy for any database. The policy causes the weekly full database backups be
copied to the Recovery Services vault and retained for the specified retention period (up to 10 years).
You can then restore from any of these backups to a new database in any server in the subscription. The copy
is performed by Azure storage from existing backups and has no performance impact on the existing
database.

TIP
For a how-to guide, see Configure and restore from Azure SQL Database long-term backup retention

How do I enable long-term backup retention?

To configure long-term backup retention for a database:
1. Create an Azure Recovery Services vault in the same region, subscription, and resource group as your SQL
Database server.
2. Register the server to the vault
3. Create an Azure Recovery Services Protection Policy
4. Apply the protection policy to the databases that require long-term backup retention
Azure portal
To configure, manage, and restore from long-term backup retention of automated backups in an Azure
Recovery Services vault using the Azure portal, click Long-term backup retention, select a database, and then
click Configure.

To configure, manage, and restore from long-term backup retention of automated backups in an Azure
Recovery Services vault using PowerShell, see Configure and restore from Azure SQL Database long-term
backup retention.

How do I restore a database stored with the long-term backup

retention feature?
To recover from a long-term backup retention backup:
1. List the vault where the backup is stored
2. List the container that is mapped to your logical server
3. List the data source within the vault that is mapped to your database
4. List the recovery points available to restore
5. Restore from the recovery point to the target server within your subscription
To configure, manage, and restore from long-term backup retention of automated backups in an Azure
Recovery Services vault using the Azure portal, see Manage long-term backup retention usihg the Azure portal.
To configure, manage, and restore from long-term backup retention of automated backups in an Azure
Recovery Services vault using PowerShell, see Manage long-term backup retention usihg PowerShell.

How much does long-term backup retention cost?

Long-term backup retention of an Azure SQL database is charged according to the Azure backup services
pricing rates.
After the Azure SQL Database server is registered to the vault, you are charged for the total storage that is used
by the weekly backups stored in the vault.

View available backups stored in long-term backup retention

To configure, manage, and restore from long-term backup retention of automated backups in an Azure
Recovery Services vault using the Azure portal, see Manage long-term backup retention using the Azure portal.
To configure, manage, and restore from long-term backup retention of automated backups in an Azure
Recovery Services vault using PowerShell, see Manage long-term backup retention using PowerShell.

Disabling Long-term Retention

The Recovery Service automatically handles cleanup of backups based on the provided retention policy.
To stop sending the backups for a specific database to the vault, remove the retention policy for that
database.

Set-AzureRmSqlDatabaseBackupLongTermRetentionPolicy -ResourceGroupName 'RG1' -ServerName 'Server1' -DatabaseName 'DB1'

-State 'Disabled' -ResourceId $policy.Id

NOTE
The backups already in the vault are not be impacted. They are automatically deleted by the Recovery Service when their
retention period expires.

Removing long-term backup retention backups from the Azure

Recovery Services vault
To remove long-term backup retention backups from the vault, see Delete long-term backup retention backups

Long-term backup retention FAQ:

1. Q: Can I manually delete specific backups in the vault?
A: Not currently. The vault automatically cleans up backups when the retention period has expired.
2. Q: Can I register my server to store Backups to more than one vault?
A: No, today you can only store backups to one vault at a time.
3. Q: Can I have a vault and server in different subscriptions?
A: No, currently the vault and server must be in both the same subscription and resource group.
4. Q: Can I use a vault I created in a different region than my server’s region?
A: No, the vault and server must be in the same region to minimize the copy time and avoid the traffic
charges.
5. Q: How many databases can I store in one vault?
A: Currently, we only support up to 1000 databases per vault.
6. Q. How many vaults can I create per subscription?
A. You can create up to 25 vaults per subscription.
7. Q. How many databases can I configure per day per vault?
A. You can only set up 200 databases per day per vault.
8. Q: Does long-term backup retention work with elastic pools?
A: Yes. Any database in the pool can be configured with the retention policy.
9. Q: Can I choose the time at which the backup is created?
A: No, SQL Database controls the backups schedule to minimize the performance impact on your
databases.
10. Q: I have TDE enabled for my database. Can I use TDE with the vault?
A. Yes, TDE is supported. You can restore the database from the vault even if the original database no
longer exists.
11. Q. What happens with the backups in the vault if my subscription is suspended?
A. If your subscription is suspended, we retain the existing databases and backups. New backups are not
being copied to the vault. After you reactivate the subscription, the service resumes copying backups to
the vault. Your vault becomes accessible to the restore operations using the backups that had been
copied there before the subscription was suspended.
12. Q: Can I get access to the SQL Database backup files so I can download / restore to SQL Server?
A: No, not currently.
13. Q: Is it possible to have multiple Schedules (Daily, Weekly, Monthly, Yearly) within a SQL Retention Policy.
A: No, this is currently only available for virtual machine backups.
14. Q. What if we set up long-term backup retention on a database that is an active geo-replication
secondary?
A: Currently we don't take backups on replicas, and therefore, there is no option for long-term backup
retention on secondaries. However, it is important for a customer to set up long-term backup retention
on an active geo-replication secondary for these reasons:
When a failover happens and the database becomes a primary, we take a full backup and this full
backup is uploaded to vault.
There is no extra cost to the customer for setting up long-term backup retention on a secondary.

SQL Database provides these options for database recovery using automated database backups and backups in
long-term retention. You can restore from a database backup to:
A new database on the same logical server recovered to a specified point in time within the retention period.
A database on the same logical server recovered to the deletion time for a deleted database.
A new database on any logical server in any region recovered to the point of the most recent daily backups in
geo-replicated blob storage (RA-GRS).
You can also use automated database backups to create a database copy on any logical server in any region.

Recovery time
The recovery time to restore a database using automated database backups is impacted by several factors:
The size of the database
The performance level of the database
The number of transaction logs involved
The amount of activity that needs to be replayed to recover to the restore point
The network bandwidth if the restore is to a different region
The number of concurrent restore requests being processed in the target region.
For a very large, and/or active database, the restore may take several hours. If there is prolonged outage
in a region, it is possible that there are large numbers of Geo-Restore requests being processed by other
regions. When there are many requests, the recovery time may increase for databases in that region.
Most database restores complete within 12 hours.
There is no built-in functionality to do bulk restore. The Azure SQL Database: Full Server Recovery script
is an example of one way of accomplishing this task.

IMPORTANT
To recover using automated backups, you must be a member of the SQL Server Contributor role in the subscription or be
the subscription owner. You can recover using the Azure portal, PowerShell, or the REST API. You cannot use Transact-
SQL.

Point-In-Time Restore
You can restore an existing database to an earlier point in time as a new database on the same logical server
using the Azure portal, PowerShell, or the REST API.

IMPORTANT
You cannot overwrite the existing database during restore.

The database can be restored to any service tier or performance level, and as a single database or into an elastic
pool. Ensure you have sufficient resources on the logical server or in the elastic pool to which you are restoring
the database. Once complete, the restored database is a normal, fully accessible, online database. The restored
database is charged at normal rates based on its service tier and performance level. You do not incur charges
until the database restore is complete.
You generally restore a database to an earlier point for recovery purposes. When doing so, you can treat the
restored database as a replacement for the original database or use it to retrieve data from and then update the
original database.
Database replacement: If the restored database is intended as a replacement for the original database, you
should verify the performance level and/or service tier are appropriate and scale the database if necessary.
You can rename the original database and then give the restored database the original name using the
ALTER DATABASE command in T-SQL.
Data recovery: If you plan to retrieve data from the restored database to recover from a user or application
error, you need to write and execute the necessary data recovery scripts to extract data from the restored
database to the original database. Although the restore operation may take a long time to complete, the
restoring database is visible in the database list throughout the restore process. If you delete the database
during the restore, the restore operation is canceled and you are not charged for the database that did not
complete the restore.
Azure portal
To recover to a point in time using the Azure portal, open the page for your database and click Restore on the
toolbar.

Deleted database restore

You can restore a deleted database to the deletion time for a deleted database on the same logical server using
the Azure portal, PowerShell, or the REST (createMode=Restore).

IMPORTANT
If you delete an Azure SQL Database server instance, all its databases are also deleted and cannot be recovered. There is
currently no support for restoring a deleted server.

Azure portal
To recover a deleted database during its retention period using the Azure portal, open the page for your server
and in the Operations area, click Deleted databases.
Geo-Restore
You can restore a SQL database on any server in any Azure region from the most recent geo-replicated full and
differential backups. Geo-Restore uses a geo-redundant backup as its source and can be used to recover a
database even if the database or datacenter is inaccessible due to an outage.
Geo-Restore is the default recovery option when your database is unavailable because of an incident in the
region where the database is hosted. If a large-scale incident in a region results in unavailability of your
database application, you can restore a database from the geo-replicated backups to a server in any other
region. There is a delay between when a differential backup is taken and when it is geo-replicated to an Azure
blob in a different region. This delay can be up to an hour, so, if a disaster occurs, there can be up to one hour
data loss. The following illustration shows restore of the database from the last available backup in another
region.

For detailed information about using Geo-Restore to recover from an outage, see Recover from an outage
IMPORTANT
Recovery from backups is the most basic of the disaster recovery solutions available in SQL Database with the longest
RPO and Estimate Recovery Time (ERT). For Basic databases with maximum size of 2 GB Geo-Restore, provides a
reasonable DR solution with an ERT of 12 hours. For larger Standard or Premium databases, if shorter recovery times are
desired, or to reduce the likelihood of data loss you should consider using Active Geo-Replication. Active Geo-Replication
offers a much lower RPO and ERT as it only requires you initiate a failover to a continuously replicated secondary. For
details, see Active Geo-Replication.

Azure portal
To geo-restore a database during its retention period using the Azure portal, open the SQL Databases page and
then click Add. In the Select source text box, select Backup. Specify the backup from which to perform the
recovery in the region and on the server of your choice.

Programmatically performing recovery using automated backups

As previously discussed, in addition to the Azure portal, database recovery can be performed programmatically
using Azure PowerShell or the REST API. The following tables describe the set of commands available.
PowerShell
CMDLET DESCRIPTION

Get-AzureRmSqlDatabase Gets one or more databases.

Get-AzureRMSqlDeletedDatabaseBackup Gets a deleted database that you can restore.

Get-AzureRmSqlDatabaseGeoBackup Gets a geo-redundant backup of a database.

Restore-AzureRmSqlDatabase Restores a SQL database.

REST API
API DESCRIPTION

REST (createMode=Recovery) Restores a database

Get Create or Update Database Status Returns the status during a restore operation

Summary
Automatic backups protect your databases from user and application errors, accidental database deletion, and
prolonged outages. This built-in capability is available for all service tiers and performance levels.

Next steps
For a business continuity overview and scenarios, see Business continuity overview
To learn about Azure SQL Database automated backups, see SQL Database automated backups
To learn about long-term backup retention, see Long-term backup retention
To configure, manage, and restore from long-term retention of automated backups in an Azure Recovery
Services vault using the Azure portal, see Configure and use long-term backup retention.
To learn about faster recovery options, see Active-Geo-Replication
Overview: SQL Database Active Geo-Replication
4/14/2017 • 11 min to read • Edit Online

Active Geo-Replication enables you to configure up to four readable secondary databases in the same or
different data center locations (regions). Secondary databases are available for querying and for failover in the
case of a data center outage or the inability to connect to the primary database. Active geo-replication must be
between databases within the same subscription.

NOTE
Active Geo-Replication (readable secondaries) is now available for all databases in all service tiers. In April 2017, the
non-readable secondary type will be retired and existing non-readable databases will automatically be upgraded to
readable secondaries.

You can configure Active Geo-Replication using the Azure portal, PowerShell, Transact-SQL, or the REST API -
Create or Update Database.
If for any reason your primary database fails, or simply needs to be taken offline, you can failover to any of
your secondary databases. When failover is activated to one of the secondary databases, all other secondaries
are automatically linked to the new primary.
You can failover to a secondary using the Azure portal, PowerShell, Transact-SQL, the REST API - Planned
Failover, or REST API - Unplanned Failover.
After failover, ensure the authentication requirements for your server and database are configured on the new
primary. For details, see SQL Database security after disaster recovery.
The Active Geo-Replication feature implements a mechanism to provide database redundancy within the same
Microsoft Azure region or in different regions (geo-redundancy). Active Geo-Replication asynchronously
replicates committed transactions from a database to up to four copies of the database on different servers,
using read committed snapshot isolation (RCSI) for isolation. When Active Geo-Replication is configured, a
secondary database is created on the specified server. The original database becomes the primary database.
The primary database asynchronously replicates committed transactions to each of the secondary databases.
Only full transactions are replicated. While at any given point, the secondary database might be slightly behind
the primary database, the secondary data is guaranteed to never have partial transactions. The specific RPO
data can be found at Overview of Business Continuity.
One of the primary benefits of Active Geo-Replication is that it provides a database-level disaster recovery
solution with low recovery time. When you place the secondary database on a server in a different region, you
add maximum resilience to your application. Cross-region redundancy enables applications to recover from a
permanent loss of an entire datacenter or parts of a datacenter caused by natural disasters, catastrophic
human errors, or malicious acts. The following figure shows an example of Active Geo-Replication configured
with a primary in the North Central US region and secondary in the South Central US region.
Another key benefit is that the secondary databases are readable and can be used to offload read-only
workloads such as reporting jobs. If you only intend to use the secondary database for load-balancing, you can
create it in the same region as the primary. Creating a secondary in the same region, does not increase the
application's resilience to catastrophic failures.
Other scenarios where Active Geo-Replication can be used include:
Database migration: You can use Active Geo-Replication to migrate a database from one server to
another online with minimum downtime.
Application upgrades: You can create an extra secondary as a fail back copy during application upgrades.
To achieve real business continuity, adding database redundancy between datacenters is only part of the
solution. Recovering an application (service) end-to-end after a catastrophic failure requires recovery of all
components that constitute the service and any dependent services. Examples of these components include
the client software (for example, a browser with a custom JavaScript), web front ends, storage, and DNS. It is
critical that all components are resilient to the same failures and become available within the recovery time
objective (RTO) of your application. Therefore, you need to identify all dependent services and understand the
guarantees and capabilities they provide. Then, you must take adequate steps to ensure that your service
functions during the failover of the services on which it depends. For more information about designing
solutions for disaster recovery, see Designing Cloud Solutions for Disaster Recovery Using Active Geo-
Replication.

Active Geo-Replication capabilities

The Active Geo-Replication feature provides the following essential capabilities:
Automatic Asynchronous Replication: You can only create a secondary database by adding to an
existing database. The secondary can only be created in a different Azure SQL Database server. Once
created, the secondary database is populated with the data copied from the primary database. This process
is known as seeding. After secondary database has been created and seeded, updates to the primary
database are asynchronously replicated to the secondary database automatically. Asynchronous replication
means that transactions are committed on the primary database before they are replicated to the
secondary database.
Multiple secondary databases: Two or more secondary databases increase redundancy and level of
protection for the primary database and application. If multiple secondary databases exist, the application
remains protected even if one of the secondary databases fails. If there is only one secondary database, and
it fails, the application is exposed to higher risk until a new secondary database is created.
Readable secondary databases: An application can access a secondary database for read-only operations
using the same or different security principals used for accessing the primary database. The secondary
databases operate in snapshot isolation mode to ensure replication of the updates of the primary (log
replay) is not delayed by queries executed on the secondary.

NOTE
The log replay is delayed on the secondary database if there are schema updates that it is receiving from the Primary
that require a schema lock on the secondary database.

Active geo-replication of elastic pool databases: Active Geo-Replication can be configured for any
database in any elastic pool. The secondary database can be in another elastic pool. For regular databases,
the secondary can be an elastic pool and vice versa as long as the service tiers are the same.
Configurable performance level of the secondary database: A secondary database can be created
with lower performance level than the primary. Both primary and secondary databases are required to
have the same service tier. This option is not recommended for applications with high database write
activity because the increased replication lag increases the risk of substantial data loss after a failover. In
addition, after failover the application’s performance is impacted until the new primary is upgraded to a
higher performance level. The log IO percentage chart on Azure portal provides a good way to estimate the
minimal performance level of the secondary that is required to sustain the replication load. For example, if
your Primary database is P6 (1000 DTU) and its log IO percent is 50% the secondary needs to be at least P4
(500 DTU). You can also retrieve the log IO data using sys.resource_stats or sys.dm_db_resource_stats
database views. For more information on the SQL Database performance levels, see SQL Database options
and performance.
User-controlled failover and failback: A secondary database can explicitly be switched to the primary
role at any time by the application or the user. During a real outage the “unplanned” option should be used,
which immediately promotes a secondary to be the primary. When the failed primary recovers and is
available again, the system automatically marks the recovered primary as a secondary and bring it up-to-
date with the new primary. Due to the asynchronous nature of replication, a small amount of data can be
lost during unplanned failovers if a primary fails before it replicates the most recent changes to the
secondary. When a primary with multiple secondaries fails over, the system automatically reconfigures the
replication relationships and links the remaining secondaries to the newly promoted primary without
requiring any user intervention. After the outage that caused the failover is mitigated, it may be desirable to
return the application to the primary region. To do that, the failover command should be invoked with the
“planned” option.
Keeping credentials and firewall rules in sync: We recommend using database firewall rules for geo-
replicated databases so these rules can be replicated with the database to ensure all secondary databases
have the same firewall rules as the primary. This approach eliminates the need for customers to manually
configure and maintain firewall rules on servers hosting both the primary and secondary databases.
Similarly, using contained database users for data access ensures both primary and secondary databases
always have the same user credentials so during a failover, there is no disruptions due to mismatches with
logins and passwords. With the addition of Azure Active Directory, customers can manage user access to
both primary and secondary databases and eliminating the need for managing credentials in databases
altogether.

Upgrading or downgrading a primary database

You can upgrade or downgrade a primary database to a different performance level (within the same service
tier) without disconnecting any secondary databases. When upgrading, we recommend that you upgrade the
secondary database first, and then upgrade the primary. When downgrading, reverse the order: downgrade
the primary first, and then downgrade the secondary.
The secondary database must be in the same service tier as the primary, so migrating your primary database
to a different service tier requires you to either terminate the geo-replication link and rename the secondary
database, or simply drop it. Then migrate the primary to the new service tier and reconfigure geo-replication.
Your new secondary will be created automatically with the same performance level as the primary by default.

Preventing the loss of critical data

Due to the high latency of wide area networks, continuous copy uses an asynchronous replication mechanism.
Asynchronous replication makes some data loss unavoidable if a failure occurs. However, some applications
may require no data loss. To protect these critical updates, an application developer can call the
sp_wait_for_database_copy_sync system procedure immediately after committing the transaction. Calling
sp_wait_for_database_copy_sync blocks the calling thread until the last committed transaction has been
replicated to the secondary database. The procedure will wait until all queued transactions have been
acknowledged by the secondary database. sp_wait_for_database_copy_sync is scoped to a specific
continuous copy link. Any user with the connection rights to the primary database can call this procedure.

NOTE
The delay caused by a sp_wait_for_database_copy_sync procedure call can be significant. The delay depends on the
size of the transaction log length at the moment and this call does not return until the entire log is replicated. Avoid
calling this procedure unless absolutely necessary.

Programmatically managing Active Geo-Replication

As discussed previously, Active Geo-Replication can also be managed programmatically using Azure
PowerShell and the REST API. The following tables describe the set of commands available.
Azure Resource Manager API and role-based security: Active Geo-Replication includes a set of Azure
Resource Manager APIs for management, including Azure Resource Manager-based PowerShell cmdlets.
These APIs require the use of resource groups and support role-based security (RBAC). For more
information on how to implement access roles see Azure Role-Based Access Control.

NOTE
Many new features of Active Geo-Replication are only supported using Azure Resource Manager based Azure SQL REST
API and Azure SQL Database PowerShell cmdlets. The (classic) REST API and Azure SQL Database (classic) cmdlets are
supported for backward compatibility so using the Azure Resource Manager-based APIs are recommended.

Transact-SQL
COMMAND DESCRIPTION

ALTER DATABASE (Azure SQL Database) Use ADD SECONDARY ON SERVER argument to create a
secondary database for an existing database and starts data
replication

ALTER DATABASE (Azure SQL Database) Use FAILOVER or FORCE_FAILOVER_ALLOW_DATA_LOSS

to switch a secondary database to be primary to initiate
failover

ALTER DATABASE (Azure SQL Database) Use REMOVE SECONDARY ON SERVER to terminate a data
replication between a SQL Database and the specified
secondary database.

sys.geo_replication_links (Azure SQL Database) Returns information about all existing replication links for
each database on the Azure SQL Database logical server.

sys.dm_geo_replication_link_status (Azure SQL Database) Gets the last replication time, last replication lag, and other
information about the replication link for a given SQL
database.

sys.dm_operation_status (Azure SQL Database) Shows the status for all database operations including the
status of the replication links.

sp_wait_for_database_copy_sync (Azure SQL Database) causes the application to wait until all committed
transactions are replicated and acknowledged by the active
secondary database.

PowerShell
CMDLET DESCRIPTION

Get-AzureRmSqlDatabase Gets one or more databases.

New-AzureRmSqlDatabaseSecondary Creates a secondary database for an existing database and

starts data replication.

Set-AzureRmSqlDatabaseSecondary Switches a secondary database to be primary to initiate

failover.

Remove-AzureRmSqlDatabaseSecondary Terminates data replication between a SQL Database and

the specified secondary database.

Get-AzureRmSqlDatabaseReplicationLink Gets the geo-replication links between an Azure SQL

Database and a resource group or SQL Server.

REST API
API DESCRIPTION

Create or Update Database (createMode=Restore) Creates, updates, or restores a primary or a secondary

database.
API DESCRIPTION

Get Create or Update Database Status Returns the status during a create operation.

Set Secondary Database as Primary (Planned Failover) Promote a secondary database in a Geo-Replication
partnership to become the new primary database.

Set Secondary Database as Primary (Unplanned Failover) To force a failover to the secondary database and set the
secondary as the primary.

Get Replication Links Gets all replication links for a given SQL database in a geo-
replication partnership. It retrieves the information visible in
the sys.geo_replication_links catalog view.

Get Replication Link Gets a specific replication link for a given SQL database in a
geo-replication partnership. It retrieves the information
visible in the sys.geo_replication_links catalog view.

Next steps
For a business continuity overview and scenarios, see Business continuity overview
To learn about Azure SQL Database automated backups, see SQL Database automated backups.
To learn about using automated backups for recovery, see Restore a database from the service-initiated
backups.
To learn about using automated backups for archiving, see database copy.
To learn about authentication requirements for a new primary server and database, see SQL Database
security after disaster recovery.
Configure and manage Azure SQL Database security
for geo-restore or failover
2/16/2017 • 4 min to read • Edit Online

NOTE
Active Geo-Replication is now available for all databases in all service tiers.

Overview of authentication requirements for disaster recovery

This topic describes the authentication requirements to configure and control Active Geo-Replication and the
steps required to set up user access to the secondary database. It also describes how enable access to the
recovered database after using geo-restore. For more information on recovery options, see Business Continuity
Overview.

Disaster recovery with contained users

Unlike traditional users, which must be mapped to logins in the master database, a contained user is managed
completely by the database itself. This has two benefits. In the disaster recovery scenario, the users can continue
to connect to the new primary database or the database recovered using geo-restore without any additional
configuration, because the database manages the users. There are also potential scalability and performance
benefits from this configuration from a login perspective. For more information, see Contained Database Users -
Making Your Database Portable.
The main trade-off is that managing the disaster recovery process at scale is more challenging. When you have
multiple databases that use the same login, maintaining the credentials using contained users in multiple
database may negate the benefits of contained users. For example, the password rotation policy requires that
changes be made consistently in multiple databases rather than changing the password for the login once in the
master database. For this reason, if you have multiple databases that use the same user name and password,
using contained users is not recommended.

How to configure logins and users

If you are using logins and users (rather than contained users), you must make take extra steps to insure that the
same logins exist in the master database. The following sections outline the steps involved and additional
considerations.
Set up user access to a secondary or recovered database
In order for the secondary database to be usable as a read-only secondary database, and to ensure proper access
to the new primary database or the database recovered using geo-restore, the master database of the target
server must have the appropriate security configuration in place before the recovery.
The specific permissions for each step are described later in this topic.
Preparing user access to a Geo-Replication secondary should be performed as part configuring Geo-Replication.
Preparing user access to the geo-restored databases should be performed at any time when the original server is
online (e.g. as part of the DR drill).
NOTE
If you failover or geo-restore to a server that does not have properly configured logins access to it will be limited to the
server admin account.

Setting up logins on the target server involves three steps outlined below:
1. Determine logins with access to the primary database:
The first step of the process is to determine which logins must be duplicated on the target server. This is
accomplished with a pair of SELECT statements, one in the logical master database on the source server and one
in the primary database itself.
Only the server admin or a member of the LoginManager server role can determine the logins on the source
server with the following SELECT statement.

SELECT [name], [sid]

FROM [sys].[sql_logins]
WHERE [type_desc] = 'SQL_Login'

Only a member of the db_owner database role, the dbo user, or server admin, can determine all of the database
user principals in the primary database.

SELECT [name], [sid]

FROM [sys].[database_principals]
WHERE [type_desc] = 'SQL_USER'

2. Find the SID for the logins identified in step 1:

By comparing the output of the queries from the previous section and matching the SIDs, you can map the server
login to database user. Logins that have a database user with a matching SID have user access to that database as
that database user principal.
The following query can be used to see all of the user principals and their SIDs in a database. Only a member of
the db_owner database role or server admin can run this query.

SELECT [name], [sid]

FROM [sys].[database_principals]
WHERE [type_desc] = 'SQL_USER'

NOTE
The INFORMATION_SCHEMA and sys users have NULL SIDs, and the guest SID is 0x00. The dbo SID may start with
0x01060000000001648000000000048454, if the database creator was the server admin instead of a member of
DbManager.

3. Create the logins on the target server:

The last step is to go to the target server, or servers, and generate the logins with the appropriate SIDs. The basic
syntax is as follows.

CREATE LOGIN [<login name>]

WITH PASSWORD = <login password>,
SID = <desired login SID>
NOTE
If you want to grant user access to the secondary, but not to the primary, you can do that by altering the user login on the
primary server by using the following syntax.
ALTER LOGIN DISABLE
DISABLE doesn’t change the password, so you can always enable it if needed.

Next steps
For more information on managing database access and logins, see SQL Database security: Manage database
access and login security.
For more information on contained database users, see Contained Database Users - Making Your Database
Portable.
For information about using and configuring Active Geo-Replication, see Active Geo-Replication
For informatin about using Geo-Restore, see Geo-Restore
Application design for cloud disaster recovery using
Active Geo-Replication in SQL Database
1/23/2017 • 10 min to read • Edit Online

NOTE
Active Geo-Replication is now available for all databases in all tiers.

Learn how to use Active Geo-Replication in SQL Database to design database applications resilient to regional
failures and catastrophic outages. For business continuity planning, consider the application deployment topology,
the service level agreement you are targeting, traffic latency, and costs. In this article, we look at the common
application patterns, and discuss the benefits and trade-offs of each option. For information about Active Geo-
Replication with Elastic Pools, see Elastic Pool disaster recovery strategies.

Design pattern 1: Active-passive deployment for cloud disaster

recovery with a co-located database
This option is best suited for applications with the following characteristics:
Active instance in a single Azure region
Strong dependency on read-write (RW) access to data
Cross-region connectivity between the application logic and the database is not acceptable due to latency and
traffic cost
In this case, the application deployment topology is optimized for handling regional disasters when all application
components are impacted and need to failover as a unit. For geographic redundancy, the application logic and the
database are replicated to another region but they are not used for the application workload under the normal
conditions. The application in the secondary region should be configured to use a SQL connection string to the
secondary database. Traffic manager is set up to use failover routing method.

NOTE
Azure traffic manager is used throughout this article for illustration purposes only. You can use any load-balancing solution
that supports failover routing method.

In addition to the main application instances, you should consider deploying a small worker role application that
monitors your primary database by issuing periodic T-SQL read-only (RO) commands. You can use it to
automatically trigger failover, to generate an alert on your application's admin console, or to do both. To ensure
that monitoring is not impacted by region-wide outages, you should deploy the monitoring application instances
to each region and have each instance connected to the primary database in the primary region and the
secondary database in the local region.

NOTE
Both monitoring applications should be active and probe both primary and secondary databases. The latter can be used to
detect a failure in the secondary region and alert when the application is not protected.
The following diagram shows this configuration before an outage.

After an outage in the primary region, the monitoring application detects that the primary database is not
accessible and registers an alert. Depending on your application SLA, you can decide how many consecutive
monitoring probes should fail before it declares a database outage. To achieve coordinated failover of the
application end-point and the database, you should have the monitoring application perform the following steps:
1. Update the status of the primary end-point in order to trigger end-point failover.
2. Call the secondary database to initiate database failover.
After failover, the application will process the user requests in the secondary region but will remain co-located
with the database because the primary database will now be in the secondary region. This scenario is illustrated
by the next diagram. In all diagrams, solid lines indicate active connections, dotted lines indicate suspended
connections and stop signs indicate action triggers.
If an outage happens in the secondary region, the replication link between the primary and the secondary
database is suspended and the monitoring application registers an alert that the primary database is exposed. The
application's performance is not impacted, but the application operates exposed and therefore at higher risk in
case both regions fail in succession.

NOTE
We only recommend deployment configurations with a single DR region. This is because most of the Azure geographies
have two regions. These configurations will not protect your application from a catastrophic failure of both regions. In an
unlikely event of such a failure, you can recover your databases in a third region using geo-restore operation.

Once the outage is mitigated, the secondary database is automatically synchronized with the primary. During
synchronization, performance of the primary could be slightly impacted depending on the amount of data that
needs to be synchronized. The following diagram illustrates an outage in the secondary region.
The key advantages of this design pattern are:
The SQL connection string is set during the application deployment in each region and doesn’t change after
failover.
The application's performance is not impacted by failover as the application and the database are always co-
located.
The main tradeoff is that the redundant application instance in the secondary region is only used for disaster
recovery.

Design pattern 2: Active-active deployment for application load

balancing
This cloud disaster recovery option is best suited for applications with the following characteristics:
High ratio of database reads to writes
Database write latency does not impact the end-user experience
Read-only logic can be separated from read-write logic by using a different connection string
Read-only logic does not depend on data being fully synchronized with the latest updates
If your applications have these characteristics, load balancing the end-user connections across multiple application
instances in different regions can improve performance and the end-user experience. To implement load-
balancing, each region should have an active instance of the application with the read-write (RW) logic connected
to the primary database in the primary region. The read-only (RO) logic should be connected to a secondary
database in the same region as the application instance. Traffic manager should be set up to use round-robin
routing or performance routing with end-point monitoring enabled for each application instance.
As in pattern #1, you should consider deploying a similar monitoring application. But unlike pattern #1, the
monitoring application will not be responsible for triggering the end-point failover.

NOTE
While this pattern uses more than one secondary database, only one of the secondaries would be used for failover for the
reasons noted earlier. Because this pattern requires read-only access to the secondary, it requires Active Geo-Replication.

Traffic manager should be configured for performance routing to direct the user connections to the application
instance closest to the user's geographic location. The following diagram illustrates this configuration before an
outage.

If a database outage is detected in the primary region, you initiate failover of the primary database to one of the
secondary regions, changing the location of the primary database. The traffic manager automatically excludes the
offline end-point from the routing table, but continues routing the end-user traffic to the remaining online
instances. Because the primary database is now in a different region, all online instances must change their read-
write SQL connection string to connect to the new primary. It is important that you make this change prior to
initiating the database failover. The read-only SQL connection strings should remain unchanged as they always
point to the database in the same region. The failover steps are:
1. Change read-write SQL connection strings to point to the new primary.
2. Call the designated secondary database in order to initiate database failover.
The following diagram illustrates the new configuration after the failover.
In case of an outage in one of the secondary regions, the traffic manager automatically removes the offline end-
point in that region from the routing table. The replication channel to the secondary database in that region is
suspended. Because the remaining regions get additional user traffic in this scenario, the application's
performance is impacted during the outage. Once the outage is mitigated, the secondary database in the impacted
region is immediately synchronized with the primary. During synchronization performance of the primary could
be slightly impacted depending on the amount of data that needs to be synchronized. The following diagram
illustrates an outage in one of the secondary regions.
The key advantage of this design pattern is that you can scale the application workload across multiple
secondaries to achieve the optimal end-user performance. The tradeoffs of this option are:
Read-write connections between the application instances and database have varying latency and cost
Application performance is impacted during the outage
Application instances are required to dynamically change the SQL connection string after database failover.

NOTE
A similar approach can be used to offload specialized workloads such as reporting jobs, business intelligence tools, or
backups. Typically these workloads consume significant database resources therefore it is recommended that you designate
one of the secondary databases for them with the performance level matched to the anticipated workload.

Design pattern 3: Active-passive deployment for data preservation

This option is best suited for applications with the following characteristics:
Any data loss is high business risk. The database failover can only be used as a last resort if the outage is
permanent.
The application can operate in "read-only mode" for a period of time.
In this pattern, the application switches to read-only mode when connected to the secondary database. The
application logic in the primary region is co-located with the primary database and operates in read-write mode
(RW). The application logic in the secondary region is co-located with the secondary database and is ready to
operate in read-only mode (RO). Traffic manager should be set up to use failover routing with end-point
monitoring enabled for both application instances.
The following diagram illustrates this configuration before an outage.
When the traffic manager detects a connectivity failure to the primary region, it automatically switches user traffic
to the application instance in the secondary region. With this pattern, it is important that you do not initiate
database failover after the outage is detected. The application in the secondary region is activated and operates in
read-only mode using the secondary database. This is illustrated by the following diagram.
Once the outage in the primary region is mitigated, traffic manager detects the restoration of connectivity in the
primary region and switches user traffic back to the application instance in the primary region. That application
instance resumes and operates in read-write mode using the primary database.

NOTE
Because this pattern requires read-only access to the secondary it requires Active Geo-Replication.

In case of an outage in the secondary region, the traffic manager marks the application end-point in the primary
region as degraded and the replication channel is suspended. However, this outage does not impact the
application's performance during the outage. Once the outage is mitigated, the secondary database is
immediately synchronized with the primary. During synchronization performance of the primary could be slightly
impacted depending on the amount of data that needs to be synchronized.
This design pattern has several advantages:
It avoids data loss during the temporary outages.
It does not require you to deploy a monitoring application as the recovery is triggered by traffic manager.
Downtime depends only on how quickly traffic manager detects the connectivity failure, which is configurable.
The tradeoffs are:
Application must be able to operate in read-only mode.
It is required to dynamically switch to it when it is connected to a read-only database.
Resumption of read-write operations requires recovery of the primary region, which means full data access
may be disabled for hours or even days.

NOTE
In case of a permanent service outage in the region, you must manually activate database failover and accept the data loss.
The application will be functional in the secondary region with read-write access to the database.

Business continuity planning: Choose an application design for cloud

disaster recovery
Your specific cloud disaster recovery strategy can combine or extend these design patterns to best meet the needs
of your application. As mentioned earlier, the strategy you choose is based on the SLA you want to offer to your
customers and the application deployment topology. To help guide your decision, the following table compares
the choices based on the estimated data loss or recovery point objective (RPO) and estimated recovery time (ERT).
PATTERN RPO ERT

Active-passive deployment for disaster Read-write access < 5 sec Failure detection time + failover API call
recovery with co-located database + application verification test
access

Active-active deployment for Read-write access < 5 sec Failure detection time + failover API call
application load balancing + SQL connection string change +
application verification test

Active-passive deployment for data Read-only access < 5 sec Read-write Read-only access = connectivity failure
preservation access = zero detection time + application verification
test
Read-write access = time to mitigate
the outage

Next steps
To learn about Azure SQL Database automated backups, see SQL Database automated backups
For a business continuity overview and scenarios, see Business continuity overview
To learn about using automated backups for recovery, see restore a database from the service-initiated
backups
To learn about faster recovery options, see Active-Geo-Replication
To learn about using automated backups for archiving, see database copy
For information about Active Geo-Replication with Elastic Pools, see Elastic Pool disaster recovery strategies.
Disaster recovery strategies for applications using
SQL Database elastic pools
4/13/2017 • 14 min to read • Edit Online

Over the years we have learned that cloud services are not foolproof and catastrophic incidents can and will
happen. SQL Database provides a number of capabilities to provide for the business continuity of your application
when these incidents occur. Elastic pools and single databases support the same kind of disaster recovery
capabilities. This article describes several DR strategies for elastic pools that leverage these SQL Database business
continuity features.
For the purposes of this article we will use the canonical SaaS ISV application pattern:
A modern cloud based web application provisions one SQL database for each end user. The ISV has a large
number of customers and therefore uses many databases, known as tenant databases. Because the tenant
databases typically have unpredictable activity patterns, the ISV uses an elastic pool to make the database cost
very predictable over extended periods of time. The elastic pool also simplifies the performance management
when the user activity spikes. In addition to the tenant databases the application also uses several databases to
manage user profiles, security, collect usage patterns etc. Availability of the individual tenants does not impact the
application’s availability as whole. However, the availability and performance of management databases is
critical for the application’s function and if the management databases are offline the entire application is offline.
In the rest of the paper we will discuss the DR strategies covering a range of scenarios from the cost sensitive
startup applications to the ones with stringent availability requirements.

Scenario 1. Cost sensitive startup

I am a startup business and am extremely cost sensitive. I want to simplify deployment and management of the
application and I am willing to have a limited SLA for individual customers. But I want to ensure the application as
a whole is never offline.
To satisfy the simplicity requirement, you should deploy all tenant databases into one elastic pool in the Azure
region of your choice and deploy the management database(s) as geo-replicated single database(s). For the
disaster recovery of tenants, use geo-restore, which comes at no additional cost. To ensure the availability of the
management databases, they should be geo-replicated to another region (step 1). The ongoing cost of the disaster
recovery configuration in this scenario is equal to the total cost of the secondary database(s). This configuration is
illustrated on the next diagram.
In case of an outage in the primary region, the recovery steps to bring your application online are illustrated by
the next diagram.
Immediately failover the management databases (2) to the DR region.
Change the the application's connection string to point to the DR region. All new accounts and tenant
databases will be created in the DR region. The existing customers will see their data temporarily unavailable.
Create the elastic pool with the same configuration as the original pool (3).
Use geo-restore to create copies of the tenant databases (4). You can consider triggering the individual restores
by the end-user connections or use some other application specific priority scheme.
At this point your application is back online in the DR region, but some customers will experience delay when
accessing their data.

If the outage was temporary, it is possible that the primary region will be recovered by Azure before all the
restores are complete in the DR region. In this case, you should orchestrate moving the application back to the
primary region. The process will take the steps illustrated on the next diagram.
Cancel all outstanding geo-restore requests.
Failover the management database(s) to the primary region (5). Note: After the region’s recovery the old
primaries have automatically become secondaries. Now they will switch roles again.
Change the application's connection string to point back to the primary region. Now all new accounts and
tenant databases will be created in the primary region. Some existing customers will see their data temporarily
unavailable.
Set all databases in the DR pool to read-only to ensure they cannot be modified in the DR region (6).
For each database in the DR pool that has changed since the recovery, rename or delete the corresponding
databases in the primary pool (7).
Copy the updated databases from the DR pool to the primary pool (8).
Delete the DR pool (9)
At this point your application will be online in the primary region with all tenant databases available in the primary
pool.

The key benefit of this strategy is low ongoing cost for data tier redundancy. Backups are taken automatically by
the SQL Database service with no application rewrite and at no additional cost. The cost is incurred only when the
elastic databases are restored. The trade-off is that the complete recovery of all tenant databases will take
significant time. It will depend on the total number of restores you will initiate in the DR region and overall size of
the tenant databases. Even if you prioritize some tenants' restores over others, you will be competing with all the
other restores that are initiated in the same region as the service will arbitrate and throttle to minimize the overall
impact on the existing customers' databases. In addition, the recovery of the tenant databases cannot start until
the new elastic pool in the DR region is created.

Scenario 2. Mature application with tiered service

I am a mature SaaS application with tiered service offers and different SLAs for trial customers and for paying
customers. For the trial customers, I have to reduce the cost as much as possible. Trial customers can take
downtime but I want to reduce its likelihood. For the paying customers, any downtime is a flight risk. So I want to
make sure that paying customers are always able to access their data.
To support this scenario, you should separate the trial tenants from paid tenants by putting them into separate
elastic pools. The trial customers would have lower eDTU per tenant and lower SLA with a longer recovery time.
The paying customers would be in a pool with higher eDTU per tenant and a higher SLA. To guarantee the lowest
recovery time, the paying customers' tenant databases should be geo-replicated. This configuration is illustrated
on the next diagram.

As in the first scenario, the management database(s) will be quite active so you use a single geo-replicated
database for it (1). This will ensure the predictable performance for new customer subscriptions, profile updates
and other management operations. The region in which the primaries of the management database(s) reside will
be the primary region and the region in which the secondaries of the management database(s) reside will be the
DR region.
The paying customers’ tenant databases will have active databases in the “paid” pool provisioned in the primary
region. You should provision a secondary pool with the same name in the DR region. Each tenant would be geo-
replicated to the secondary pool (2). This will enable a quick recovery of all tenant databases using failover.
If an outage occurs in the primary region, the recovery steps to bring your application online are illustrated in the
next diagram:

Immediately fail over the management database(s) to the DR region (3).

Change the application’s connection string to point to the DR region. Now all new accounts and tenant
databases will be created in the DR region. The existing trial customers will see their data temporarily
unavailable.
Failover the paid tenant's databases to the pool in the DR region to immediately restore their availability (4).
Since the failover is a quick metadata level change you may consider an optimization where the individual
failovers are triggered on demand by the end user connections.
If your secondary pool eDTU size was lower than the primary because the secondary databases only required
the capacity to process the change logs while they were secondaries, you should immediately increase the pool
capacity now to accommodate the full workload of all tenants (5).
Create the new elastic pool with the same name and the same configuration in the DR region for the trial
customers' databases (6).
Once the trial customers’ pool is created, use geo-restore to restore the individual trial tenant databases into
the new pool (7). You can consider triggering the individual restores by the end-user connections or use some
other application specific priority scheme.
At this point your application is back online in the DR region. All paying customers have access to their data while
the trial customers will experience delay when accessing their data.
When the primary region is recovered by Azure after you have restored the application in the DR region you can
continue running the application in that region or you can decide to fail back to the primary region. If the primary
region is recovered before the failover process is completed, you should consider failing back right away. The
failback will take the steps illustrated in the next diagram:

Cancel all outstanding geo-restore requests.

Failover the management database(s) (8). After the region’s recovery the old primary had automatically
become the secondary. Now it becomes the primary again.
Failover the paid tenant databases (9). Similarly, after the region’s recovery the old primaries automatically
become the secondaries. Now they will become the primaries again.
Set the restored trial databases that have changed in the DR region to read-only (10).
For each database in the trial customers DR pool that changed since the recovery, rename or delete the
corresponding database in the trial customers primary pool (11).
Copy the updated databases from the DR pool to the primary pool (12).
Delete the DR pool (13)
NOTE
The failover operation is asynchronous. To minimize the recovery time it is important that you execute the tenant databases'
failover command in batches of at least 20 databases.

The key benefit of this strategy is that it provides the highest SLA for the paying customers. It also guarantees
that the new trials are unblocked as soon as the trial DR pool is created. The trade-off is that this setup will
increase the total cost of the tenant databases by the cost of the secondary DR pool for paid customers. In
addition, if the secondary pool has a different size, the paying customers will experience lower performance after
failover until the pool upgrade in the DR region is completed.

Scenario 3. Geographically distributed application with tiered service

I have a mature SaaS application with tiered service offers. I want to offer a very aggressive SLA to my paid
customers and minimize the risk of impact when outages occur because even brief interruption can cause
customer dissatisfaction. It is critical that the paying customers can always access their data. The trials are free and
an SLA is not offered during the trial period.
To support this scenario, you should have three separate elastic pools. Two equal size pools with high eDTUs per
database should be provisioned in two different regions to contain the paid customers' tenant databases. The third
pool containing the trial tenants would have a lower eDTUs per database and be provisioned in one of the two
region.
To guarantee the lowest recovery time during outages the paying customers' tenant databases should be geo-
replicated with 50% of the primary databases in each of the two regions. Similarly, each region would have 50% of
the secondary databases. This way if a region is offline only 50% of the paid customers' databases would be
impacted and would have to failover. The other databases would remain intact. This configuration is illustrated in
the following diagram:

As in the previous scenarios, the management database(s) will be quite active so you should configure them as
single geo-replicated database(s) (1). This will ensure the predictable performance of the new customer
subscriptions, profile updates and other management operations. Region A would be the primary region for the
management database(s) and the region B will be used for recovery of the management database(s).
The paying customers’ tenant databases will be also geo-replicated but with primaries and secondaries split
between region A and region B (2). This way the tenant primary databases impacted by the outage can failover to
the other region and become available. The other half of the tenant databases will not be impacted at all.
The next diagram illustrates the recovery steps to take if an outage occurs in region A.

Immediately fail over the management databases to region B (3).

Change the application’s connection string to point to the management database(s) in region B. Modify the
management database(s) to make sure the new accounts and tenant databases will be created in region B and
the existing tenant databases will be found there as well. The existing trial customers will see their data
temporarily unavailable.
Failover the paid tenant's databases to pool 2 in region B to immediately restore their availability (4). Since the
failover is a quick metadata level change you may consider an optimization where the individual failovers are
triggered on demand by the end user connections.
Since now pool 2 contains only primary databases the total workload in the pool will increase so you should
immediately increase its eDTU size (5).
Create the new elastic pool with the same name and the same configuration in the region B for the trial
customers' databases (6).
Once the pool is created use geo-restore to restore the individual trial tenant database into the pool (7). You
can consider triggering the individual restores by the end-user connections or use some other application
specific priority scheme.

NOTE
The failover operation is asynchronous. To minimize the recovery time it is important that you execute the tenant databases'
failover command in batches of at least 20 databases.

At this point your application is back online in region B. All paying customers have access to their data while the
trial customers will experience delay when accessing their data.
When region A is recovered you need to decide if you want to use region B for trial customers or failback to using
the trial customers pool in region A. One criteria could be the % of trial tenant databases modified since the
recovery. Regardless of that decision you will need to re-balance the paid tenants between two pools. the next
diagram illustrates the process when the trial tenant databases fail back to region A.
Cancel all outstanding geo-restore requests to trial DR pool.
Failover the management database (8). After the region’s recovery, the old primary had automatically became
the secondary. Now it becomes the primary again.
Select which paid tenant databases will fail back to pool 1 and initiate failover to their secondaries (9). After the
region’s recovery all databases in pool 1 automatically became secondaries. Now 50% of them will become
primaries again.
Reduce the size of pool 2 to the original eDTU (10).
Set all restored trial databases in the region B to read-only (11).
For each database in the trial DR pool that has changed since the recovery rename or delete the corresponding
database in the trial primary pool (12).
Copy the updated databases from the DR pool to the primary pool (13).
Delete the DR pool (14)
The key benefits of this strategy are:
It supports the most aggressive SLA for the paying customers because it ensures that an outage cannot impact
more than 50% of the tenant databases.
It guarantees that the new trials are unblocked as soon as the trail DR pool is created during the recovery.
It allows more efficient use of the pool capacity as 50% of secondary databases in pool 1 and pool 2 are
guaranteed to be less active then the primary databases.
The main trade-offs are:
The CRUD operations against the management database(s) will have lower latency for the end users connected
to region A than for the end users connected to region B as they will be executed against the primary of the
management database(s).
It requires more complex design of the management database. For example, each tenant record would have to
have a location tag that needs to be changed during failover and failback.
The paying customers may experience lower performance than usual until the pool upgrade in region B is
completed.

Summary
This article focuses on the disaster recovery strategies for the database tier used by a SaaS ISV multi-tenant
application. The strategy you choose should be based on the needs of the application, such as the business model,
the SLA you want to offer to your customers, budget constraint etc.. Each described strategy outlines the benefits
and trade-off so you could make an informed decision. Also, your specific application will likely include other
Azure components. So you should review their business continuity guidance and orchestrate the recovery of the
database tier with them. To learn more about managing recovery of database applications in Azure, refer to
Designing cloud solutions for disaster recovery .

Next steps
To learn about Azure SQL Database automated backups, see SQL Database automated backups
For a business continuity overview and scenarios, see Business continuity overview
To learn about using automated backups for recovery, see restore a database from the service-initiated
backups
To learn about faster recovery options, see Active-Geo-Replication
To learn about using automated backups for archiving, see database copy
Managing rolling upgrades of cloud applications
using SQL Database Active Geo-Replication
1/23/2017 • 8 min to read • Edit Online

NOTE
Active Geo-Replication is now available for all databases in all tiers.

Learn how to use Geo-Replication in SQL Database to enable rolling upgrades of your cloud application. Because
upgrade is a disruptive operation, it should be part of your business continuity planning and design. In this article
we look at two different methods of orchestrating the upgrade process, and discuss the benefits and trade-offs of
each option. For the purposes of this article we will use a simple application that consists of a web site connected
to a single database as its data tier. Our goal is to upgrade version 1 of the application to version 2 without any
significant impact on the end user experience.
When evaluating the upgrade options you should consider the following factors:
Impact on application availability during upgrades. How long the application function may be limited or
degraded.
Ability to roll back in case of an upgrade failure.
Vulnerability of the application if an unrelated catastrophic failure occurs during the upgrade.
Total dollar cost. This includes additional redundancy and incremental costs of the temporary components used
by the upgrade process.

Upgrading applications that rely on database backups for disaster

recovery
If your application relies on automatic database backups and uses geo-restore for disaster recovery, it is usually
deployed to a single Azure region. In this case the upgrade process involves creating a backup deployment of all
application components involved in the upgrade. To minimize the end-user disruption you will leverage Azure
Traffic Manager (WATM) with the failover profile. The following diagram illustrates the operational environment
prior to the upgrade process. The endpoint contoso-1.azurewebsites.net represents a production slot of the
application that needs to be upgraded. To enable the ability to rollback the upgrade, you need create a stage slot
with a fully synchronized copy of the application. The following steps are required to prepare the application for
the upgrade:
1. Create a stage slot for the upgrade. To do that create a secondary database (1) and deploy a identical web site in
the same Azure region. Monitor the secondary to see if the seeding process is completed.
2. Create a failover profile in WATM with contoso-1.azurewebsites.net as online endpoint and
contoso-2.azurewebsites.net as offline.

NOTE
Note the preparation steps will not impact the application in the production slot and it can function in full access mode.
Once the preparation steps are completed the application is ready for the actual upgrade. The following diagram
illustrates the steps involved in the upgrade process.
1. Set the primary database in the production slot to read-only mode (3). This will guarantee that the production
instance of the application (V1) will remain read-only during the upgrade thus preventing the data divergence
between the V1 and V2 database instances.
2. Disconnect the secondary database using the planned termination mode (4). It will create a fully synchronized
independent copy of the primary database. This database will be upgraded.
3. Turn the primary database to read-write mode and run the upgrade script in the stage slot (5).
If the upgrade completed successfully you are now ready to switch the end users to the staged copy the
application. It will now become the production slot of the application. This involves a few more steps as illustrated
on the following diagram.
1. Switch the online endpoint in the WATM profile to contoso-2.azurewebsites.net, which points to the V2 version
of the web site (6). It now becomes the production slot with the V2 application and the end user traffic is
directed to it.
2. If you no longer need the V1 application components so you can safely remove them (7).
If the upgrade process is unsuccessful, for example due to an error in the upgrade script, the stage slot should be
considered compromised. To rollback the application to the pre-upgrade state you simply revert the application in
the production slot to full access. The steps involved are shown on the next diagram.
1. Set the database copy to read-write mode (8). This will restore the full V1 functionally in the production slot.
2. Perform the root cause analysis and remove the compromised components in the stage slot (9).
At this point the application is fully functional and the upgrade steps can be repeated.

NOTE
The rollback does not require changes in WATM profile as it already points to contoso-1.azurewebsites.net as the active
endpoint.
The key advantage of this option is that you can upgrade a application in a single region using a set of simple
steps. The dollar cost of the upgrade is relatively low. The main tradeoff is that if a catastrophic failure occurs
during the upgrade the recovery to the pre-upgrade state will involve re-deployment of the application in a
different region and restoring the database from backup using geo-restore. This process will result in significant
downtime.

Upgrading applications that rely on database Geo-Replication for

disaster recovery
If your application leverages Geo-Replication for business continuity, it is deployed to at least two different regions
with an active deployment in Primary region and a standby deployment in Backup region. In addition to the factors
mentioned earlier, the upgrade process must guarantee that:
The application remains protected from catastrophic failures at all times during the upgrade process
The geo-redundant components of the application are upgraded in parallel with the active components
To achieve these goals you will leverage Azure Traffic Manager (WATM) using the failover profile with one active
and three backup endpoints. The following diagram illustrates the operational environment prior to the upgrade
process. The web sites contoso-1.azurewebsites.net and contoso-dr.azurewebsites.net represent a production slot
of the application with full geographic redundancy. To enable the ability to rollback the upgrade, you need create a
stage slot with a fully synchronized copy of the application. Because you you need to ensure that the application
can quickly recover in case a catastrophic failure occurs during the upgrade process the stage slot needs to be geo-
redundant as well. The following steps are required to prepare the application for the upgrade:
1. Create a stage slot for the upgrade. To do that create a secondary database (1) and deploy a identical copy of
the web site in the same Azure region . Monitor the secondary to see if the seeding process is completed.
2. Create a geo-redundant secondary database in the stage slot by geo-replicating the secondary database to the
backup region (this is called "chained geo-replication"). Monitor the backup secondary to see if the seeding
process is completed (3).
3. Create a standby copy of the web site in the backup region and link it to the geo-redundant secondary (4).
4. Add the additional endpoints contoso-2.azurewebsites.net and contoso-3.azurewebsites.net to the failover
profile in WATM as offline endpoints (5).

NOTE
Note the preparation steps will not impact the application in the production slot and it can function in full access mode.

Once the preparation steps are completed, the stage slot is ready for the upgrade. The following diagram illustrates
the upgrade steps.
1. Set the primary database in the production slot to read-only mode (6). This will guarantee that the production
instance of the application (V1) will remain read-only during the upgrade thus preventing the data divergence
between the V1 and V2 database instances.
2. Disconnect the secondary database in the same region using the planned termination mode (7). It will create a
fully synchronized independent copy of the primary database, which will automatically become a primary after
the termination. This database will be upgraded.
3. Turn the primary database in the stage slot to read-write mode and run the upgrade script (8).

If the upgrade completed successfully you are now ready to switch the end users to the V2 version of the
application. The following diagram illustrates the steps involved.
1. Switch the active endpoint in the WATM profile to contoso-2.azurewebsites.net, which now points to the V2
version of the web site (9). It now becomes a production slot with the V2 application and end user traffic is
directed to it.
2. If you no longer need the V1 application so you can safely remove it (10 and 11).
If the upgrade process is unsuccessful, for example due to an error in the upgrade script, the stage slot should be
considered compromised. To rollback the application to the pre-upgrade state you simply revert to using the
application in the production slot with full access. The steps involved are shown on the next diagram.
1. Set the primary database copy in the production slot to read-write mode (12). This will restore the full V1
functionally in the production slot.
2. Perform the root cause analysis and remove the compromised components in the stage slot (13 and 14).
At this point the application is fully functional and the upgrade steps can be repeated.

NOTE
The rollback does not require changes in WATM profile as it already points to contoso-1.azurewebsites.net as the active
endpoint.
The key advantage of this option is that you can upgrade both the application and its geo-redundant copy in
parallel without compromising your business continuity during the upgrade. The main tradeoff is that it requires
double redundancy of each application component and therefore incurs higher dollar cost. It also involves a more
complicated workflow.

Summary
The two upgrade methods described in the article differ in complexity and the dollar cost but they both focus on
minimizing the time when the end user is limited to read-only operations. That time is directly defined by the
duration of the upgrade script. It does not depend on the database size, the service tier you chose, the web site
configuration and other factors that you cannot easily control. This is because all the preparation steps are
decoupled from the upgrade steps and can be done without impacting the production application. The efficiency of
the upgrade script is the key factor that determines the end-user experience during upgrades. So the best way you
can improve it is by focusing your efforts on making the upgrade script as efficient as possible.
Next steps
For a business continuity overview and scenarios, see Business continuity overview
To learn about Azure SQL Database automated backups, see SQL Database automated backups
To learn about using automated backups for recovery, see restore a database from automated backups
To learn about faster recovery options, see Active-Geo-Replication
To learn about using automated backups for archiving, see database copy

Additionale Resources
The following pages will help you learn about the specific operations required to implement the upgrade workflow:
Add secondary database
Failover database to secondary
Disconnect Geo-Replication secondary
Geo-restore database
Drop database
Copy database
Set database to read-only or read-write mode
SQL Database Application Development Overview
4/14/2017 • 2 min to read • Edit Online

This article walks through the basic considerations that a developer should be aware of when writing code to
connect to Azure SQL Database.

TIP
For a tutorial showing you how to create a server, create a server-based firewall, view server properties, connect using SQL
Server Management Studio, query the master database, create a sample database and a blank database, query database
properties, connect using SQL Server Management Studio, and query the sample database, see Get Started Tutorial.

Language and platform

There are code samples available for various programming languages and platforms. You can find links to the
code samples at:
More Information: Connection libraries for SQL Database and SQL Server

Tools
You can leverage open source tools like cheetah, sql-cli, VS Code. Additionally, Azure SQL Database works with
Microsoft tools like Visual Studio and SQL Server Management Studio. You can also use the Azure Management
Portal, PowerShell, and REST APIs help you gain additional productivity.

Resource limitations
Azure SQL Database manages the resources available to a database using two different mechanisms: Resources
Governance and Enforcement of Limits.
More Information: Azure SQL Database resource limits

Security
Azure SQL Database provides resources for limiting access, protecting data, and monitoring activities on a SQL
Database.
More Information: Securing your SQL Database

Authentication
Azure SQL Database supports both SQL Server authentication users and logins, as well as Azure Active
Directory authentication users and logins.
You need to specify a particular database, instead of defaulting to the master database.
You cannot use the Transact-SQL USE myDatabaseName; statement on SQL Database to switch to another
database.
More information: SQL Database security: Manage database access and login security

Resiliency
When a transient error occurs while connecting to SQL Database, your code should retry the call. We recommend
that retry logic use backoff logic, so that it does not overwhelm the SQL Database with multiple clients retrying
simultaneously.
Code samples: For code samples that illustrate retry logic, see samples for the language of your choice at:
Connection libraries for SQL Database and SQL Server
More information: Error messages for SQL Database client programs

Managing Connections
In your client connection logic, override the default timeout to be 30 seconds. The default of 15 seconds is too
short for connections that depend on the internet.
If you are using a connection pool, be sure to close the connection the instant your program is not actively
using it, and is not preparing to reuse it.

Network Considerations
On the computer that hosts your client program, ensure the firewall allows outgoing TCP communication on
port 1433. More information: Configure an Azure SQL Database firewall
If your client program connects to SQL Database while your client runs on an Azure virtual machine (VM), you
must open certain port ranges on the VM. More information: Ports beyond 1433 for ADO.NET 4.5 and SQL
Database
Client connections to Azure SQL Database sometimes bypass the proxy and interact directly with the database.
Ports other than 1433 become important. More information: Ports beyond 1433 for ADO.NET 4.5 and SQL
Database

Data Sharding with Elastic Scale

Elastic Scale simplifies the process of scaling out (and in).
Design Patterns for Multi-tenant SaaS Applications with Azure SQL Database
Data dependent routing
Get Started with Azure SQL Database Elastic Scale Preview

Next steps
Explore all the capabilities of SQL Database
Connection libraries for SQL Database and SQL
Server
4/14/2017 • 1 min to read • Edit Online

This topic lists each library or driver that client programs can use when connecting to Azure SQL Database or to
Microsoft SQL Server.

LANGUAGE PLATFORM ADDITIONAL RESOURCES DOWNLOAD

ADO.NET Windows, Linux, Mac Microsoft ADO.NET for SQL Download

Server

Java Windows, Linux, Mac Microsoft JDBC Driver for Download

SQL Server

PHP Windows Microsoft PHP Driver for Download

SQL Server

Node.js Windows, Linux, Mac Node.js Driver for SQL Install

Server

Python Windows, Linux, Mac Python SQL Driver Install choices:

* pymssql
* pyodbc

Ruby Windows, Linux, Mac Ruby Driver for SQL Server Install

C/C++ Windows, Linux Microsoft ODBC Driver for Install choices:

SQL Server * Windows
* Linux

Related links
SQL Server Drivers, for connecting from a client
Connect to SQL Database by using .NET (C#)
Connect to SQL Database by using PHP
Connect to SQL Database by using Node.js
Connect to SQL Database by using Java
Connect to SQL Database by using Python
Connect to SQL Database by using Ruby
Design patterns for multitenant SaaS applications
and Azure SQL Database
4/12/2017 • 13 min to read • Edit Online

In this article, you can learn about the requirements and common data architecture patterns of multitenant
software as a service (SaaS) database applications that run in a cloud environment. It also explains the factors
you need to consider and the trade-offs of different design patterns. Elastic pools and elastic tools in Azure SQL
Database can help you meet your specific requirements without compromising other objectives.
Developers sometimes make choices that work against their long-term best interests when they design tenancy
models for the data tiers of multitenant applications. Initially, at least, a developer might perceive ease of
development and lower cloud service provider costs as more important than tenant isolation or the scalability of
an application. This choice can lead to customer satisfaction concerns and a costly course-correction later.
A multitenant application is an application hosted in a cloud environment and that provides the same set of
services to hundreds or thousands of tenants who do not share or see each other’s data. An example is an SaaS
application that provides services to tenants in a cloud-hosted environment.

Multitenant applications
In multitenant applications, data and workload can be easily partitioned. You can partition data and workload, for
example, along tenant boundaries, because most requests occur within the confines of a tenant. That property is
inherent in the data and the workload, and it favors the application patterns discussed in this article.
Developers use this type of application across the whole spectrum of cloud-based applications, including:
Partner database applications that are being transitioned to the cloud as SaaS applications
SaaS applications built for the cloud from the ground up
Direct, customer-facing applications
Employee-facing enterprise applications
SaaS applications that are designed for the cloud and those with roots as partner database applications typically
are multitenant applications. These SaaS applications deliver a specialized software application as a service to
their tenants. Tenants can access the application service and have full ownership of associated data stored as part
of the application. But to take advantage of the benefits of SaaS, tenants must surrender some control over their
own data. They trust the SaaS service provider to keep their data safe and isolated from other tenants’ data.
Examples of this kind of multitenant SaaS application are MYOB, SnelStart, and Salesforce.com. Each of these
applications can be partitioned along tenant boundaries and support the application design patterns we discuss
in this article.
Applications that provide a direct service to customers or to employees within an organization (often referred to
as users, rather than tenants) are another category on the multitenant application spectrum. Customers
subscribe to the service and do not own the data that the service provider collects and stores. Service providers
have less stringent requirements to keep their customers’ data isolated from each other beyond government-
mandated privacy regulations. Examples of this kind of customer-facing multitenant application are media
content providers like Netflix, Spotify, and Xbox LIVE. Other examples of easily partitionable applications are
customer-facing, Internet-scale applications, or Internet of Things (IoT) applications in which each customer or
device can serve as a partition. Partition boundaries can separate users and devices.
Not all applications partition easily along a single property such as tenant, customer, user, or device. A complex
enterprise resource planning (ERP) application, for example, has products, orders, and customers. It usually has a
complex schema with thousands of highly interconnected tables.
No single partition strategy can apply to all tables and work across an application's workload. This article focuses
on multitenant applications that have easily partitionable data and workloads.

Multitenant application design trade-offs

The design pattern that a multitenant application developer chooses typically is based on a consideration of the
following factors:
Tenant isolation. The developer needs to ensure that no tenant has unwanted access to other tenants’ data.
The isolation requirement extends to other properties, such as providing protection from noisy neighbors,
being able to restore a tenant’s data, and implementing tenant-specific customizations.
Cloud resource cost. An SaaS application needs to be cost-competitive. A multitenant application developer
might choose to optimize for lower cost in the use of cloud resources, such as storage and compute costs.
Ease of DevOps. A multitenant application developer needs to incorporate isolation protection, maintain, and
monitor the health of their application and database schema, and troubleshoot tenant issues. Complexity in
application development and operation translates directly to increased cost and challenges with tenant
satisfaction.
Scalability. The ability to incrementally add more tenants and capacity for tenants who require it is
imperative to a successful SaaS operation.
Each of these factors has trade-offs compared to another. The lowest-cost cloud offering might not offer the
most convenient development experience. It’s important for a developer to make informed choices about these
options and their trade-offs during the application design process.
A popular development pattern is to pack multiple tenants into one or a few databases. The benefits of this
approach are a lower cost because you pay for a few databases, and the relative simplicity of working with a
limited number of databases. But over time, a SaaS multitenant application developer will realize that this choice
has substantial downsides in tenant isolation and scalability. If tenant isolation becomes important, additional
effort is required to protect tenant data in shared storage from unauthorized access or noisy neighbors. This
additional effort might significantly boost development efforts and isolation maintenance costs. Similarly, if
adding tenants is required, this design pattern typically requires expertise to redistribute tenant data across
databases to properly scale the data tier of an application.
Tenant isolation often is a fundamental requirement in SaaS multitenant applications that cater to businesses
and organizations. A developer might be tempted by perceived advantages in simplicity and cost over tenant
isolation and scalability. This trade-off can prove complex and expensive as the service grows and tenant
isolation requirements become more important and managed at the application layer. However, in multitenant
applications that provide a direct, consumer-facing service to customers, tenant isolation might be a lower
priority than optimizing for cloud resource cost.

Multitenant data models

Common design practices for placing tenant data follow three distinct models, shown in Figure 1.
Figure 1: Common design practices for multitenant data models
Database-per-tenant. Each tenant has its own database. All tenant-specific data is confined to the tenant’s
database and isolated from other tenants and their data.
Shared database-sharded. Multiple tenants share one of multiple databases. A distinct set of tenants is
assigned to each database by using a partitioning strategy such as hash, range, or list partitioning. This data
distribution strategy often is referred to as sharding.
Shared database-single. A single, sometimes large, database contains data for all tenants, which are
disambiguated in a tenant ID column.

NOTE
An application developer might choose to place different tenants in different database schemas, and then use the schema
name to disambiguate the different tenants. We do not recommend this approach because it usually requires the use of
dynamic SQL, and it can’t be effective in plan caching. In the remainder of this article, we focus on the shared table
approach for this category of multitenant application.

Popular multitenant data models

It’s important to evaluate the different types of multitenant data models in terms of the application design trade-
offs we’ve already identified. These factors help characterize the three most common multitenant data models
described earlier and their database usage as shown in Figure 2.
Isolation. The degree of isolation between tenants can be a measure of how much tenant isolation a data
model achieves.
Cloud resource cost. The amount of resource sharing between tenants can optimize cloud resource cost. A
resource can be defined as the compute and storage cost.
DevOps cost. The ease of application development, deployment, and manageability reduces overall SaaS
operation cost.
In Figure 2, the Y axis shows the level of tenant isolation. The X axis shows the level of resource sharing. The gray,
diagonal arrow in the middle indicates the direction of DevOps costs, tending to increase or decrease.
Figure 2: Popular multitenant data models
The lower-right quadrant in Figure 2 shows an application pattern that uses a potentially large, shared single
database, and the shared table (or separate schema) approach. It's good for resource sharing because all tenants
use the same database resources (CPU, memory, input/output) in a single database. But tenant isolation is
limited. You might need to take additional steps to protect tenants from each other at the application layer. These
additional steps can significantly increase the DevOps cost of developing and managing the application.
Scalability is limited by the scale of the hardware that hosts the database.
The lower-left quadrant in Figure 2 illustrates multiple tenants sharded across multiple databases (typically,
different hardware scale units). Each database hosts a subset of tenants, which addresses the scalability concern
of other patterns. If more capacity is required for more tenants, you can easily place the tenants on new
databases allocated to new hardware scale units. However, the amount of resource sharing is reduced. Only
tenants placed on the same scale units share resources. This approach provides little improvement to tenant
isolation because many tenants are still collocated without being automatically protected from each other’s
actions. Application complexity remains high.
The upper-left quadrant in Figure 2 is the third approach. It places each tenant’s data in its own database. This
approach has good tenant-isolation properties but limits resource sharing when each database has its own
dedicated resources. This approach is good if all tenants have predictable workloads. If tenant workloads become
less predictable, the provider cannot optimize resource sharing. Unpredictability is common for SaaS
applications. The provider must either over-provision to meet demands or lower resources. Either action results
in either higher costs or lower tenant satisfaction. A higher degree of resource sharing across tenants becomes
desirable to make the solution more cost-effective. Increasing the number of databases also increases DevOps
cost to deploy and maintain the application. Despite these concerns, this method provides the best and easiest
isolation for tenants.
These factors also influence the design pattern a customer chooses:
Ownership of tenant data. An application in which tenants retain ownership of their own data favors the
pattern of a single database per tenant.
Scale. An application that targets hundreds of thousands or millions of tenants favors database sharing
approaches such as sharding. Isolation requirements still can pose challenges.
Value and business model. If an application’s per-tenant revenue if small (less than a dollar), isolation
requirements become less critical and a shared database makes sense. If per-tenant revenue is a few dollars
or more, a database-per-tenant model is more feasible. It might help reduce development costs.
Given the design trade-offs shown in Figure 2, an ideal multitenant model needs to incorporate good tenant
isolation properties with optimal resource sharing among tenants. This model fits in the category described in
the upper-right quadrant of Figure 2.

Multitenancy support in Azure SQL Database

Azure SQL Database supports all multitenant application patterns outlined in Figure 2. With elastic pools, it also
supports an application pattern that combines good resource sharing and the isolation benefits of the database-
per-tenant approach (see the upper-right quadrant in Figure 3). Elastic database tools and capabilities in SQL
Database help reduce the cost to develop and operate an application that has many databases (shown in the
shaded area in Figure 3). These tools can help you build and manage applications that use any of the multi-
database patterns.

Figure 3: Multitenant application patterns in Azure SQL Database

Database-per-tenant model with elastic pools and tools

Elastic pools in SQL Database combine tenant isolation with resource sharing among tenant databases to better
support the database-per-tenant approach. SQL Database is a data tier solution for SaaS providers who build
multitenant applications. The burden of resource sharing among tenants shifts from the application layer to the
database service layer. The complexity of managing and querying at scale across databases is simplified with
elastic jobs, elastic query, elastic transactions, and the elastic database client library.

APPLICATION REQUIREMENTS SQL DATABASE CAPABILITIES

Tenant isolation and resource sharing Elastic pools: Allocate a pool of SQL Database resources and
share the resources across various databases. Also, individual
databases can draw as much resources from the pool as
needed to accommodate capacity demand spikes due to
changes in tenant workloads. The elastic pool itself can be
scaled up or down as needed. Elastic pools also provide ease
of manageability and monitoring and troubleshooting at the
pool level.

Ease of DevOps across databases Elastic pools: As noted earlier.

Elastic query: Query across databases for reporting or cross-

tenant analysis.
APPLICATION REQUIREMENTS SQL DATABASE CAPABILITIES

Elastic jobs: Package and reliably deploy database

maintenance operations or database schema changes to
multiple databases.

Elastic transactions: Process changes to several databases in

an atomic and isolated way. Elastic transactions are needed
when applications need “all or nothing” guarantees over
several database operations.

Elastic database client library: Manage data distributions and

map tenants to databases.

Shared models
As described earlier, for most SaaS providers, a shared model approach might pose problems with tenant
isolation issues and complexities with application development and maintenance. However, for multitenant
applications that provide a service directly to consumers, tenant isolation requirements may not be as high a
priority as minimizing cost. They might be able to pack tenants in one or more databases at a high density to
reduce costs. Shared-database models using a single database or multiple sharded databases might result in
additional efficiencies in resource sharing and overall cost. Azure SQL Database provides some features that help
customers build isolation for improved security and management at scale in the data tier.

APPLICATION REQUIREMENTS SQL DATABASE CAPABILITIES

Security isolation features Row-level security

Database schema

Ease of DevOps across databases Elastic query

Elastic jobs

Elastic transactions

Elastic database client library

Elastic database split and merge

Summary
Tenant isolation requirements are important for most SaaS multitenant applications. The best option to provide
isolation leans heavily toward the database-per-tenant approach. The other two approaches require investments
in complex application layers that require skilled development staff to provide isolation, which significantly
increases cost and risk. If isolation requirements are not accounted for early in the service development,
retrofitting them can be even more costly in the first two models. The main drawbacks associated with the
database-per-tenant model are related to increased cloud resource costs due to reduced sharing, and
maintaining and managing many databases. SaaS application developers often struggle when they make these
trade-offs.
Although trade-offs might be major barriers with most cloud database service providers, Azure SQL Database
eliminates the barriers with its elastic pool and elastic database capabilities. SaaS developers can combine the
isolation characteristics of a database-per-tenant model and optimize resource sharing and the manageability
improvements of many databases by using elastic pools and associated tools.
Multitenant application providers who have no tenant isolation requirements and who can pack tenants in a
database at a high density might find that shared data models result in additional efficiency in resource sharing
and reduce overall cost. Azure SQL Database elastic database tools, sharding libraries, and security features help
SaaS providers build and manage multitenant applications.

Next steps
Get started with elastic database tools with a sample app that demonstrates the client library.
Create an elastic pool custom dashboard for SaaS with a sample app that uses elastic pools for a cost-effective,
scalable database solution.
Use the Azure SQL Database tools to migrate existing databases to scale out.
To create an elastic pool using the Azure portal, see create an elastic pool.
Learn how to monitor and manage an elastic pool.

Additional resources
What is an Azure elastic pool?
Scaling out with Azure SQL Database
Multitenant applications with elastic database tools and row-level security
Authentication in multitenant apps by using Azure Active Directory and OpenID Connect
Tailspin Surveys application

Questions and feature requests

For questions, find us in the SQL Database forum. Add a feature request in the SQL Database feedback forum.
Multi-tenant applications with elastic database tools
and row-level security
4/10/2017 • 11 min to read • Edit Online

Elastic database tools and row-level security (RLS) offer a powerful set of capabilities for flexibly and efficiently
scaling the data tier of a multi-tenant application with Azure SQL Database. See Design Patterns for Multi-tenant
SaaS Applications with Azure SQL Database for more information.
This article illustrates how to use these technologies together to build an application with a highly scalable data
tier that supports multi-tenant shards, using ADO.NET SqlClient and/or Entity Framework.
Elastic database tools enables developers to scale out the data tier of an application via industry-standard
sharding practices using a set of .NET libraries and Azure service templates. Managing shards with using the
Elastic Database Client Library helps automate and streamline many of the infrastructural tasks typically
associated with sharding.
Row-level security enables developers to store data for multiple tenants in the same database using security
policies to filter out rows that do not belong to the tenant executing a query. Centralizing access logic with RLS
inside the database, rather than in the application, simplifies maintenance and reduces the risk of error as an
application’s codebase grows.
Using these features together, an application can benefit from cost savings and efficiency gains by storing data for
multiple tenants in the same shard database. At the same time, an application still has the flexibility to offer
isolated, single-tenant shards for “premium” tenants who require stricter performance guarantees since multi-
tenant shards do not guarantee equal resource distribution among tenants.
In short, the elastic database client library’s data dependent routing APIs automatically connect tenants to the
correct shard database containing their sharding key (generally a “TenantId”). Once connected, an RLS security
policy within the database ensures that tenants can only access rows that contain their TenantId. It is assumed that
all tables contain a TenantId column to indicate which rows belong to each tenant.

Download the sample project

Prerequisites
Use Visual Studio (2012 or higher)
Create three Azure SQL Databases
Download sample project: Elastic DB Tools for Azure SQL - Multi-Tenant Shards
Fill in the information for your databases at the beginning of Program.cs
This project extends the one described in Elastic DB Tools for Azure SQL - Entity Framework Integration by adding
support for multi-tenant shard databases. It builds a simple console application for creating blogs and posts, with
four tenants and two multi-tenant shard databases as illustrated in the above diagram.
Build and run the application. This will bootstrap the elastic database tools’ shard map manager and run the
following tests:
1. Using Entity Framework and LINQ, create a new blog and then display all blogs for each tenant
2. Using ADO.NET SqlClient, display all blogs for a tenant
3. Try to insert a blog for the wrong tenant to verify that an error is thrown
Notice that because RLS has not yet been enabled in the shard databases, each of these tests reveals a problem:
tenants are able to see blogs that do not belong to them, and the application is not prevented from inserting a
blog for the wrong tenant. The remainder of this article describes how to resolve these problems by enforcing
tenant isolation with RLS. There are two steps:
1. Application tier: Modify the application code to always set the current TenantId in the SESSION_CONTEXT
after opening a connection. The sample project has already done this.
2. Data tier: Create an RLS security policy in each shard database to filter rows based on the TenantId stored in
SESSION_CONTEXT. You will need to do this for each of your shard databases, otherwise rows in multi-tenant
shards will not be filtered.

Step 1) Application tier: Set TenantId in the SESSION_CONTEXT

After connecting to a shard database using the elastic database client library’s data dependent routing APIs, the
application still needs to tell the database which TenantId is using that connection so that an RLS security policy
can filter out rows belonging to other tenants. The recommended way to pass this information is to store the
current TenantId for that connection in the SESSION_CONTEXT. (Note: You could alternatively use CONTEXT_INFO,
but SESSION_CONTEXT is a better option because it is easier to use, returns NULL by default, and supports key-
value pairs.)
Entity Framework
For applications using Entity Framework, the easiest approach is to set the SESSION_CONTEXT within the
ElasticScaleContext override described in Data Dependent Routing using EF DbContext. Before returning the
connection brokered through data dependent routing, simply create and execute a SqlCommand that sets
'TenantId' in the SESSION_CONTEXT to the shardingKey specified for that connection. This way, you only need to
write code once to set the SESSION_CONTEXT.
// ElasticScaleContext.cs
// ...
// C'tor for data dependent routing. This call will open a validated connection routed to the proper
// shard by the shard map manager. Note that the base class c'tor call will fail for an open connection
// if migrations need to be done and SQL credentials are used. This is the reason for the
// separation of c'tors into the DDR case (this c'tor) and the internal c'tor for new shards.
public ElasticScaleContext(ShardMap shardMap, T shardingKey, string connectionStr)
: base(OpenDDRConnection(shardMap, shardingKey, connectionStr), true /* contextOwnsConnection */)
{
}

public static SqlConnection OpenDDRConnection(ShardMap shardMap, T shardingKey, string connectionStr)

{
// No initialization
Database.SetInitializer<ElasticScaleContext<T>>(null);

// Ask shard map to broker a validated connection for the given key
SqlConnection conn = null;
try
{
conn = shardMap.OpenConnectionForKey(shardingKey, connectionStr, ConnectionOptions.Validate);

// Set TenantId in SESSION_CONTEXT to shardingKey to enable Row-Level Security filtering

SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = @"exec sp_set_session_context @key=N'TenantId', @value=@shardingKey";
cmd.Parameters.AddWithValue("@shardingKey", shardingKey);
cmd.ExecuteNonQuery();

return conn;
}
catch (Exception)
{
if (conn != null)
{
conn.Dispose();
}

throw;
}
}
// ...

Now the SESSION_CONTEXT is automatically set with the specified TenantId whenever ElasticScaleContext is
invoked:

// Program.cs
SqlDatabaseUtils.SqlRetryPolicy.ExecuteAction(() =>
{
using (var db = new ElasticScaleContext<int>(sharding.ShardMap, tenantId, connStrBldr.ConnectionString))
{
var query = from b in db.Blogs
orderby b.Name
select b;

Console.WriteLine("All blogs for TenantId {0}:", tenantId);

foreach (var item in query)
{
Console.WriteLine(item.Name);
}
}
});

ADO.NET SqlClient
For applications using ADO.NET SqlClient, the recommended approach is to create a wrapper function around
ShardMap.OpenConnectionForKey() that automatically sets 'TenantId' in the SESSION_CONTEXT to the correct
TenantId before returning a connection. To ensure that SESSION_CONTEXT is always set, you should only open
connections using this wrapper function.

// Program.cs
// ...

// Wrapper function for ShardMap.OpenConnectionForKey() that automatically sets SESSION_CONTEXT with the correct
// tenantId before returning a connection. As a best practice, you should only open connections using this
// method to ensure that SESSION_CONTEXT is always set before executing a query.
public static SqlConnection OpenConnectionForTenant(ShardMap shardMap, int tenantId, string connectionStr)
{
SqlConnection conn = null;
try
{
// Ask shard map to broker a validated connection for the given key
conn = shardMap.OpenConnectionForKey(tenantId, connectionStr, ConnectionOptions.Validate);

// Set TenantId in SESSION_CONTEXT to shardingKey to enable Row-Level Security filtering

return conn;
}
catch (Exception)
{
if (conn != null)
{
conn.Dispose();
}

throw;
}
}

// ...

// Example query via ADO.NET SqlClient

// If row-level security is enabled, only Tenant 4's blogs will be listed
SqlDatabaseUtils.SqlRetryPolicy.ExecuteAction(() =>
{
using (SqlConnection conn = OpenConnectionForTenant(sharding.ShardMap, tenantId4, connStrBldr.ConnectionString))
{
SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = @"SELECT * FROM Blogs";

Console.WriteLine("--\nAll blogs for TenantId {0} (using ADO.NET SqlClient):", tenantId4);

SqlDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
Console.WriteLine("{0}", reader["Name"]);
}
}
});

Step 2) Data tier: Create row-level security policy

Create a security policy to filter the rows each tenant can access
Now that the application is setting SESSION_CONTEXT with the current TenantId before querying, an RLS security
policy can filter queries and exclude rows that have a different TenantId.
RLS is implemented in T-SQL: a user-defined function defines the access logic, and a security policy binds this
function to any number of tables. For this project, the function will simply verify that the application (rather than
some other SQL user) is connected to the database, and that the 'TenantId' stored in the SESSION_CONTEXT
matches the TenantId of a given row. A filter predicate will allow rows that meet these conditions to pass through
the filter for SELECT, UPDATE, and DELETE queries; and a block predicate will prevent rows that violate these
conditions from being INSERTed or UPDATEd. If SESSION_CONTEXT has not been set, it will return NULL and no
rows will be visible or able to be inserted.
To enable RLS, execute the following T-SQL on all shards using either Visual Studio (SSDT), SSMS, or the
PowerShell script included in the project (or if you are using Elastic Database Jobs, you can use it to automate
execution of this T-SQL on all shards):

CREATE SCHEMA rls -- separate schema to organize RLS objects

CREATE FUNCTION rls.fn_tenantAccessPredicate(@TenantId int)

RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN SELECT 1 AS fn_accessResult
WHERE DATABASE_PRINCIPAL_ID() = DATABASE_PRINCIPAL_ID('dbo') -- the user in your application’s connection string (dbo is
only for demo purposes!)
AND CAST(SESSION_CONTEXT(N'TenantId') AS int) = @TenantId
GO

CREATE SECURITY POLICY rls.tenantAccessPolicy

ADD FILTER PREDICATE rls.fn_tenantAccessPredicate(TenantId) ON dbo.Blogs,
ADD BLOCK PREDICATE rls.fn_tenantAccessPredicate(TenantId) ON dbo.Blogs,
ADD FILTER PREDICATE rls.fn_tenantAccessPredicate(TenantId) ON dbo.Posts,
ADD BLOCK PREDICATE rls.fn_tenantAccessPredicate(TenantId) ON dbo.Posts
GO

TIP
For more complex projects that need to add the predicate on hundreds of tables, you can use a helper stored procedure
that automatically generates a security policy adding a predicate on all tables in a schema. See Apply Row-Level Security to
all tables - helper script (blog).

Now if you run the sample application again, tenants will able to see only rows that belong to them. In addition,
the application cannot insert rows that belong to tenants other than the one currently connected to the shard
database, and it cannot update visible rows to have a different TenantId. If the application attempts to do either, a
DbUpdateException will be raised.
If you add a new table later on, simply ALTER the security policy and add filter and block predicates on the new
table:

ALTER SECURITY POLICY rls.tenantAccessPolicy

ADD FILTER PREDICATE rls.fn_tenantAccessPredicate(TenantId) ON dbo.MyNewTable,
ADD BLOCK PREDICATE rls.fn_tenantAccessPredicate(TenantId) ON dbo.MyNewTable
GO

Add default constraints to automatically populate TenantId for INSERTs

You can put a default constraint on each table to automatically populate the TenantId with the value currently
stored in SESSION_CONTEXT when inserting rows. For example:
-- Create default constraints to auto-populate TenantId with the value of SESSION_CONTEXT for inserts
ALTER TABLE Blogs
ADD CONSTRAINT df_TenantId_Blogs
DEFAULT CAST(SESSION_CONTEXT(N'TenantId') AS int) FOR TenantId
GO

ALTER TABLE Posts

ADD CONSTRAINT df_TenantId_Posts
DEFAULT CAST(SESSION_CONTEXT(N'TenantId') AS int) FOR TenantId
GO

Now the application does not need to specify a TenantId when inserting rows:

SqlDatabaseUtils.SqlRetryPolicy.ExecuteAction(() =>
{
using (var db = new ElasticScaleContext<int>(sharding.ShardMap, tenantId, connStrBldr.ConnectionString))
{
var blog = new Blog { Name = name }; // default constraint sets TenantId automatically
db.Blogs.Add(blog);
db.SaveChanges();
}
});

NOTE
If you use default constraints for an Entity Framework project, it is recommended that you do NOT include the TenantId
column in your EF data model. This is because Entity Framework queries automatically supply default values which will
override the default constraints created in T-SQL that use SESSION_CONTEXT. To use default constraints in the sample
project, for instance, you should remove TenantId from DataClasses.cs (and run Add-Migration in the Package Manager
Console) and use T-SQL to ensure that the field only exists in the database tables. This way, EF will not automatically supply
incorrect default values when inserting data.

(Optional) Enable a "superuser" to access all rows

Some applications may want to create a "superuser" who can access all rows, for instance, in order to enable
reporting across all tenants on all shards, or to perform Split/Merge operations on shards that involve moving
tenant rows between databases. To enable this, you should create a new SQL user ("superuser" in this example) in
each shard database. Then alter the security policy with a new predicate function that allows this user to access all
rows:
-- New predicate function that adds superuser logic
CREATE FUNCTION rls.fn_tenantAccessPredicateWithSuperUser(@TenantId int)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN SELECT 1 AS fn_accessResult
WHERE
(
DATABASE_PRINCIPAL_ID() = DATABASE_PRINCIPAL_ID('dbo') -- note, should not be dbo!
AND CAST(SESSION_CONTEXT(N'TenantId') AS int) = @TenantId
)
OR
(
DATABASE_PRINCIPAL_ID() = DATABASE_PRINCIPAL_ID('superuser')
)
GO

-- Atomically swap in the new predicate function on each table

ALTER SECURITY POLICY rls.tenantAccessPolicy
ALTER FILTER PREDICATE rls.fn_tenantAccessPredicateWithSuperUser(TenantId) ON dbo.Blogs,
ALTER BLOCK PREDICATE rls.fn_tenantAccessPredicateWithSuperUser(TenantId) ON dbo.Blogs,
ALTER FILTER PREDICATE rls.fn_tenantAccessPredicateWithSuperUser(TenantId) ON dbo.Posts,
ALTER BLOCK PREDICATE rls.fn_tenantAccessPredicateWithSuperUser(TenantId) ON dbo.Posts
GO

Maintenance
Adding new shards: You must execute the T-SQL script to enable RLS on any new shards, otherwise queries
on these shards will not be filtered.
Adding new tables: You must add a filter and block predicate to the security policy on all shards whenever a
new table is created, otherwise queries on the new table will not be filtered. This can be automated using a DDL
trigger, as described in Apply Row-Level Security automatically to newly created tables (blog).

Summary
Elastic database tools and row-level security can be used together to scale out an application’s data tier with
support for both multi-tenant and single-tenant shards. Multi-tenant shards can be used to store data more
efficiently (particularly in cases where a large number of tenants have only a few rows of data), while single-tenant
shards can be used to support premium tenants with stricter performance and isolation requirements. For more
information, see Row-Level Security reference.

Additional resources
What is an Azure elastic pool?
Scaling out with Azure SQL Database
Design Patterns for Multi-tenant SaaS Applications with Azure SQL Database
Authentication in multitenant apps, using Azure AD and OpenID Connect
Tailspin Surveys application

Questions and Feature Requests

For questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Ports beyond 1433 for ADO.NET 4.5
4/10/2017 • 1 min to read • Edit Online

This topic describes the Azure SQL Database connection behavior for clients that use ADO.NET 4.5 or a later
version.

Outside vs inside
For connections to Azure SQL Database, we must first ask whether your client program runs outside or inside the
Azure cloud boundary. The subsections discuss two common scenarios.
Outside: Client runs on your desktop computer
Port 1433 is the only port that must be open on your desktop computer that hosts your SQL Database client
application.
Inside: Client runs on Azure
When your client runs inside the Azure cloud boundary, it uses what we can call a direct route to interact with the
SQL Database server. After a connection is established, further interactions between the client and database
involve no middleware proxy.
The sequence is as follows:
1. ADO.NET 4.5 (or later) initiates a brief interaction with the Azure cloud, and receives a dynamically
identified port number.
The dynamically identified port number is in the range of 11000-11999 or 14000-14999.
2. ADO.NET then connects to the SQL Database server directly, with no middleware in between.
3. Queries are sent directly to the database, and results are returned directly to the client.
Ensure that the port ranges of 11000-11999 and 14000-14999 on your Azure client machine are left available for
ADO.NET 4.5 client interactions with SQL Database.
In particular, ports in the range must be free of any other outbound blockers.
On your Azure VM, the Windows Firewall with Advanced Security controls the port settings.
You can use the firewall's user interface to add a rule for which you specify the TCP protocol along with
a port range with the syntax like 11000-11999.

Version clarifications
This section clarifies the monikers that refer to product versions. It also lists some pairings of versions between
products.
ADO.NET
ADO.NET 4.0 supports the TDS 7.3 protocol, but not 7.4.
ADO.NET 4.5 and later supports the TDS 7.4 protocol.

Related links
ADO.NET 4.6 was released on July 20, 2015. A blog announcement from the .NET team is available here.
ADO.NET 4.5 was released on August 15, 2012. A blog announcement from the .NET team is available here.
A blog post about ADO.NET 4.5.1 is available here.
TDS protocol version list
SQL Database Development Overview
Azure SQL Database firewall
How to: Configure firewall settings on SQL Database
Get the required values for authenticating an
application to access SQL Database from code
1/17/2017 • 1 min to read • Edit Online

To create and manage SQL Database from code you must register your app in the Azure Active Directory (AAD)
domain in the subscription where your Azure resources have been created.

Create a service principal to access resources from an application

You need to have the latest Azure PowerShell installed and running. For detailed information, see How to install
and configure Azure PowerShell.
The following PowerShell script creates the Active Directory (AD) application and the service principal that we need
to authenticate our C# app. The script outputs values we need for the preceding C# sample. For detailed
information, see Use Azure PowerShell to create a service principal to access resources.

# Sign in to Azure.
Add-AzureRmAccount

# If you have multiple subscriptions, uncomment and set to the subscription you want to work with.
#$subscriptionId = "{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}"
#Set-AzureRmContext -SubscriptionId $subscriptionId

# Provide these values for your new AAD app.

# $appName is the display name for your app, must be unique in your directory.
# $uri does not need to be a real uri.
# $secret is a password you create.

$appName = "{app-name}"
$uri = "http://{app-name}"
$secret = "{app-password}"

# Create a AAD app

$azureAdApplication = New-AzureRmADApplication -DisplayName $appName -HomePage $Uri -IdentifierUris $Uri -Password $secret

# Create a Service Principal for the app

$svcprincipal = New-AzureRmADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId

# To avoid a PrincipalNotFound error, I pause here for 15 seconds.

Start-Sleep -s 15

# If you still get a PrincipalNotFound error, then rerun the following until successful.
$roleassignment = New-AzureRmRoleAssignment -RoleDefinitionName Contributor -ServicePrincipalName
$azureAdApplication.ApplicationId.Guid

# Output the values we need for our C# application to successfully authenticate

Write-Output "Copy these values into the C# sample app"

Write-Output "_subscriptionId:" (Get-AzureRmContext).Subscription.SubscriptionId

Write-Output "_tenantId:" (Get-AzureRmContext).Tenant.TenantId
Write-Output "_applicationId:" $azureAdApplication.ApplicationId.Guid
Write-Output "_applicationSecret:" $secret

See also
Create a SQL database with C#
Connecting to SQL Database By Using Azure Active Directory Authentication
SQL error codes for SQL Database client
applications: Database connection error and other
issues
2/16/2017 • 18 min to read • Edit Online

This article lists SQL error codes for SQL Database client applications, including database connection errors,
transient errors (also called transient faults), resource governance errors, database copy issues, elastic pool, and
other errors. Most categories are particular to Azure SQL Database, and do not apply to Microsoft SQL Server.

Database connection errors, transient errors, and other temporary

errors
The following table covers the SQL error codes for connection loss errors, and other transient errors you might
encounter when your application attempts to access SQL Database. For getting started tutorials on how to connect
to Azure SQL Database, see Connecting to Azure SQL Database.
Most common database connection errors and transient fault errors
The Azure infrastructure has the ability to dynamically reconfigure servers when heavy workloads arise in the SQL
Database service. This dynamic behavior might cause your client program to lose its connection to SQL Database.
This kind of error condition is called a transient fault.
If your client program has retry logic, it can try to reestablish a connection after giving the transient fault time to
correct itself. We recommend that you delay for 5 seconds before your first retry. Retrying after a delay shorter
than 5 seconds risks overwhelming the cloud service. For each subsequent retry the delay should grow
exponentially, up to a maximum of 60 seconds.
Transient fault errors typically manifest as one of the following error messages from your client programs:
Database on server is not currently available. Please retry the connection later. If the problem persists, contact
customer support, and provide them the session tracing ID of
Database on server is not currently available. Please retry the connection later. If the problem persists, contact
customer support, and provide them the session tracing ID of . (Microsoft SQL Server, Error: 40613)
An existing connection was forcibly closed by the remote host.
System.Data.Entity.Core.EntityCommandExecutionException: An error occurred while executing the command
definition. See the inner exception for details. ---> System.Data.SqlClient.SqlException: A transport-level error
has occurred when receiving results from the server. (provider: Session Provider, error: 19 - Physical
connection is not usable)
For code examples of retry logic, see:
Connection Libraries for SQL Database and SQL Server
Actions to fix connection errors and transient errors in SQL Database
A discussion of the blocking period for clients that use ADO.NET is available in SQL Server Connection Pooling
(ADO.NET).
Transient fault error codes
The following errors are transient, and should be retried in application logic
ERROR CODE SEVERITY DESCRIPTION

4060 16 Cannot open database "%.*ls"

requested by the login. The login failed.

40197 17 The service has encountered an error

processing your request. Please try
again. Error code %d.

You will receive this error, when the

service is down due to software or
hardware upgrades, hardware failures,
or any other failover problems. The
error code (%d) embedded within the
message of error 40197 provides
additional information about the kind
of failure or failover that occurred.
Some examples of the error codes are
embedded within the message of error
40197 are 40020, 40143, 40166, and
40540.

Reconnecting to your SQL Database

server will automatically connect you to
a healthy copy of your database. Your
application must catch error 40197, log
the embedded error code (%d) within
the message for troubleshooting, and
try reconnecting to SQL Database until
the resources are available, and your
connection is established again.

40501 20 The service is currently busy. Retry the

request after 10 seconds. Incident ID:
%ls. Code: %d.

Note: For more information, see:

• Azure SQL Database resource limits.

40613 17 Database '%.ls' on server '%.ls' is not

currently available. Please retry the
connection later. If the problem
persists, contact customer support, and
provide them the session tracing ID of
'%.*ls'.

49918 16 Cannot process request. Not enough

resources to process request.

The service is currently busy. Please

retry the request later.
ERROR CODE SEVERITY DESCRIPTION

49919 16 Cannot process create or update

request. Too many create or update
operations in progress for subscription
"%ld".

The service is busy processing multiple

create or update requests for your
subscription or server. Requests are
currently blocked for resource
optimization. Query
sys.dm_operation_status for pending
operations. Wait till pending create or
update requests are complete or delete
one of your pending requests and retry
your request later.

49920 16 Cannot process request. Too many

operations in progress for subscription
"%ld".

The service is busy processing multiple

requests for this subscription. Requests
are currently blocked for resource
optimization. Query
sys.dm_operation_status for operation
status. Wait until pending requests are
complete or delete one of your pending
requests and retry your request later.

Database copy errors

The following errors can be encountered while copying a database in Azure SQL Database. For more information,
see Copy an Azure SQL Database.

ERROR CODE SEVERITY DESCRIPTION

40635 16 Client with IP address '%.*ls' is

temporarily disabled.

40637 16 Create database copy is currently

disabled.

40561 16 Database copy failed. Either the source

or target database does not exist.

40562 16 Database copy failed. The source

database has been dropped.

40563 16 Database copy failed. The target

database has been dropped.

40564 16 Database copy failed due to an internal

error. Please drop target database and
try again.
ERROR CODE SEVERITY DESCRIPTION

40565 16 Database copy failed. No more than 1

concurrent database copy from the
same source is allowed. Please drop
target database and try again later.

40566 16 Database copy failed due to an internal

error. Please drop target database and
try again.

40567 16 Database copy failed due to an internal

error. Please drop target database and
try again.

40568 16 Database copy failed. Source database

has become unavailable. Please drop
target database and try again.

40569 16 Database copy failed. Target database

has become unavailable. Please drop
target database and try again.

40570 16 Database copy failed due to an internal

error. Please drop target database and
try again later.

40571 16 Database copy failed due to an internal

error. Please drop target database and
try again later.

Resource governance errors

The following errors are caused by excessive use of resources while working with Azure SQL Database. For
example:
A transaction has been open for too long.
A transaction is holding too many locks.
An application is consuming too much memory.
An application is consuming too much TempDb space.

Related topics:
More detailed information is available here: Azure SQL Database resource limits

ERROR CODE SEVERITY DESCRIPTION

10928 20 Resource ID: %d. The %s limit for the

database is %d and has been reached.
For more information, see
http://go.microsoft.com/fwlink/?
LinkId=267637.

The Resource ID indicates the resource

that has reached the limit. For worker
threads, the Resource ID = 1. For
sessions, the Resource ID = 2.

Note: For more information about this

error and how to resolve it, see:
• Azure SQL Database resource limits.

10929 20 Resource ID: %d. The %s minimum

guarantee is %d, maximum limit is %d
and the current usage for the database
is %d. However, the server is currently
too busy to support requests greater
than %d for this database. For more
information, see
http://go.microsoft.com/fwlink/?
LinkId=267637. Otherwise, please try
again later.

The Resource ID indicates the resource

that has reached the limit. For worker
threads, the Resource ID = 1. For
sessions, the Resource ID = 2.

Note: For more information about this

error and how to resolve it, see:
• Azure SQL Database resource limits.

40544 20 The database has reached its size

quota. Partition or delete data, drop
indexes, or consult the documentation
for possible resolutions.

40549 16 Session is terminated because you have

a long-running transaction. Try
shortening your transaction.

40550 16 The session has been terminated

because it has acquired too many locks.
Try reading or modifying fewer rows in
a single transaction.

40551 16 The session has been terminated

because of excessive TEMPDB usage.
Try modifying your query to reduce the
temporary table space usage.

Tip: If you are using temporary objects,

conserve space in the TEMPDB
database by dropping temporary
objects after they are no longer needed
by the session.
ERROR CODE SEVERITY DESCRIPTION

40552 16 The session has been terminated

because of excessive transaction log
space usage. Try modifying fewer rows
in a single transaction.

Tip: If you perform bulk inserts using

the bcp.exe utility or the
System.Data.SqlClient.SqlBulkCopy
class, try using the -b batchsize or
BatchSize options to limit the
number of rows copied to the server in
each transaction. If you are rebuilding
an index with the ALTER INDEX
statement, try using the
REBUILD WITH ONLINE = ON option.

40553 16 The session has been terminated

because of excessive memory usage.
Try modifying your query to process
fewer rows.

Tip: Reducing the number of

ORDER BY and GROUP BY operations
in your Transact-SQL code reduces the
memory requirements of your query.

Elastic pool errors

The following errors are related to creating and using Elastics Pools.

ERRORCORRECTIV
ERRORNUMBER ERRORSEVERITY ERRORFORMAT ERRORINSERTS ERRORCAUSE EACTION

1132 EX_RESOURCE The elastic pool Elastic pool space Attempting to Please consider
has reached its limit in MBs. write data to a increasing the
storage limit. The database when DTUs of the
storage usage for the storage limit elastic pool if
the elastic pool of the elastic pool possible in order
cannot exceed has been to increase its
(%d) MBs. reached. storage limit,
reduce the
storage used by
individual
databases within
the elastic pool,
or remove
databases from
the elastic pool.
ERRORCORRECTIV
ERRORNUMBER ERRORSEVERITY ERRORFORMAT ERRORINSERTS ERRORCAUSE EACTION

10929 EX_USER The %s minimum DTU min per The total number Please consider
guarantee is %d, database; DTU of concurrent increasing the
maximum limit is max per database workers DTUs of the
%d and the (requests) across elastic pool if
current usage for all databases in possible in order
the database is the elastic pool to increase its
%d. However, the attempted to worker limit, or
server is currently exceed the pool remove
too busy to limit. databases from
support requests the elastic pool.
greater than %d
for this database.
See
http://go.microso
ft.com/fwlink/?
LinkId=267637
for assistance.
Otherwise, please
try again later.

40844 EX_USER Database '%ls' on database name, A Coming soon

Server '%ls' is a database edition, StartDatabaseCo
'%ls' edition server name py command is
database in an issued for a non-
elastic pool and premium db in
cannot have a an elastic pool.
continuous copy
relationship.

40857 EX_USER Elastic pool not name of server; Specified elastic Please provide a
found for server: elastic pool name pool does not valid elastic pool
'%ls', elastic pool exist in the name.
name: '%ls'. specified server.

40858 EX_USER Elastic pool '%ls' elastic pool Specified elastic Provide new
already exists in name, server pool already elastic pool
server: '%ls' name exists in the name.
specified logical
server.

40859 EX_USER Elastic pool does elastic pool Specified service Provide the
not support service tier tier is not correct edition or
service tier '%ls'. supported for leave service tier
elastic pool blank to use the
provisioning. default service
tier.

40860 EX_USER Elastic pool '%ls' elastic pool Elastic pool and Please specify
and service name; service service objective correct
objective '%ls' level objective can be specified combination of
combination is name together only if elastic pool and
invalid. service objective service objective.
is specified as
'ElasticPool'.
ERRORCORRECTIV
ERRORNUMBER ERRORSEVERITY ERRORFORMAT ERRORINSERTS ERRORCAUSE EACTION

40861 EX_USER The database database edition, The database Please do not
edition '%.ls' elastic pool edition is specify a
cannot be service tier different than the database edition
different than elastic pool which is different
the elastic pool service tier. than the elastic
service tier which pool service tier.
is '%.ls'. Note that the
database edition
does not need to
be specified.

40862 EX_USER Elastic pool name None Elastic pool Please specify the
must be specified service objective elastic pool name
if the elastic pool does not if using the
service objective uniquely identify elastic pool
is specified. an elastic pool. service objective.

40864 EX_USER The DTUs for the DTUs for elastic Attempting to Please retry
elastic pool must pool; elastic pool set the DTUs for setting the DTUs
be at least (%d) service tier. the elastic pool for the elastic
DTUs for service below the pool to at least
tier '%.*ls'. minimum limit. the minimum
limit.

40865 EX_USER The DTUs for the DTUs for elastic Attempting to Please retry
elastic pool pool; elastic pool set the DTUs for setting the DTUs
cannot exceed service tier. the elastic pool for the elastic
(%d) DTUs for above the pool to no
service tier '%.*ls'. maximum limit. greater than the
maximum limit.

40867 EX_USER The DTU max per DTU max per Attempting to Please consider
database must database; elastic set the DTU max using the elastic
be at least (%d) pool service tier per database pool service tier
for service tier below the that supports the
'%.*ls'. supported limit. desired setting.

40868 EX_USER The DTU max per DTU max per Attempting to Please consider
database cannot database; elastic set the DTU max using the elastic
exceed (%d) for pool service tier. per database pool service tier
service tier '%.*ls'. beyond the that supports the
supported limit. desired setting.

40870 EX_USER The DTU min per DTU min per Attempting to Please consider
database cannot database; elastic set the DTU min using the elastic
exceed (%d) for pool service tier. per database pool service tier
service tier '%.*ls'. beyond the that supports the
supported limit. desired setting.
ERRORCORRECTIV
ERRORNUMBER ERRORSEVERITY ERRORFORMAT ERRORINSERTS ERRORCAUSE EACTION

40873 EX_USER The number of Number Attempting to Please consider

databases (%d) databases in specify DTU min increasing the
and DTU min per elastic pool; DTU for databases in DTUs of the
database (%d) min per the elastic pool elastic pool, or
cannot exceed database; DTUs that exceeds the decrease the DTU
the DTUs of the of elastic pool. DTUs of the min per
elastic pool (%d). elastic pool. database, or
decrease the
number of
databases in the
elastic pool.

40877 EX_USER An elastic pool None The elastic pool Please remove
cannot be contains one or databases from
deleted unless it more databases the elastic pool in
does not contain and therefore order to delete it.
any databases. cannot be
deleted.

40881 EX_USER The elastic pool Name of elastic Attempting to Please consider
'%.*ls' has pool; database create or add increasing the
reached its count limit of database to DTUs of the
database count elastic pool;e elastic pool when elastic pool if
limit. The DTUs for the database possible in order
database count resource pool. count limit of the to increase its
limit for the elastic pool has database limit, or
elastic pool been reached. remove
cannot exceed databases from
(%d) for an elastic the elastic pool.
pool with (%d)
DTUs.

40889 EX_USER The DTUs or Name of elastic Attempting to Please consider

storage limit for pool. decrease the reducing the
the elastic pool storage limit of storage usage of
'%.*ls' cannot be the elastic pool individual
decreased since below its storage databases in the
that would not usage. elastic pool or
provide sufficient remove
storage space for databases from
its databases. the pool in order
to reduce its
DTUs or storage
limit.

40891 EX_USER The DTU min per DTU min per Attempting to Please ensure the
database (%d) database; DTU set the DTU min DTU min per
cannot exceed max per per database databases does
the DTU max per database. higher than the not exceed the
database (%d). DTU max per DTU max per
database. database.
ERRORCORRECTIV
ERRORNUMBER ERRORSEVERITY ERRORFORMAT ERRORINSERTS ERRORCAUSE EACTION

TBD EX_USER The storage size elastic pool The max size for Please set the
for an individual service tier the database max size of the
database in a exceeds the max database within
elastic pool size allowed by the limits of the
cannot exceed the elastic pool max size allowed
the max size service tier. by the elastic
allowed by '%.*ls' pool service tier.
service tier elastic
pool.

Related topics:
Create an elastic pool (C#)
Manage an elastic pool (C#).
Create an elastic pool (PowerShell)
Monitor and manage an elastic pool (PowerShell).

General errors
The following errors do not fall into any previous categories.

ERROR CODE SEVERITY DESCRIPTION

15006 16 is not a valid name because it contains

invalid characters.

18452 14 Login failed. The login is from an

untrusted domain and cannot be used
with Windows authentication.%.*ls
(Windows logins are not supported in
this version of SQL Server.)

18456 14 Login failed for user

'%.*ls'.%.*ls%.*ls(The login failed for user
"%.*ls". The password change failed.
Password change during login is not
supported in this version of SQL
Server.)

18470 14 Login failed for user '%.*ls'. Reason: The

account is disabled.%.*ls

40014 16 Multiple databases cannot be used in

the same transaction.

40054 16 Tables without a clustered index are not

supported in this version of SQL Server.
Create a clustered index and try again.

40133 15 This operation is not supported in this

version of SQL Server.

40506 16 Specified SID is invalid for this version

of SQL Server.
ERROR CODE SEVERITY DESCRIPTION

40507 16 '%.*ls' cannot be invoked with

parameters in this version of SQL
Server.

40508 16 USE statement is not supported to

switch between databases. Use a new
connection to connect to a different
database.

40510 16 Statement '%.*ls' is not supported in

this version of SQL Server

40511 16 Built-in function '%.*ls' is not supported

in this version of SQL Server.

40512 16 Deprecated feature '%ls' is not

supported in this version of SQL Server.

40513 16 Server variable '%.*ls' is not supported

in this version of SQL Server.

40514 16 '%ls' is not supported in this version of

SQL Server.

40515 16 Reference to database and/or server

name in '%.*ls' is not supported in this
version of SQL Server.

40516 16 Global temp objects are not supported

in this version of SQL Server.

40517 16 Keyword or statement option '%.*ls' is

not supported in this version of SQL
Server.

40518 16 DBCC command '%.*ls' is not

supported in this version of SQL Server.

40520 16 Securable class '%S_MSG' not

supported in this version of SQL Server.

40521 16 Securable class '%S_MSG' not

supported in the server scope in this
version of SQL Server.

40522 16 Database principal '%.*ls' type is not

supported in this version of SQL Server.

40523 16 Implicit user '%.*ls' creation is not

supported in this version of SQL Server.
Explicitly create the user before using it.

40524 16 Data type '%.*ls' is not supported in

this version of SQL Server.
ERROR CODE SEVERITY DESCRIPTION

40525 16 WITH '%.ls' is not supported in this

version of SQL Server.

40526 16 '%.*ls' rowset provider not supported in

this version of SQL Server.

40527 16 Linked servers are not supported in this

version of SQL Server.

40528 16 Users cannot be mapped to certificates,

asymmetric keys, or Windows logins in
this version of SQL Server.

40529 16 Built-in function '%.*ls' in impersonation

context is not supported in this version
of SQL Server.

40532 11 Cannot open server "%.*ls" requested

by the login. The login failed.

40553 16 The session has been terminated

because of excessive memory usage.
Try modifying your query to process
fewer rows.

Note: Reducing the number of

ORDER BY and GROUP BY operations
in your Transact-SQL code helps reduce
the memory requirements of your
query.

40604 16 Could not CREATE/ALTER DATABASE

because it would exceed the quota of
the server.

40606 16 Attaching databases is not supported in

this version of SQL Server.

40607 16 Windows logins are not supported in

this version of SQL Server.

40611 16 Servers can have at most 128 firewall

rules defined.

40614 16 Start IP address of firewall rule cannot

exceed End IP address.

40615 16 Cannot open server '{0}' requested by

the login. Client with IP address '{1}' is
not allowed to access the server. To
enable access, use the SQL Database
Portal or run sp_set_firewall_rule on the
master database to create a firewall rule
for this IP address or address range. It
may take up to five minutes for this
change to take effect.
ERROR CODE SEVERITY DESCRIPTION

40617 16 The firewall rule name that starts with is

too long. Maximum length is 128.

40618 16 The firewall rule name cannot be empty.

40620 16 The login failed for user "%.*ls". The

password change failed. Password
change during login is not supported in
this version of SQL Server.

40627 20 Operation on server '{0}' and database

'{1}' is in progress. Please wait a few
minutes before trying again.

40630 16 Password validation failed. The

password does not meet policy
requirements because it is too short.

40631 16 The password that you specified is too

long. The password should have no
more than 128 characters.

40632 16 Password validation failed. The

password does not meet policy
requirements because it is not complex
enough.

40636 16 Cannot use a reserved database name

'%.*ls' in this operation.

40638 16 Invalid subscription id . Subscription

does not exist.

40639 16 Request does not conform to schema: .

40640 20 The server encountered an unexpected

exception.

40641 16 The specified location is invalid.

40642 17 The server is currently too busy. Please

try again later.

40643 16 The specified x-ms-version header value

is invalid.

40644 14 Failed to authorize access to the

specified subscription.

40645 16 Servername cannot be empty or null. It

can only be made up of lowercase
letters 'a'-'z', the numbers 0-9 and the
hyphen. The hyphen may not lead or
trail in the name.
ERROR CODE SEVERITY DESCRIPTION

40646 16 Subscription ID cannot be empty.

40647 16 Subscription <subscription-id does not

have server servername.

40648 17 Too many requests have been

performed. Please retry later.

40649 16 Invalid content-type is specified. Only

application/xml is supported.

40650 16 Subscription does not exist or is not

ready for the operation.

40651 16 Failed to create server because the

subscription is disabled.

40652 16 Cannot move or create server.

Subscription will exceed server quota.

40671 17 Communication failure between the

gateway and the management service.
Please retry later.

40852 16 Cannot open database '%.ls' on server

'%.ls' requested by the login. Access to
the database is only allowed using a
security-enabled connection string. To
access this database, modify your
connection strings to contain ‘secure’ in
the server FQDN - 'server
name'.database.windows.net should be
modified to 'server name'.database.
secure .windows.net.

45168 16 The SQL Azure system is under load,

and is placing an upper limit on
concurrent DB CRUD operations for a
single server (e.g., create database). The
server specified in the error message
has exceeded the maximum number of
concurrent connections. Try again later.

45169 16 The SQL azure system is under load,

and is placing an upper limit on the
number of concurrent server CRUD
operations for a single subscription
(e.g., create server). The subscription
specified in the error message has
exceeded the maximum number of
concurrent connections, and the
request was denied. Try again later.

Related links
Azure SQL Database Features
Azure SQL Database resource limits
Using elastic database client library with Dapper
1/17/2017 • 8 min to read • Edit Online

This document is for developers that rely on Dapper to build applications, but also want to embrace elastic
database tooling to create applications that implement sharding to scale-out their data tier. This document
illustrates the changes in Dapper-based applications that are necessary to integrate with elastic database tools. Our
focus is on composing the elastic database shard management and data dependent routing with Dapper.
Sample Code: Elastic database tools for Azure SQL Database - Dapper integration.
Integrating Dapper and DapperExtensions with the elastic database client library for Azure SQL Database is easy.
Your applications can use data dependent routing by changing the creation and opening of new SqlConnection
objects to use the OpenConnectionForKey call from the client library. This limits changes in your application to only
where new connections are created and opened.

Dapper overview
Dapper is an object-relational mapper. It maps .NET objects from your application to a relational database (and
vice versa). The first part of the sample code illustrates how you can integrate the elastic database client library with
Dapper-based applications. The second part of the sample code illustrates how to integrate when using both
Dapper and DapperExtensions.
The mapper functionality in Dapper provides extension methods on database connections that simplify submitting
T-SQL statements for execution or querying the database. For instance, Dapper makes it easy to map between your
.NET objects and the parameters of SQL statements for Execute calls, or to consume the results of your SQL
queries into .NET objects using Query calls from Dapper.
When using DapperExtensions, you no longer need to provide the SQL statements. Extensions methods such as
GetList or Insert over the database connection create the SQL statements behind the scenes.
Another benefit of Dapper and also DapperExtensions is that the application controls the creation of the database
connection. This helps interact with the elastic database client library which brokers database connections based on
the mapping of shardlets to databases.
To get the Dapper assemblies, see Dapper dot net. For the Dapper extensions, see DapperExtensions.

A quick Look at the elastic database client library

With the elastic database client library, you define partitions of your application data called shardlets , map them to
databases, and identify them by sharding keys. You can have as many databases as you need and distribute your
shardlets across these databases. The mapping of sharding key values to the databases is stored by a shard map
provided by the library’s APIs. This capability is called shard map management. The shard map also serves as the
broker of database connections for requests that carry a sharding key. This capability is referred to as data
dependent routing.
The shard map manager protects users from inconsistent views into shardlet data that can occur when concurrent
shardlet management operations are happening on the databases. To do so, the shard maps broker the database
connections for an application built with the library. When shard management operations could impact the
shardlet, this allows the shard map functionality to automatically kill a database connection.
Instead of using the traditional way to create connections for Dapper, we need to use the OpenConnectionForKey
method. This ensures that all the validation takes place and connections are managed properly when any data
moves between shards.
Requirements for Dapper integration
When working with both the elastic database client library and the Dapper APIs, we want to retain the following
properties:
Scaleout: We want to add or remove databases from the data tier of the sharded application as necessary for
the capacity demands of the application.
Consistency: Since our application is scaled out using sharding, we need to perform data dependent routing.
We want to use the Data dependent routing capabilities of the library to do so. In particular, we want to retain
the validation and consistency guarantees provided by connections that are brokered through the shard map
manager in order to avoid corruption or wrong query results. This ensures that connections to a given shardlet
are rejected or stopped if (for instance) the shardlet is currently moved to a different shard using Split/Merge
APIs.
Object Mapping: We want to retain the convenience of the mappings provided by Dapper to translate between
classes in the application and the underlying database structures.
The following section provides guidance for these requirements for applications based on Dapper and
DapperExtensions.

Technical Guidance
Data dependent routing with Dapper
With Dapper, the application is typically responsible for creating and opening the connections to the underlying
database. Given a type T by the application, Dapper returns query results as .NET collections of type T. Dapper
performs the mapping from the T-SQL result rows to the objects of type T. Similarly, Dapper maps .NET objects
into SQL values or parameters for data manipulation language (DML) statements. Dapper offers this functionality
via extension methods on the regular SqlConnection object from the ADO .NET SQL Client libraries. The SQL
connection returned by the Elastic Scale APIs for DDR are also regular SqlConnection objects. This allows us to
directly use Dapper extensions over the type returned by the client library’s DDR API, as it is also a simple SQL
Client connection.
These observations make it straightforward to use connections brokered by the elastic database client library for
Dapper.
This code example (from the accompanying sample) illustrates the approach where the sharding key is provided by
the application to the library to broker the connection to the right shard.

using (SqlConnection sqlconn = shardingLayer.ShardMap.OpenConnectionForKey(

key: tenantId1,
connectionString: connStrBldr.ConnectionString,
options: ConnectionOptions.Validate))
{
var blog = new Blog { Name = name };
sqlconn.Execute(@"
INSERT INTO
Blog (Name)
VALUES (@name)", new { name = blog.Name }
);
}

The call to the OpenConnectionForKey API replaces the default creation and opening of a SQL Client connection.
The OpenConnectionForKey call takes the arguments that are required for data dependent routing:
The shard map to access the data dependent routing interfaces
The sharding key to identify the shardlet
The credentials (user name and password) to connect to the shard
The shard map object creates a connection to the shard that holds the shardlet for the given sharding key. The
elastic database client APIs also tag the connection to implement its consistency guarantees. Since the call to
OpenConnectionForKey returns a regular SQL Client connection object, the subsequent call to the Execute
extension method from Dapper follows the standard Dapper practice.
Queries work very much the same way - you first open the connection using OpenConnectionForKey from the
client API. Then you use the regular Dapper extension methods to map the results of your SQL query into .NET
objects:

using (SqlConnection sqlconn = shardingLayer.ShardMap.OpenConnectionForKey(

key: tenantId1,
connectionString: connStrBldr.ConnectionString,
options: ConnectionOptions.Validate ))
{
// Display all Blogs for tenant 1
IEnumerable<Blog> result = sqlconn.Query<Blog>(@"
SELECT *
FROM Blog
ORDER BY Name");

Console.WriteLine("All blogs for tenant id {0}:", tenantId1);

foreach (var item in result)
{
Console.WriteLine(item.Name);
}
}

Note that the using block with the DDR connection scopes all database operations within the block to the one
shard where tenantId1 is kept. The query only returns blogs stored on the current shard, but not the ones stored on
any other shards.

Data dependent routing with Dapper and DapperExtensions

Dapper comes with an ecosystem of additional extensions that can provide further convenience and abstraction
from the database when developing database applications. DapperExtensions is an example.
Using DapperExtensions in your application does not change how database connections are created and managed.
It is still the application’s responsibility to open connections, and regular SQL Client connection objects are
expected by the extension methods. We can rely on the OpenConnectionForKey as outlined above. As the following
code samples show, the only change is that we do no longer have to write the T-SQL statements:

using (SqlConnection sqlconn = shardingLayer.ShardMap.OpenConnectionForKey(

key: tenantId2,
connectionString: connStrBldr.ConnectionString,
options: ConnectionOptions.Validate))
{
var blog = new Blog { Name = name2 };
sqlconn.Insert(blog);
}

And here is the code sample for the query:

using (SqlConnection sqlconn = shardingLayer.ShardMap.OpenConnectionForKey(

key: tenantId2,
connectionString: connStrBldr.ConnectionString,
options: ConnectionOptions.Validate))
{
// Display all Blogs for tenant 2
IEnumerable<Blog> result = sqlconn.GetList<Blog>();
Console.WriteLine("All blogs for tenant id {0}:", tenantId2);
foreach (var item in result)
{
Console.WriteLine(item.Name);
}
}

Handling transient faults

The Microsoft Patterns & Practices team published the Transient Fault Handling Application Block to help
application developers mitigate common transient fault conditions encountered when running in the cloud. For
more information, see Perseverance, Secret of All Triumphs: Using the Transient Fault Handling Application Block.
The code sample relies on the transient fault library to protect against transient faults.

SqlDatabaseUtils.SqlRetryPolicy.ExecuteAction(() =>
{
using (SqlConnection sqlconn =
shardingLayer.ShardMap.OpenConnectionForKey(tenantId2, connStrBldr.ConnectionString, ConnectionOptions.Validate))
{
var blog = new Blog { Name = name2 };
sqlconn.Insert(blog);
}
});

SqlDatabaseUtils.SqlRetryPolicy in the code above is defined as a

SqlDatabaseTransientErrorDetectionStrategy with a retry count of 10, and 5 seconds wait time between
retries. If you are using transactions, make sure that your retry scope goes back to the beginning of the transaction
in the case of a transient fault.

Limitations
The approaches outlined in this document entail a couple of limitations:
The sample code for this document does not demonstrate how to manage schema across shards.
Given a request, we assume that all its database processing is contained within a single shard as identified by
the sharding key provided by the request. However, this assumption does not always hold, for example, when it
is not possible to make a sharding key available. To address this, the elastic database client library includes the
MultiShardQuery class. The class implements a connection abstraction for querying over several shards. Using
MultiShardQuery in combination with Dapper is beyond the scope of this document.

Conclusion
Applications using Dapper and DapperExtensions can easily benefit from elastic database tools for Azure SQL
Database. Through the steps outlined in this document, those applications can use the tool's capability for data
dependent routing by changing the creation and opening of new SqlConnection objects to use the
OpenConnectionForKey call of the elastic database client library. This limits the application changes required to
those places where new connections are created and opened.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Getting started with JSON features in Azure SQL
Database
1/17/2017 • 6 min to read • Edit Online

Azure SQL Database lets you parse and query data represented in JavaScript Object Notation (JSON) format, and
export your relational data as JSON text.
JSON is a popular data format used for exchanging data in modern web and mobile applications. JSON is also used
for storing semi-structured data in log files or in NoSQL databases like Azure DocumentDB. Many REST web
services return results formatted as JSON text or accept data formatted as JSON. Most Azure services such as
Azure Search, Azure Storage, and Azure DocumentDB have REST endpoints that return or consume JSON.
Azure SQL Database lets you work with JSON data easily and integrate your database with modern services.

Overview
Azure SQL Database provides the following functions for working with JSON data:

If you have JSON text, you can extract data from JSON or verify that JSON is properly formatted by using the built-
in functions JSON_VALUE, JSON_QUERY, and ISJSON. The JSON_MODIFY function lets you update value inside
JSON text. For more advanced querying and analysis, OPENJSON function can transform an array of JSON objects
into a set of rows. Any SQL query can be executed on the returned result set. Finally, there is a FOR JSON clause
that lets you format data stored in your relational tables as JSON text.

Formatting relational data in JSON format

If you have a web service that takes data from the database layer and provides a response in JSON format, or
client-side JavaScript frameworks or libraries that accept data formatted as JSON, you can format your database
content as JSON directly in a SQL query. You no longer have to write application code that formats results from
Azure SQL Database as JSON, or include some JSON serialization library to convert tabular query results and then
serialize objects to JSON format. Instead, you can use the FOR JSON clause to format SQL query results as JSON in
Azure SQL Database and use it directly in your application.
In the following example, rows from the Sales.Customer table are formatted as JSON by using the FOR JSON
clause:
select CustomerName, PhoneNumber, FaxNumber
from Sales.Customers
FOR JSON PATH

The FOR JSON PATH clause formats the results of the query as JSON text. Column names are used as keys, while
the cell values are generated as JSON values:

[
{"CustomerName":"Eric Torres","PhoneNumber":"(307) 555-0100","FaxNumber":"(307) 555-0101"},
{"CustomerName":"Cosmina Vlad","PhoneNumber":"(505) 555-0100","FaxNumber":"(505) 555-0101"},
{"CustomerName":"Bala Dixit","PhoneNumber":"(209) 555-0100","FaxNumber":"(209) 555-0101"}
]

The result set is formatted as a JSON array where each row is formatted as a separate JSON object.
PATH indicates that you can customize the output format of your JSON result by using dot notation in column
aliases. The following query changes the name of the "CustomerName" key in the output JSON format, and puts
phone and fax numbers in the "Contact" sub-object:

select CustomerName as Name, PhoneNumber as [Contact.Phone], FaxNumber as [Contact.Fax]

from Sales.Customers
where CustomerID = 931
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER

The output of this query looks like this:

{
"Name":"Nada Jovanovic",
"Contact":{
"Phone":"(215) 555-0100",
"Fax":"(215) 555-0101"
}
}

In this example we returned a single JSON object instead of an array by specifying the
WITHOUT_ARRAY_WRAPPER option. You can use this option if you know that you are returning a single object as a
result of query.
The main value of the FOR JSON clause is that it lets you return complex hierarchical data from your database
formatted as nested JSON objects or arrays. The following example shows how to include Orders that belong to
the Customer as a nested array of Orders:

select CustomerName as Name, PhoneNumber as Phone, FaxNumber as Fax,

Orders.OrderID, Orders.OrderDate, Orders.ExpectedDeliveryDate
from Sales.Customers Customer
join Sales.Orders Orders
on Customer.CustomerID = Orders.CustomerID
where Customer.CustomerID = 931
FOR JSON AUTO, WITHOUT_ARRAY_WRAPPER

Instead of sending separate queries to get Customer data and then to fetch a list of related Orders, you can get all
the necessary data with a single query, as shown in the following sample output:
{
"Name":"Nada Jovanovic",
"Phone":"(215) 555-0100",
"Fax":"(215) 555-0101",
"Orders":[
{"OrderID":382,"OrderDate":"2013-01-07","ExpectedDeliveryDate":"2013-01-08"},
{"OrderID":395,"OrderDate":"2013-01-07","ExpectedDeliveryDate":"2013-01-08"},
{"OrderID":1657,"OrderDate":"2013-01-31","ExpectedDeliveryDate":"2013-02-01"}
]
}

Working with JSON data

If you don’t have strictly structured data, if you have complex sub-objects, arrays, or hierarchical data, or if your
data structures evolve over time, the JSON format can help you to represent any complex data structure.
JSON is a textual format that can be used like any other string type in Azure SQL Database. You can send or store
JSON data as a standard NVARCHAR:

CREATE TABLE Products (

Id int identity primary key,
Title nvarchar(200),
Data nvarchar(max)
)
go
CREATE PROCEDURE InsertProduct(@title nvarchar(200), @json nvarchar(max))
AS BEGIN
insert into Products(Title, Data)
values(@title, @json)
END

The JSON data used in this example is represented by using the NVARCHAR(MAX) type. JSON can be inserted into
this table or provided as an argument of the stored procedure using standard Transact-SQL syntax as shown in the
following example:

EXEC InsertProduct 'Toy car', '{"Price":50,"Color":"White","tags":["toy","children","games"]}'

Any client-side language or library that works with string data in Azure SQL Database will also work with JSON
data. JSON can be stored in any table that supports the NVARCHAR type, such as a Memory-optimized table or a
System-versioned table. JSON does not introduce any constraint either in the client-side code or in the database
layer.

Querying JSON data

If you have data formatted as JSON stored in Azure SQL tables, JSON functions let you use this data in any SQL
query.
JSON functions that are available in Azure SQL database let you treat data formatted as JSON as any other SQL
data type. You can easily extract values from the JSON text, and use JSON data in any query:
select Id, Title, JSON_VALUE(Data, '$.Color'), JSON_QUERY(Data, '$.tags')
from Products
where JSON_VALUE(Data, '$.Color') = 'White'

update Products
set Data = JSON_MODIFY(Data, '$.Price', 60)
where Id = 1

The JSON_VALUE function extracts a value from JSON text stored in the Data column. This function uses a
JavaScript-like path to reference a value in JSON text to extract. The extracted value can be used in any part of SQL
query.
The JSON_QUERY function is similar to JSON_VALUE. Unlike JSON_VALUE, this function extracts complex sub-
object such as arrays or objects that are placed in JSON text.
The JSON_MODIFY function lets you specify the path of the value in the JSON text that should be updated, as well
as a new value that will overwrite the old one. This way you can easily update JSON text without reparsing the
entire structure.
Since JSON is stored in a standard text, there are no guarantees that the values stored in text columns are properly
formatted. You can verify that text stored in JSON column is properly formatted by using standard Azure SQL
Database check constraints and the ISJSON function:

ALTER TABLE Products

ADD CONSTRAINT [Data should be formatted as JSON]
CHECK (ISJSON(Data) > 0)

If the input text is properly formatted JSON, the ISJSON function returns the value 1. On every insert or update of
JSON column, this constraint will verify that new text value is not malformed JSON.

Transforming JSON into tabular format

Azure SQL Database also lets you transform JSON collections into tabular format and load or query JSON data.
OPENJSON is a table-value function that parses JSON text, locates an array of JSON objects, iterates through the
elements of the array, and returns one row in the output result for each element of the array.

In the example above, we can specify where to locate the JSON array that should be opened (in the $.Orders path),
what columns should be returned as result, and where to find the JSON values that will be returned as cells.
We can transform a JSON array in the @orders variable into a set of rows, analyze this result set, or insert rows
into a standard table:

CREATE PROCEDURE InsertOrders(@orders nvarchar(max))

AS BEGIN

insert into Orders(Number, Date, Customer, Quantity)

select Number, Date, Customer, Quantity
OPENJSON (@orders)
WITH (
Number varchar(200),
Date datetime,
Customer varchar(200),
Quantity int
)

END

The collection of orders formatted as a JSON array and provided as a parameter to the stored procedure can be
parsed and inserted into the Orders table.

Next steps
To learn how to integrate JSON into your application, check out these resources:
TechNet Blog
MSDN documentation
Channel 9 video
To learn about various scenarios for integrating JSON into your application, see the demos in this Channel 9 video
or find a scenario that matches your use case in JSON Blog posts.
Optimize performance by using In-Memory
technologies in SQL Database
4/14/2017 • 16 min to read • Edit Online

By using In-Memory technologies in Azure SQL Database, you can achieve performance improvements with
various workloads: transactional (online transactional processing (OLTP)), analytics (online analytical processing
(OLAP)), and mixed (hybrid transaction/analytical processing (HTAP)). Because of the more efficient query and
transaction processing, In-Memory technologies also help you to reduce cost. You typically don't need to upgrade
the pricing tier of the database to achieve performance gains. In some cases, you might even be able reduce the
pricing tier, while still seeing performance improvements with In-Memory technologies.
Here are two examples of how In-Memory OLTP helped to significantly improve performance:
By using In-Memory OLTP, Quorum Business Solutions was able to double their workload while improving
DTUs (i.e., resource consumption) by 70%.
The following video demonstrates significant improvement in resource consumption with a sample workload:
In-Memory OLTP in Azure SQL Database.
In-Memory technologies are available in all databases in the Premium tier, including databases in Premium elastic
pools.
The following video explains potential performance gains with In-Memory technologies in Azure SQL Database.
Remember that the performance gain that you see always depends on many factors, including the nature of the
workload and data, access pattern of the database, and so on.

Azure SQL Database has the following In-Memory technologies:

In-Memory OLTP increases throughput and reduces latency for transaction processing. Scenarios that benefit
from In-Memory OLTP are: high-throughput transaction processing such as trading and gaming, data ingestion
from events or IoT devices, caching, data load, and temporary table and table variable scenarios.
Clustered columnstore indexes reduce your storage footprint (up to 10 times) and improve performance for
reporting and analytics queries. You can use it with fact tables in your data marts to fit more data in your
database and improve performance. Also, you can use it with historical data in your operational database to
archive and be able to query up to 10 times more data.
Nonclustered columnstore indexes for HTAP help you to gain real-time insights into your business through
querying the operational database directly, without the need to run an expensive extract, transform, and load
(ETL) process and wait for the data warehouse to be populated. Nonclustered columnstore indexes allow very
fast execution of analytics queries on the OLTP database, while reducing the impact on the operational
workload.
You can also combine In-Memory OLTP and columnstore indexes. You can have a memory-optimized table
with a columnstore index. This allows you to both perform very fast transaction processing and run analytics
queries very quickly on the same data.
Both columnstore indexes and In-Memory OLTP have been part of the SQL Server product since 2012 and 2014,
respectively. Azure SQL Database and SQL Server share the same implementation of In-Memory technologies.
Going forward, new capabilities for these technologies are released in Azure SQL Database first, before they are
released in SQL Server.
This topic describes aspects of In-Memory OLTP and columnstore indexes that are specific to Azure SQL Database
and also includes samples. First, you'll see the impact of these technologies on storage and data size limits. Then,
you'll see how to manage the movement of databases that use these technologies between the different pricing
tiers. Finally, you'll see two samples that illustrate the use of In-Memory OLTP, as well as columnstore indexes in
Azure SQL Database.
See the following resources for more information.
In-depth information about the technologies:
In-Memory OLTP Overview and Usage Scenarios (includes references to customer case studies and information
to get started)
Documentation for In-Memory OLTP
Columnstore Indexes Guide
Hybrid transactional/analytical processing (HTAP), also known as real-time operational analytics
A quick primer on In-Memory OLTP: Quick Start 1: In-Memory OLTP Technologies for Faster T-SQL Performance
(another article to help you get started)
In-depth videos about the technologies:
In-Memory OLTP in Azure SQL Database (which contains a demo of performance benefits and steps to
reproduce these results yourself)
In-Memory OLTP Videos: What it is and When/How to use it
Columnstore Index: In-Memory Analytics (i.e. columnstore index) Videos from Ignite 2016

Storage and data size

Data size and storage cap for In-Memory OLTP
In-Memory OLTP includes memory-optimized tables, which are used for storing user data. These tables are
required to fit in memory. Because you manage memory directly in the SQL Database service, we have the concept
of a quota for user data. This idea is referred to as In-Memory OLTP storage.
Each supported standalone database pricing tier and each elastic pool pricing tier includes a certain amount of In-
Memory OLTP storage. At the time of writing, you get a gigabyte of storage for every 125 database transaction
units (DTUs) or elastic database transaction units (eDTUs).
The SQL Database service tiers article has the official list of the In-Memory OLTP storage that is available for each
supported standalone database and elastic pool pricing tier.
The following counts toward your In-Memory OLTP storage cap:
Active user data rows in memory-optimized tables and table variables. Note that old row versions don't count
toward the cap.
Indexes on memory-optimized tables.
Operational overhead of ALTER TABLE operations.
If you hit the cap, you'll receive an out-of-quota error and will no longer be able to insert or update data. To
mitigate this, delete data or increase the pricing tier of the database or pool.
For details about monitoring In-Memory OLTP storage utilization and configuring alerts when you almost hit the
cap, see Monitor In-Memory storage.
About elastic pools
With elastic pools, the In-Memory OLTP storage is shared across all databases in the pool. Therefore, the usage in
one database can potentially affect other databases. Two mitigations for this are:
Configure a Max-eDTU for databases that is lower than the eDTU count for the pool as a whole. This caps the
In-Memory OLTP storage utilization in any database in the pool to the size that corresponds to the eDTU count.
Configure a Min-eDTU that is greater than 0. This guarantees that each database in the pool has the amount of
available In-Memory OLTP storage that corresponds to the configured Min-eDTU.
Data size and storage for columnstore indexes
Columnstore indexes aren't required to fit in memory. Therefore, the only cap on the size of the indexes is the
maximum overall database size, which is documented in the SQL Database service tiers article.
When you use clustered columnstore indexes, columnar compression is used for the base table storage. This can
significantly reduce the storage footprint of your user data, which means that you can fit more data in the
database. And this can be further increased with columnar archival compression. The amount of compression that
you can achieve depends on the nature of the data, but 10 times the compression is not uncommon.
For example, if you have a database with a maximum size of 1 terabyte (TB) and you achieve 10 times the
compression by using columnstore indexes, you can fit a total of 10 TB of user data in the database.
When you use nonclustered columnstore indexes, the base table is still stored in the traditional rowstore format.
Therefore, the storage savings aren't as big as with clustered columnstore indexes. However, if you're replacing a
number of traditional nonclustered indexes with a single columnstore index, you can still see an overall savings in
the storage footprint for the table.

Moving databases that use In-Memory technologies between pricing

tiers
You don't need to have special considerations for increasing the pricing tier for a database that uses In-Memory
technologies because higher pricing tiers always have more functionality and more resources. Decreasing the
pricing tier can have implications for your database. This is especially true when you're moving from Premium to
Standard or Basic, and when you're moving a database that uses In-Memory OLTP to a lower Premium tier. The
same considerations apply when you're lowering the pricing tier of an elastic pool or moving databases with In-
Memory technologies into a Standard or Basic elastic pool.
In-Memory OLTP
Downgrading to Basic/Standard: In-Memory OLTP isn't supported in databases in the Standard or Basic tier. In
addition, it isn't possible to move a database that has any In-Memory OLTP objects to the Standard or Basic tier.
Before you downgrade the database to Standard/Basic, remove all memory-optimized tables and table types, as
well as all natively compiled T-SQL modules.
There is a programmatic way to understand whether a given database supports In-Memory OLTP. You can execute
the following Transact-SQL query:

SELECT DatabasePropertyEx(DB_NAME(), 'IsXTPSupported');

If the query returns 1, In-Memory OLTP is supported in this database.

Downgrading to a lower Premium tier: Data in memory-optimized tables must fit within the In-Memory OLTP
storage that is associated with the pricing tier of the database or is available in the elastic pool. If you try to lower
the pricing tier or move the database into a pool that doesn't have enough available In-Memory OLTP storage, the
operation will fail.
Columnstore indexes
Downgrading to Basic/Standard: Columnstore indexes aren't supported in databases in the Standard or Basic tier.
When you downgrade a database to Standard/Basic, columnstore indexes will become unavailable. If you use a
clustered columnstore index, this means that the table as a whole becomes unavailable.
Before you downgrade the database to Standard/Basic, drop all clustered columnstore indexes.
Downgrading to a lower Premium tier: This will succeed as long as the database as a whole fits within the
maximum database size for the target pricing tier or available storage in the elastic pool. There is no specific
impact from the columnstore indexes.

1. Install the In-Memory OLTP sample

You can create the AdventureWorksLT sample database with a few clicks in the Azure portal. Then, the steps in this
section explain how you can enrich your AdventureWorksLT database with In-Memory OLTP objects and
demonstrate performance benefits.
For a more simplistic, but more visually appealing performance demo for In-Memory OLTP, see:
Release: in-memory-oltp-demo-v1.0
Source code: in-memory-oltp-demo-source-code
Installation steps
1. In the Azure portal, create a Premium database on a server. Set the Source to the AdventureWorksLT
sample database. For detailed instructions, see Create your first Azure SQL database.
2. Connect to the database with SQL Server Management Studio (SSMS.exe).
3. Copy the In-Memory OLTP Transact-SQL script to your clipboard. The T-SQL script creates the necessary In-
Memory objects in the AdventureWorksLT sample database that you created in step 1.
4. Paste the T-SQL script into SSMS, and then execute the script. The MEMORY_OPTIMIZED = ON clause CREATE
TABLE statements are crucial. For example:

CREATE TABLE [SalesLT].[SalesOrderHeader_inmem](

[SalesOrderID] int IDENTITY NOT NULL PRIMARY KEY NONCLUSTERED ...,
...
) WITH (MEMORY_OPTIMIZED = ON);

Error 40536
If you get error 40536 when you run the T-SQL script, run the following T-SQL script to verify whether the
database supports In-Memory:

SELECT DatabasePropertyEx(DB_Name(), 'IsXTPSupported');

A result of 0 means that In-Memory isn't supported, and 1 means that it is supported. To diagnose the problem,
ensure that the database is at the Premium service tier.
About the created memory-optimized items
Tables: The sample contains the following memory-optimized tables:
SalesLT.Product_inmem
SalesLT.SalesOrderHeader_inmem
SalesLT.SalesOrderDetail_inmem
Demo.DemoSalesOrderHeaderSeed
Demo.DemoSalesOrderDetailSeed
You can inspect memory-optimized tables through the Object Explorer in SSMS. Right-click Tables > Filter >
Filter Settings > Is Memory Optimized. The value equals 1.
Or you can query the catalog views, such as:

SELECT is_memory_optimized, name, type_desc, durability_desc

FROM sys.tables
WHERE is_memory_optimized = 1;

Natively compiled stored procedure: You can inspect SalesLT.usp_InsertSalesOrder_inmem through a catalog
view query:

SELECT uses_native_compilation, OBJECT_NAME(object_id), definition

FROM sys.sql_modules
WHERE uses_native_compilation = 1;

Run the sample OLTP workload

The only difference between the following two stored procedures is that the first procedure uses memory-
optimized versions of the tables, while the second procedure uses the regular on-disk tables:
SalesLT.usp_InsertSalesOrder_inmem
SalesLT.usp_InsertSalesOrder_ondisk
In this section, you see how to use the handy ostress.exe utility to execute the two stored procedures at stressful
levels. You can compare how long it takes for the two stress runs to finish.
When you run ostress.exe, we recommend that you pass parameter values designed for both of the following:
Run a large number of concurrent connections, by using -n100.
Have each connection loop hundreds of times, by using -r500.
However, you might want to start with much smaller values like -n10 and -r50 to ensure that everything is
working.
Script for ostress.exe
This section displays the T-SQL script that is embedded in our ostress.exe command line. The script uses items that
were created by the T-SQL script that you installed earlier.
The following script inserts a sample sales order with five line items into the following memory-optimized tables:
SalesLT.SalesOrderHeader_inmem
SalesLT.SalesOrderDetail_inmem
DECLARE
@i int = 0,
@od SalesLT.SalesOrderDetailType_inmem,
@SalesOrderID int,
@DueDate datetime2 = sysdatetime(),
@CustomerID int = rand() * 8000,
@BillToAddressID int = rand() * 10000,
@ShipToAddressID int = rand() * 10000;

INSERT INTO @od

SELECT OrderQty, ProductID
FROM Demo.DemoSalesOrderDetailSeed
WHERE OrderID= cast((rand()*60) as int);

WHILE (@i < 20)

begin;
EXECUTE SalesLT.usp_InsertSalesOrder_inmem @SalesOrderID OUTPUT,
@DueDate, @CustomerID, @BillToAddressID, @ShipToAddressID, @od;
SET @i = @i + 1;
end

To make the _ondisk version of the preceding T-SQL script for ostress.exe, you would simply replace both
occurrences of the _inmem substring with _ondisk. These replacements affect the names of tables and stored
procedures.
Install RML utilities and ostress
Ideally, you would plan to run ostress.exe on an Azure virtual machine (VM). You would create an Azure VM in the
same Azure geographic region where your AdventureWorksLT database resides. But you can run ostress.exe on
your laptop instead.
On the VM, or on whatever host you choose, install the Replay Markup Language (RML) utilities. The utilities
include ostress.exe.
For more information, see:
The ostress.exe discussion in Sample Database for In-Memory OLTP.
Sample Database for In-Memory OLTP.
The blog for installing ostress.exe.
Run the _inmem stress workload first
You can use an RML Cmd Prompt window to run our ostress.exe command line. The command-line parameters
direct ostress to:
Run 100 connections concurrently (-n100).
Have each connection run the T-SQL script 50 times (-r50).

ostress.exe -n100 -r50 -S<servername>.database.windows.net -U<login> -P<password> -d<database> -q -Q"DECLARE @i int = 0, @od
SalesLT.SalesOrderDetailType_inmem, @SalesOrderID int, @DueDate datetime2 = sysdatetime(), @CustomerID int = rand() * 8000,
@BillToAddressID int = rand() * 10000, @ShipToAddressID int = rand()* 10000; INSERT INTO @od SELECT OrderQty, ProductID FROM
Demo.DemoSalesOrderDetailSeed WHERE OrderID= cast((rand()*60) as int); WHILE (@i < 20) begin; EXECUTE
SalesLT.usp_InsertSalesOrder_inmem @SalesOrderID OUTPUT, @DueDate, @CustomerID, @BillToAddressID, @ShipToAddressID, @od; set
@i += 1; end"

To run the preceding ostress.exe command line:

1. Reset the database data content by running the following command in SSMS to delete all the data that was
inserted by any previous runs:
EXECUTE Demo.usp_DemoReset;

2. Copy the text of the preceding ostress.exe command line to your clipboard.
3. Replace the <placeholders> for the parameters -S -U -P -d with the correct real values.
4. Run your edited command line in an RML Cmd window.
Result is a duration
When ostress.exe finishes, it writes the run duration as its final line of output in the RML Cmd window. For
example, a shorter test run lasted about 1.5 minutes:
11/12/15 00:35:00.873 [0x000030A8] OSTRESS exiting normally, elapsed time: 00:01:31.867

Reset, edit for _ondisk, then rerun

After you have the result from the _inmem run, perform the following steps for the _ondisk run:
1. Reset the database by running the following command in SSMS to delete all the data that was inserted by
the previous run:

EXECUTE Demo.usp_DemoReset;

2. Edit the ostress.exe command line to replace all _inmem with _ondisk.
3. Rerun ostress.exe for the second time, and capture the duration result.
4. Again, reset the database (for responsibly deleting what can be a large amount of test data).
Expected comparison results
Our In-Memory tests have shown that performance improved by nine times for this simplistic workload, with
ostress running on an Azure VM in the same Azure region as the database.

2. Install the In-Memory Analytics sample

In this section, you compare the IO and statistics results when you're using a columnstore index versus a
traditional b-tree index.
For real-time analytics on an OLTP workload, it's often best to use a nonclustered columnstore index. For details,
see Columnstore Indexes Described.
Prepare the columnstore analytics test
1. Use the Azure portal to create a fresh AdventureWorksLT database from the sample.
Use that exact name.
Choose any Premium service tier.
2. Copy the sql_in-memory_analytics_sample to your clipboard.
The T-SQL script creates the necessary In-Memory objects in the AdventureWorksLT sample database
that you created in step 1.
The script creates the Dimension table and two fact tables. The fact tables are populated with 3.5 million
rows each.
The script might take 15 minutes to complete.
3. Paste the T-SQL script into SSMS, and then execute the script. The COLUMNSTORE keyword in the CREATE
INDEX statement is crucial, as in:
CREATE NONCLUSTERED COLUMNSTORE INDEX ...;

4. Set AdventureWorksLT to compatibility level 130:

ALTER DATABASE AdventureworksLT SET compatibility_level = 130;

Level 130 is not directly related to In-Memory features. But level 130 generally provides faster query
performance than 120.
Key tables and columnstore indexes
dbo.FactResellerSalesXL_CCI is a table that has a clustered columnstore index, which has advanced
compression at the data level.
dbo.FactResellerSalesXL_PageCompressed is a table that has an equivalent regular clustered index, which is
compressed only at the page level.
Key queries to compare the columnstore index
There are several T-SQL query types that you can run to see performance improvements. In step 2 in the T-SQL
script, pay attention to this pair of queries. They differ only on one line:
FROM FactResellerSalesXL_PageCompressed a
FROM FactResellerSalesXL_CCI a

A clustered columnstore index is in the FactResellerSalesXL_CCI table.

The following T-SQL script excerpt prints statistics for IO and TIME for the query of each table.
/*********************************************************************
Step 2 -- Overview
-- Page Compressed BTree table v/s Columnstore table performance differences
-- Enable actual Query Plan in order to see Plan differences when Executing
*/
-- Ensure Database is in 130 compatibility mode
ALTER DATABASE AdventureworksLT SET compatibility_level = 130
GO

-- Execute a typical query that joins the Fact Table with dimension tables
-- Note this query will run on the Page Compressed table, Note down the time
SET STATISTICS IO ON
SET STATISTICS TIME ON
GO

SELECT c.Year
,e.ProductCategoryKey
,FirstName + ' ' + LastName AS FullName
,count(SalesOrderNumber) AS NumSales
,sum(SalesAmount) AS TotalSalesAmt
,Avg(SalesAmount) AS AvgSalesAmt
,count(DISTINCT SalesOrderNumber) AS NumOrders
,count(DISTINCT a.CustomerKey) AS CountCustomers
FROM FactResellerSalesXL_PageCompressed a
INNER JOIN DimProduct b ON b.ProductKey = a.ProductKey
INNER JOIN DimCustomer d ON d.CustomerKey = a.CustomerKey
Inner JOIN DimProductSubCategory e on e.ProductSubcategoryKey = b.ProductSubcategoryKey
INNER JOIN DimDate c ON c.DateKey = a.OrderDateKey
GROUP BY e.ProductCategoryKey,c.Year,d.CustomerKey,d.FirstName,d.LastName
GO
SET STATISTICS IO OFF
SET STATISTICS TIME OFF
GO

-- This is the same Prior query on a table with a clustered columnstore index CCI
-- The comparison numbers are even more dramatic the larger the table is (this is an 11 million row table only)
SET STATISTICS IO ON
SET STATISTICS TIME ON
GO
SELECT c.Year
,e.ProductCategoryKey
,FirstName + ' ' + LastName AS FullName
,count(SalesOrderNumber) AS NumSales
,sum(SalesAmount) AS TotalSalesAmt
,Avg(SalesAmount) AS AvgSalesAmt
,count(DISTINCT SalesOrderNumber) AS NumOrders
,count(DISTINCT a.CustomerKey) AS CountCustomers
FROM FactResellerSalesXL_CCI a
INNER JOIN DimProduct b ON b.ProductKey = a.ProductKey
INNER JOIN DimCustomer d ON d.CustomerKey = a.CustomerKey
Inner JOIN DimProductSubCategory e on e.ProductSubcategoryKey = b.ProductSubcategoryKey
INNER JOIN DimDate c ON c.DateKey = a.OrderDateKey
GROUP BY e.ProductCategoryKey,c.Year,d.CustomerKey,d.FirstName,d.LastName
GO

SET STATISTICS IO OFF

SET STATISTICS TIME OFF
GO

In a database with the P2 pricing tier, you can expect about nine times the performance gain for this query by
using the clustered columnstore index compared with the traditional index. With P15, you can expect about 57
times the performance gain by using the columnstore index.

Next steps
Quick Start 1: In-Memory OLTP Technologies for Faster T-SQL Performance
Use In-Memory OLTP in an existing Azure SQL application
Monitor In-Memory OLTP storage for In-Memory OLTP

Additional resources
Deeper information
Learn how Quorum doubles key database’s workload while lowering DTU by 70% with In-Memory OLTP in
SQL Database
Learn about In-Memory OLTP
Learn about columnstore indexes
Learn about real-time operational analytics
See Common Workload Patterns and Migration Considerations (which describes workload patterns where
In-Memory OLTP commonly provides significant performance gains)
Application design
In-Memory OLTP (In-Memory Optimization)
Use In-Memory OLTP in an existing Azure SQL application
Tools
Azure portal
SQL Server Management Studio (SSMS)
SQL Server Data Tools (SSDT)
Getting Started with Temporal Tables in Azure SQL
Database
2/2/2017 • 7 min to read • Edit Online

Temporal Tables are a new programmability feature of Azure SQL Database that allows you to track and analyze
the full history of changes in your data, without the need for custom coding. Temporal Tables keep data closely
related to time context so that stored facts can be interpreted as valid only within the specific period. This property
of Temporal Tables allows for efficient time-based analysis and getting insights from data evolution.

Temporal Scenario
This article illustrates the steps to utilize Temporal Tables in an application scenario. Suppose that you want to track
user activity on a new website that is being developed from scratch or on an existing website that you want to
extend with user activity analytics. In this simplified example, we assume that the number of visited web pages
during a period of time is an indicator that needs to be captured and monitored in the website database that is
hosted on Azure SQL Database. The goal of the historical analysis of user activity is to get inputs to redesign
website and provide better experience for the visitors.
The database model for this scenario is very simple - user activity metric is represented with a single integer field,
PageVisited, and is captured along with basic information on the user profile. Additionally, for time based
analysis, you would keep a series of rows for each user, where every row represents the number of pages a
particular user visited within a specific period of time.

Fortunately, you do not need to put any effort in your app to maintain this activity information. With Temporal
Tables, this process is automated - giving you full flexibility during website design and more time to focus on the
data analysis itself. The only thing you have to do is to ensure that WebSiteInfo table is configured as temporal
system-versioned. The exact steps to utilize Temporal Tables in this scenario are described below.

Step 1: Configure tables as temporal

Depending on whether you are starting new development or upgrading existing application, you will either create
temporal tables or modify existing ones by adding temporal attributes. In general case, your scenario can be a mix
of these two options. Perform these action using SQL Server Management Studio (SSMS), SQL Server Data Tools
(SSDT) or any other Transact-SQL development tool.
IMPORTANT
It is recommended that you always use the latest version of Management Studio to remain synchronized with updates to
Microsoft Azure and SQL Database. Update SQL Server Management Studio.

Create new table

Use context menu item “New System-Versioned Table” in SSMS Object Explorer to open the query editor with a
temporal table template script and then use “Specify Values for Template Parameters” (Ctrl+Shift+M) to populate
the template:

In SSDT, chose “Temporal Table (System-Versioned)” template when adding new items to the database project.
That will open table designer and enable you to easily specify the table layout:
You can also a create temporal table by specifying the Transact-SQL statements directly, as shown in the example
below. Note that the mandatory elements of every temporal table are the PERIOD definition and the
SYSTEM_VERSIONING clause with a reference to another user table that will store historical row versions:

CREATE TABLE WebsiteUserInfo

(
[UserID] int NOT NULL PRIMARY KEY CLUSTERED
, [UserName] nvarchar(100) NOT NULL
, [PagesVisited] int NOT NULL
, [ValidFrom] datetime2 (0) GENERATED ALWAYS AS ROW START
, [ValidTo] datetime2 (0) GENERATED ALWAYS AS ROW END
, PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo)
)
WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.WebsiteUserInfoHistory));

When you create system-versioned temporal table, the accompanying history table with the default configuration
is automatically created. The default history table contains a clustered B-tree index on the period columns (end,
start) with page compression enabled. This configuration is optimal for the majority of scenarios in which temporal
tables are used, especially for data auditing.
In this particular case, we aim to perform time-based trend analysis over a longer data history and with bigger data
sets, so the storage choice for the history table is a clustered columnstore index. A clustered columnstore provides
very good compression and performance for analytical queries. Temporal Tables give you the flexibility to
configure indexes on the current and temporal tables completely independently.

NOTE
Columnstore indexes are only available in the premium service tier.

The following script shows how default index on history table can be changed to the clustered columnstore:
CREATE CLUSTERED COLUMNSTORE INDEX IX_WebsiteUserInfoHistory
ON dbo.WebsiteUserInfoHistory
WITH (DROP_EXISTING = ON);

Temporal Tables are represented in the Object Explorer with the specific icon for easier identification, while its
history table is displayed as a child node.

Alter existing table to temporal

Let’s cover the alternative scenario in which the WebsiteUserInfo table already exists, but was not designed to keep
a history of changes. In this case, you can simply extend the existing table to become temporal, as shown in the
following example:

ALTER TABLE WebsiteUserInfo

ADD
ValidFrom datetime2 (0) GENERATED ALWAYS AS ROW START HIDDEN
constraint DF_ValidFrom DEFAULT DATEADD(SECOND, -1, SYSUTCDATETIME())
, ValidTo datetime2 (0) GENERATED ALWAYS AS ROW END HIDDEN
constraint DF_ValidTo DEFAULT '9999.12.31 23:59:59.99'
, PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo);

ALTER TABLE WebsiteUserInfo

SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.WebsiteUserInfoHistory));
GO

CREATE CLUSTERED COLUMNSTORE INDEX IX_WebsiteUserInfoHistory

ON dbo.WebsiteUserInfoHistory
WITH (DROP_EXISTING = ON);

Step 2: Run your workload regularly

The main advantage of Temporal Tables is that you do not need to change or adjust your website in any way to
perform change tracking. Once created, Temporal Tables transparently persist previous row versions every time
you perform modifications on your data.
In order to leverage automatic change tracking for this particular scenario, let’s just update column PagesVisited
every time when user ends her/his session on the website:

UPDATE WebsiteUserInfo SET [PagesVisited] = 5

WHERE [UserID] = 1;

It is important to notice that the update query doesn’t need to know the exact time when the actual operation
occurred nor how historical data will be preserved for future analysis. Both aspects are automatically handled by
the Azure SQL Database. The following diagram illustrates how history data is being generated on every update.

Step 3: Perform historical data analysis

Now when temporal system-versioning is enabled, historical data analysis is just one query away from you. In this
article, we will provide a few examples that address common analysis scenarios - to learn all details, explore
various options introduced with the FOR SYSTEM_TIME clause.
To see the top 10 users ordered by the number of visited web pages as of an hour ago, run this query:

DECLARE @hourAgo datetime2 = DATEADD(HOUR, -1, SYSUTCDATETIME());

SELECT TOP 10 * FROM dbo.WebsiteUserInfo FOR SYSTEM_TIME AS OF @hourAgo
ORDER BY PagesVisited DESC

You can easily modify this query to analyze the site visits as of a day ago, a month ago or at any point in the past
you wish.
To perform basic statistical analysis for the previous day, use the following example:

DECLARE @twoDaysAgo datetime2 = DATEADD(DAY, -2, SYSUTCDATETIME());

DECLARE @aDayAgo datetime2 = DATEADD(DAY, -1, SYSUTCDATETIME());

SELECT UserID, SUM (PagesVisited) as TotalVisitedPages, AVG (PagesVisited) as AverageVisitedPages,

MAX (PagesVisited) AS MaxVisitedPages, MIN (PagesVisited) AS MinVisitedPages,
STDEV (PagesVisited) as StDevViistedPages
FROM dbo.WebsiteUserInfo
FOR SYSTEM_TIME BETWEEN @twoDaysAgo AND @aDayAgo
GROUP BY UserId

To search for activities of a specific user, within a period of time, use the CONTAINED IN clause:
DECLARE @hourAgo datetime2 = DATEADD(HOUR, -1, SYSUTCDATETIME());
DECLARE @twoHoursAgo datetime2 = DATEADD(HOUR, -2, SYSUTCDATETIME());
SELECT * FROM dbo.WebsiteUserInfo
FOR SYSTEM_TIME CONTAINED IN (@twoHoursAgo, @hourAgo)
WHERE [UserID] = 1;

Graphic visualization is especially convenient for temporal queries as you can show trends and usage patterns in
an intuitive way very easily:

Evolving table schema

Typically, you will need to change the temporal table schema while you are doing app development. For that,
simply run regular ALTER TABLE statements and Azure SQL Database will appropriately propagate changes to the
history table. The following script shows how you can add additional attribute for tracking:

/Add new column for tracking source IP address/

ALTER TABLE dbo.WebsiteUserInfo
ADD [IPAddress] varchar(128) NOT NULL CONSTRAINT DF_Address DEFAULT 'N/A';

Similarly, you can change column definition while your workload is active:

/Increase the length of name column/

ALTER TABLE dbo.WebsiteUserInfo
ALTER COLUMN UserName nvarchar(256) NOT NULL;

Finally, you can remove a column that you do not need anymore.

/Drop unnecessary column /

ALTER TABLE dbo.WebsiteUserInfo
DROP COLUMN TemporaryColumn;

Alternatively, use latest SSDT to change temporal table schema while you are connected to the database (online
mode) or as part of the database project (offline mode).

Controlling retention of historical data

With system-versioned temporal tables, the history table may increase the database size more than regular tables.
A large and ever-growing history table can become an issue both due to pure storage costs as well as imposing a
performance tax on temporal querying. Hence, developing a data retention policy for managing data in the history
table is an important aspect of planning and managing the lifecycle of every temporal table. With Azure SQL
Database, you have the following approaches for managing historical data in the temporal table:
Table Partitioning
Custom Cleanup Script

Next steps
For detailed information on Temporal Tables, check out MSDN documentation. Visit Channel 9 to hear a real
customer temporal implemenation success story and watch a live temporal demonstration.
Manage historical data in Temporal Tables with
retention policy
4/19/2017 • 7 min to read • Edit Online

Temporal Tables may increase database size more than regular tables, especially if you retain historical data for a
longer period of time. Hence, retention policy for historical data is an important aspect of planning and managing
the lifecycle of every temporal table. Temporal Tables in Azure SQL Database come with easy-to-use retention
mechanism that helps you accomplish this task.
Temporal history retention can be configured at the individual table level, which allows users to create flexible
aging polices. Applying temporal retention is simple: it requires only one parameter to be set during table creation
or schema change.
After you define retention policy, Azure SQL Database starts checking regularly if there are historical rows that are
eligible for automatic data cleanup. Identification of matching rows and their removal from the history table occur
transparently, in the background task that is scheduled and run by the system. Age condition for the history table
rows is checked based on the column representing end of SYSTEM_TIME period. If retention period, for example, is
set to six months, table rows eligible for cleanup satisfy the following condition:

ValidTo < DATEADD (MONTH, -6, SYSUTCDATETIME())

In the preceding example, we assumed that ValidTo column corresponds to the end of SYSTEM_TIME period.

How to configure retention policy?

Before you configure retention policy for a temporal table, check first whether temporal historical retention is
enabled at the database level.

SELECT is_temporal_history_retention_enabled, name

FROM sys.databases

Database flag is_temporal_history_retention_enabled is set to ON by default, but users can change it with
ALTER DATABASE statement. It is also automatically set to OFF after point in time restore operation. To enable
temporal history retention cleanup for your database, execute the following statement:

ALTER DATABASE <myDB>

SET TEMPORAL_HISTORY_RETENTION ON

IMPORTANT
You can configure retention for temporal tables even if is_temporal_history_retention_enabled is OFF, but automatic
cleanup for aged rows is not triggered in that case.

Retention policy is configured during table creation by specifying value for the HISTORY_RETENTION_PERIOD
parameter:
CREATE TABLE dbo.WebsiteUserInfo
(
[UserID] int NOT NULL PRIMARY KEY CLUSTERED
, [UserName] nvarchar(100) NOT NULL
, [PagesVisited] int NOT NULL
, [ValidFrom] datetime2 (0) GENERATED ALWAYS AS ROW START
, [ValidTo] datetime2 (0) GENERATED ALWAYS AS ROW END
, PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo)
)
WITH
(
SYSTEM_VERSIONING = ON
(
HISTORY_TABLE = dbo.WebsiteUserInfoHistory,
HISTORY_RETENTION_PERIOD = 6 MONTHS
)
);

Azure SQL Database allows you to specify retention period by using different time units: DAYS, WEEKS, MONTHS,
and YEARS. If HISTORY_RETENTION_PERIOD is omitted, INFINITE retention is assumed. You can also use INFINITE
keyword explicitly.
In some scenarios, you may want to configure retention after table creation, or to change previously configured
value. In that case use ALTER TABLE statement:

ALTER TABLE dbo.WebsiteUserInfo

SET (SYSTEM_VERSIONING = ON (HISTORY_RETENTION_PERIOD = 9 MONTHS));

IMPORTANT
Setting SYSTEM_VERSIONING to OFF does not preserve retention period value. Setting SYSTEM_VERSIONING to ON without
HISTORY_RETENTION_PERIOD specified explicitly results in the INFINITE retention period.

To review current state of the retention policy, use the following query that joins temporal retention enablement
flag at the database level with retention periods for individual tables:

SELECT DB.is_temporal_history_retention_enabled,
SCHEMA_NAME(T1.schema_id) AS TemporalTableSchema,
T1.name as TemporalTableName, SCHEMA_NAME(T2.schema_id) AS HistoryTableSchema,
T2.name as HistoryTableName,T1.history_retention_period,
T1.history_retention_period_unit_desc
FROM sys.tables T1
OUTER APPLY (select is_temporal_history_retention_enabled from sys.databases
where name = DB_NAME()) AS DB
LEFT JOIN sys.tables T2
ON T1.history_table_id = T2.object_id WHERE T1.temporal_type = 2

How SQL Database deletes aged rows?

The cleanup process depends on the index layout of the history table. It is important to notice that only history
tables with a clustered index (B-tree or columnstore) can have finite retention policy configured. A background task
is created to perform aged data cleanup for all temporal tables with finite retention period. Cleanup logic for the
rowstore (B-tree) clustered index deletes aged row in smaller chunks (up to 10K) minimizing pressure on database
log and I/O subsystem. Although cleanup logic utilizes required B-tree index, order of deletions for the rows older
than retention period cannot be firmly guaranteed. Hence, do not take any dependency on the cleanup order in
your applications.
The cleanup task for the clustered columnstore removes entire row groups at once (typically contain 1 million of
rows each), which is very efficient, especially when historical data is generated at a high pace.

Excellent data compression and efficient retention cleanup makes clustered columnstore index a perfect choice for
scenarios when your workload rapidly generates high amount of historical data. That pattern is typical for intensive
transactional processing workloads that use temporal tables for change tracking and auditing, trend analysis, or IoT
data ingestion.

Index considerations
The cleanup task for tables with rowstore clustered index requires index to start with the column corresponding the
end of SYSTEM_TIME period. If such index doesn't exist, you cannot configure a finite retention period:
Msg 13765, Level 16, State 1
Setting finite retention period failed on system-versioned temporal table
'temporalstagetestdb.dbo.WebsiteUserInfo' because the history table
'temporalstagetestdb.dbo.WebsiteUserInfoHistory' does not contain required clustered index. Consider creating a
clustered columnstore or B-tree index starting with the column that matches end of SYSTEM_TIME period, on the
history table.
It is important to notice that the default history table created by Azure SQL Database already has clustered index,
which is compliant for retention policy. If you try to remove that index on a table with finite retention period,
operation fails with the following error:
Msg 13766, Level 16, State 1
Cannot drop the clustered index 'WebsiteUserInfoHistory.IX_WebsiteUserInfoHistory' because it is being used for
automatic cleanup of aged data. Consider setting HISTORY_RETENTION_PERIOD to INFINITE on the corresponding
system-versioned temporal table if you need to drop this index.
Cleanup on the clustered columnstore index works optimally if historical rows are inserted in the ascending order
(ordered by the end of period column), which is always the case when the history table is populated exclusively by
the SYSTEM_VERSIONIOING mechanism. If rows in the history table are not ordered by end of period column
(which may be the case if you migrated existing historical data), you should re-create clustered columnstore index
on top of B-tree rowstore index that is properly ordered, to achieve optimal performance.
Avoid rebuilding clustered columnstore index on the history table with the finite retention period, because it may
change ordering in the row groups naturally imposed by the system-versioning operation. If you need to rebuild
clustered columnstore index on the history table, do that by re-creating it on top of compliant B-tree index,
preserving ordering in the rowgroups necessary for regular data cleanup. The same approach should be taken if
you create temporal table with existing history table that has clustered column index without guaranteed data
order:
/*Create B-tree ordered by the end of period column*/
CREATE CLUSTERED INDEX IX_WebsiteUserInfoHistory ON WebsiteUserInfoHistory (ValidTo)
WITH (DROP_EXISTING = ON);
GO
/*Re-create clustered columnstore index*/
CREATE CLUSTERED COLUMNSTORE INDEX IX_WebsiteUserInfoHistory ON WebsiteUserInfoHistory
WITH (DROP_EXISTING = ON);

When finite retention period is configured for the history table with the clustered columnstore index, you cannot
create additional non-clustered B-tree indexes on that table:

CREATE NONCLUSTERED INDEX IX_WebHistNCI ON WebsiteUserInfoHistory ([UserName])

An attempt to execute above statement fails with the following error:

Msg 13772, Level 16, State 1
Cannot create non-clustered index on a temporal history table 'WebsiteUserInfoHistory' since it has finite retention
period and clustered columnstore index defined.

Querying tables with retention policy

All queries on the temporal table automatically filter out historical rows matching finite retention policy, to avoid
unpredictable and inconsistent results, since aged rows can be deleted by the cleanup task, at any point in time and
in arbitrary order.
The following picture shows the query plan for a simple query:

SELECT * FROM dbo.WebsiteUserInfo FROM SYSTEM_TIME ALL;

The query plan includes additional filter applied to end of period column (ValidTo) in the Clustered Index Scan
operator on the history table (highlighted). This example assumes that one MONTH retention period was set on
WebsiteUserInfo table.
However, if you query history table directly, you may see rows that are older than specified retention period, but
without any guarantee for repeatable query results. The following picture shows query execution plan for the query
on the history table without additional filters applied:
Do not rely your business logic on reading history table beyond retention period as you may get inconsistent or
unexpected results. We recommend that you use temporal queries with FOR SYSTEM_TIME clause for analyzing
data in temporal tables.

Point in time restore considerations

When you create new database by restoring existing database to a specific point in time, it has temporal retention
disabled at the database level. (is_temporal_history_retention_enabled flag set to OFF). This functionality allows
you to examine all historical rows upon restore, without worrying that aged rows are removed before you get to
query them. You can use it to inspect historical data beyond configured retention period.
Say that a temporal table has one MONTH retention period specified. If your database was created in Premium
Service tier, you would be able to create database copy with the database state up to 35 days back in the past. That
effectively would allow you to analyze historical rows that are up to 65 days old by querying the history table
directly.
If you want to activate temporal retention cleanup, run the following Transact-SQL statement after point in time
restore:

ALTER DATABASE <myDB>

SET TEMPORAL_HISTORY_RETENTION ON
Next steps
To learn how to use Temporal Tables in your applications, check out Getting Started with Temporal Tables in Azure
SQL Database.
Visit Channel 9 to hear a real customer temporal implementation success story and watch a live temporal
demonstration.
For detailed information about Temporal Tables, review MSDN documentation.
SQL Server database migration to SQL Database in
the cloud
4/17/2017 • 5 min to read • Edit Online

In this article, you learn about the two primary methods for migrating a SQL Server 2005 or later database to
Azure SQL Database. The first method is simpler but requires some, possibly substantial, downtime during the
migration. The second method is more complex, but substantially eliminates downtime during the migration.
In both cases, you need to ensure that the source database is compatible with Azure SQL Database using the Data
Migration Assistant (DMA). SQL Database V12 is approaching feature parity with SQL Server, other than issues
related to server-level and cross-database operations. Databases and applications that rely on partially supported
or unsupported functions need some re-engineering to fix these incompatibilities before the SQL Server database
can be migrated.

NOTE
To migrate a non-SQL Server database, including Microsoft Access, Sybase, MySQL Oracle, and DB2 to Azure SQL Database,
see SQL Server Migration Assistant.

Method 1: Migration with downtime during the migration

Use this method if you can afford some downtime or you are performing a test migration of a production
database for later migration. For a tutorial, see Migrate a SQL Server database.
The following list contains the general workflow for a SQL Server database migration using this method.

1. Assess the database for compatibility using the latest version of Data Migration Assistant (DMA).
2. Prepare any necessary fixes as Transact-SQL scripts.
3. Make a transactionally consistent copy of the source database being migrated - and ensure no further changes
are being made to the source database (or you can manually apply any such changes after the migration
completes). There are many methods to quiesce a database, from disabling client connectivity to creating a
database snapshot.
4. Deploy the Transact-SQL scripts to apply the fixes to the database copy.
5. Export the database copy to a .BACPAC file on a local drive.
6. Import the .BACPAC file as a new Azure SQL database using any of several BACPAC import tools, with
SQLPackage.exe being the recommended tool for best performance.
Optimizing data transfer performance during migration
The following list contains recommendations for best performance during the import process.
Choose the highest service level and performance tier that your budget allows to maximize the transfer
performance. You can scale down after the migration completes to save money.
Minimize the distance between your .BACPAC file and the destination data center.
Disable auto-statistics during migration
Partition tables and indexes
Drop indexed views, and recreate them once finished
Remove rarely queried historical data to another database and migrate this historical data to a separate Azure
SQL database. You can then query this historical data using elastic queries.
Optimize performance after the migration completes
Update statistics with full scan after the migration is completed.

Method 2: Use Transactional Replication

When you cannot afford to remove your SQL Server database from production while the migration is occurring,
you can use SQL Server transactional replication as your migration solution. To use this method, the source
database must meet the requirements for transactional replication and be compatible for Azure SQL Database.
To use this solution, you configure your Azure SQL Database as a subscriber to the SQL Server instance that you
wish to migrate. The transactional replication distributor synchronizes data from the database to be synchronized
(the publisher) while new transactions continue occur.
With transactional replication, all changes to your data or schema show up in your Azure SQL Database. Once the
synchronization is complete and you are ready to migrate, change the connection string of your applications to
point them to your Azure SQL Database. Once transactional replication drains any changes left on your source
database and all your applications point to Azure DB, you can uninstall transactional replication. Your Azure SQL
Database is now your production system.
TIP
You can also use transactional replication to migrate a subset of your source database. The publication that you replicate to
Azure SQL Database can be limited to a subset of the tables in the database being replicated. For each table being
replicated, you can limit the data to a subset of the rows and/or a subset of the columns.

Migration to SQL Database using Transaction Replication workflow

IMPORTANT
Use the latest version of SQL Server Management Studio to remain synchronized with updates to Microsoft Azure and SQL
Database. Older versions of SQL Server Management Studio cannot set up SQL Database as a subscriber. Update SQL
Server Management Studio.

1. Set up Distribution
Using SQL Server Management Studio (SSMS)
Using Transact-SQL
2. Create Publication
Using SQL Server Management Studio (SSMS)
Using Transact-SQL
3. Create Subscription
Using SQL Server Management Studio (SSMS)
Using Transact-SQL
Some tips and differences for migrating to SQL Database
1. Use a local distributor
This causes a performance impact on the server.
If the performance impact is unacceptable, you can use another server but it adds complexity in
management and administration.
2. When selecting a snapshot folder, make sure the folder you select is large enough to hold a BCP of every table
you want to replicate.
3. Snapshot creation locks the associated tables until it is complete, so schedule your snapshot appropriately.
4. Only push subscriptions are supported in Azure SQL Database. You can only add subscribers from the source
database.

Resolving database migration compatibility issues

There are a wide variety of compatibility issues that you might encounter, depending both on the version of SQL
Server in the source database and the complexity of the database you are migrating. Older versions of SQL Server
have more compatibility issues. Use the following resources, in addition to a targeted Internet search using your
search engine of choices:
SQL Server database features not supported in Azure SQL Database
Discontinued Database Engine Functionality in SQL Server 2016
Discontinued Database Engine Functionality in SQL Server 2014
Discontinued Database Engine Functionality in SQL Server 2012
Discontinued Database Engine Functionality in SQL Server 2008 R2
Discontinued Database Engine Functionality in SQL Server 2005
In addition to searching the Internet and using these resources, use the MSDN SQL Server community forums or
StackOverflow.

Next steps
Use the script on the Azure SQL EMEA Engineers blog to Monitor tempdb usage during migration.
Use the script on the Azure SQL EMEA Engineers blog to Monitor the transaction log space of your database
while migration is occurring.
For a SQL Server Customer Advisory Team blog about migrating using BACPAC files, see Migrating from SQL
Server to Azure SQL Database using BACPAC Files.
For information about working with UTC time after migration, see Modifying the default time zone for your
local time zone.
For information about changing the default language of a database after migration, see How to change the
default language of Azure SQL Database.
Resolving Transact-SQL differences during migration
to SQL Database
4/10/2017 • 5 min to read • Edit Online

When migrating your database from SQL Server to Azure SQL Server, you may discover that your database
requires some re-engineering before the SQL Server can be migrated. This topic provides guidance to assist you
in both performing this re-engineering and understanding the underlying reasons why the re-engineering is
necessary. To detect incompatibilities, use the Data Migration Assistant (DMA).

Overview
Most Transact-SQL features that applications use are fully supported in both Microsoft SQL Server and Azure
SQL Database. For example, the core SQL components such as data types, operators, string, arithmetic, logical,
and cursor functions, work identically in SQL Server and SQL Database. There are, however, a few T-SQL
differences in DDL (data-definition language) and DML (data manipulation language) elements resulting in T-
SQL statements and queries that are only partially supported (which we discuss later in this topic).
In addition, there are some features and syntax that is not supported at all because Azure SQL Database is
designed to isolate features from dependencies on the master database and the operating system. As such, most
server-level activities are inappropriate for SQL Database. T-SQL statements and options are not available if they
configure server-level options, operating system components, or specify file system configuration. When such
capabilities are required, an appropriate alternative is often available in some other way from SQL Database or
from another Azure feature or service.
For example, high availability is built into Azure, so configuring Always On is not necessary (although you may
want to configure Active Geo-Replication for faster recovery in the event of a disaster). So, T-SQL statements
related to availability groups are not supported by SQL Database, and the dynamic management views related to
Always On are also not supported.
For a list of the features that are supported and unsupported by SQL Database, see Azure SQL Database feature
comparison. The list on this page supplements that guidelines and features topic, and focuses on Transact-SQL
statements.

Transact-SQL syntax statements with partial differences

The core DDL (data definition language) statements are available, but some DDL statements have extensions
related to disk placement and unsupported features.
CREATE and ALTER DATABASE statements have over three dozen options. The statements include file
placement, FILESTREAM, and service broker options that only apply to SQL Server. This may not matter if you
create databases before you migrate, but if you are migrating T-SQL code that creates databases you should
compare CREATE DATABASE (Azure SQL Database) with the SQL Server syntax at CREATE DATABASE (SQL
Server Transact-SQL) to make sure all the options you use are supported. CREATE DATABASE for Azure SQL
Database also has service objective and elastic scale options that apply only to SQL Database.
The CREATE and ALTER TABLE statements have FileTable options that cannot be used on SQL Database
because FILESTREAM is not supported.
CREATE and ALTER login statements are supported but SQL Database does not offer all the options. To make
your database more portable, SQL Database encourages using contained database users instead of logins
whenever possible. For more information, see CREATE/ALTER LOGIN and Controlling and granting database
access.

Transact-SQL syntax not supported in SQL Database

In addition to Transact-SQL statements related to the unsupported features described in Azure SQL Database
feature comparison, the following statements and groups of statements, are not supported. As such, if your
database to be migrated is using any of the following features, re-engineer your T-SQL to eliminate these T-SQL
features and statements.
Collation of system objects
Connection related: Endpoint statements, ORIGINAL_DB_NAME . SQL Database does not support Windows
authentication, but does support the similar Azure Active Directory authentication. Some authentication types
require the latest version of SSMS. For more information, see Connecting to SQL Database or SQL Data
Warehouse By Using Azure Active Directory Authentication.
Cross database queries using three or four part names. (Read-only cross-database queries are supported by
using elastic database query.)
Cross database ownership chaining, TRUSTWORTHY setting
DATABASEPROPERTY Use DATABASEPROPERTYEX instead.
EXECUTE AS LOGIN Use 'EXECUTE AS USER' instead.
Encryption is supported except for extensible key management
Eventing: Events, event notifications, query notifications
File placement: Syntax related to database file placement, size, and database files that are automatically
managed by Microsoft Azure.
High availability: Syntax related to high availability, which is managed through your Microsoft Azure account.
This includes syntax for backup, restore, Always On, database mirroring, log shipping, recovery modes.
Log reader: Syntax that relies upon the log reader, which is not available on SQL Database: Push Replication,
Change Data Capture. SQL Database can be a subscriber of a push replication article.
Functions: fn_get_sql , fn_virtualfilestats , fn_virtualservernodes
Global temporary tables
Hardware: Syntax related to hardware-related server settings: such as memory, worker threads, CPU affinity,
trace flags. Use service levels instead.
HAS_DBACCESS
KILL STATS JOB
OPENQUERY , OPENROWSET , OPENDATASOURCE , and four-part names
.NET Framework: CLR integration with SQL Server
Semantic search
Server credentials: Use database scoped credentials instead.
Server-level items: Server roles, IS_SRVROLEMEMBER , sys.login_token . GRANT , REVOKE , and DENY of server
level permissions are not available though some are replaced by database-level permissions. Some useful
server-level DMVs have equivalent database-level DMVs.
SET REMOTE_PROC_TRANSACTIONS
SHUTDOWN
sp_addmessage
options and
sp_configure RECONFIGURE . Some options are available using ALTER DATABASE SCOPED
CONFIGURATION.
sp_helpuser
sp_migrate_user_to_contained
SQL Server Agent: Syntax that relies upon the SQL Server Agent or the MSDB database: alerts, operators,
central management servers. Use scripting, such as Azure PowerShell instead.
SQL Server audit: Use SQL Database auditing instead.
SQL Server trace
Trace flags: Some trace flag items have been moved to compatibility modes.
Transact-SQL debugging
Triggers: Server-scoped or logon triggers
USE statement: To change the database context to a different database, you must make a new connection to
the new database.

Full Transact-SQL reference

For more information about Transact-SQL grammar, usage, and examples, see Transact-SQL Reference
(Database Engine) in SQL Server Books Online.
About the "Applies to" tags
The Transact-SQL reference includes topics related to SQL Server versions 2008 to the present. Below the topic
title there is an icon bar, listing the four SQL Server platforms, and indicating applicability. For example,
availability groups were introduced in SQL Server 2012. The CREATE AVAILABILTY GROUP topic indicates that
the statement applies to SQL Server (starting with 2012). The statement does not apply to SQL Server 2008,
SQL Server 2008 R2, Azure SQL Database, Azure SQL Data Warehouse, or Parallel Data Warehouse.
In some cases, the general subject of a topic can be used in a product, but there are minor differences between
products. The differences are indicated at midpoints in the topic as appropriate. In some cases, the general
subject of a topic can be used in a product, but there are minor differences between products. The differences are
indicated at midpoints in the topic as appropriate. For example the CREATE TRIGGER topic is available in SQL
Database. But the ALL SERVER option for server-level triggers, indicates that server-level triggers cannot be
used in SQL Database. Use database-level triggers instead.

Next steps
For a list of the features that are supported and unsupported by SQL Database, see Azure SQL Database feature
comparison. The list on this page supplements that guidelines and features topic, and focuses on Transact-SQL
statements.
Copy an Azure SQL database
4/12/2017 • 5 min to read • Edit Online

Azure SQL Database provides several methods for creating a transactionally consistent copy of an existing Azure
SQL database on either the same server or a different server. You can copy a SQL database using the Azure
portal, PowerShell, or T-SQL.

Overview
A database copy is a snapshot of the source database as of the time of the copy request. You can select the same
server or a different server, its service tier and performance level, a different performance level within the same
service tier (edition). After the copy is complete, the copy becomes a fully functional, independent database. At
this point, you can upgrade or downgrade it to any edition. The logins, users, and permissions can be managed
independently.

Logins in the database copy

When you copy a database to the same logical server, the same logins can be used on both databases. The
security principal you use to copy the database becomes the database owner (DBO) on the new database. All
database users, their permissions, and their security identifiers (SIDs) are copied to the database copy.
When you copy a database to a different logical server, the security principal on the new server becomes the
database owner on the new database. If you use contained database users for data access, ensure that both the
primary and secondary databases always have the same user credentials, so after the copy completes you can
immediately access it with the same credentials.
If you use Azure Active Directory, you can completely eliminate the need for managing credentials in the copy.
However, when you copy the database to a new server, the login-based access may not work because the logins
do not exist on the new server. See How to manage Azure SQL database security after disaster recovery to learn
about managing logins when copying a database to a different logical server.
After the copying succeeds and before other users are remapped, only the login that initiated the copying, the
database owner (DBO), can log on to the new database. To resolve logins after the copy operation completes, see
Resolve logins

Database copy using the Azure portal

To copy a database using the Azure portal, open the page for your database and click Copy on the toolbar.
Database copy using PowerShell
To copy a database using PowerShell, use the New-AzureRmSqlDatabaseCopy cmdlet.

New-AzureRmSqlDatabaseCopy -ResourceGroupName "myResourceGroup" `

-ServerName $sourceserver `
-DatabaseName "MySampleDatabase" `
-CopyResourceGroupName "myResourceGroup" `
-CopyServerName $targetserver `
-CopyDatabaseName "CopyOfMySampleDatabase"

For a complete sample script, see Copy a database to a new server

Database copy using Transact-SQL

Log on to the master database using the server-level principal login or the login that created the database you
want to copy. Logins that are not the server-level principal must be members of the dbmanager role to copy
databases. For more information about logins and connecting to the server, see Manage logins.
Start copying the source database with the CREATE DATABASE statement. Executing this statement initiates the
database copying process. Because copying a database is an asynchronous process, the CREATE DATABASE
statement returns before the database completes copying.
Copy a SQL database to the same server
Log on to the master database using the server-level principal login or the login that created the database you
want to copy. Logins that are not the server-level principal must be members of the dbmanager role to copy
databases.
This command copies Database1 on to a new database named Database2 on the same server. Depending on the
size of your database the copy operation may take some time to complete.

-- Execute on the master database.

-- Start copying.
CREATE DATABASE Database1_copy AS COPY OF Database1;

Copy a SQL database to a different server

Log on to the master database of the destination server, the Azure SQL Database server where the new database
is to be created. Use a login that has the same name and password as the database owner (DBO) of the source
database on the source Azure SQL Database server. The login on the destination server must also be a member
of the dbmanager role or be the server-level principal login.
This command copies Database1 on server1- to a new database named Database2 on server2. Depending on the
size of your database the copy operation may take some time to complete.

-- Execute on the master database of the target server (server2)

-- Start copying from Server1 to Server2
CREATE DATABASE Database1_copy AS COPY OF server1.Database1;

Monitor the progress of the copy operation

Monitor the copying process by querying the sys.databases and sys.dm_database_copies views. While the
copying is in progress, the state_desc column of the sys.databases view for the new database is set to COPYING.
If the copying fails, the state_desc column of the sys.databases view for the new database is set to SUSPECT. In
this case, execute the DROP statement on the new database and try again later.
If the copying succeeds, the state_desc column of the sys.databases view for the new database is set to
ONLINE. In this case, the copying is complete and the new database is a regular database, able to be changed
independent of the source database.

NOTE
If you decide to cancel the copying while it is in progress, execute the DROP DATABASE statement on the new
database. Alternatively, executing the DROP DATABASE statement on the source database also cancels the copying
process.

Resolve logins
After the new database is online on the destination server, use the ALTER USER statement to remap the users
from the new database to logins on the destination server. To resolve orphaned users, see Troubleshoot
Orphaned Users. See also How to manage Azure SQL database security after disaster recovery.
All users in the new database maintain the permissions that they had in the source database. The user who
initiated the database copy becomes the database owner of the new database and is assigned a new security
identifier (SID). After the copying succeeds and before other users are remapped, only the login that initiated the
copying, the database owner (DBO), can log on to the new database.
See also How to manage Azure SQL database security after disaster recovery to learn about managing users and
logins when copying a database to a different logical server.

Next steps
For information on logins, see Manage logins and How to manage Azure SQL database security after disaster
recovery.
To export a database, see Export the database to a BACPAC
Import a BACPAC file to a new Azure SQL Database
4/12/2017 • 4 min to read • Edit Online

This article discusses importing a BACPAC file to a new Azure SQL database. This article discusses using the
following methods:
The Azure portal
the SqlPackage command-line utility
the New-AzureRmSqlDatabaseImport cmdlet

Overview
When you need to import a database from an archive or when migrating from another platform, you can import
the database schema and data from a BACPAC file. A BACPAC file is a ZIP file with an extension of BACPAC
containing the metadata and data from a SQL Server database. A BACPAC file can be imported from Azure blob
storage (standard storage only) or from local storage in an on-premises location. To maximize the import speed,
we recommend that you specify a higher service tier and performance level, such as a P6, and then scale to down
as appropriate after the import is successful. Also, the database compatibility level after the import is based on the
compatibility level of the source database.

IMPORTANT
After you migrate your database to Azure SQL Database, you can choose to operate the database at its current
compatibility level (level 100 for the AdventureWorks2008R2 database) or at a higher level. For more information on the
implications and options for operating a database at a specific compatibility level, see ALTER DATABASE Compatibility Level.
See also ALTER DATABASE SCOPED CONFIGURATION for information about additional database-level settings related to
compatibility levels. >

NOTE
To import a BACPAC to a new database, you must first create an Azure SQL Database logical server. For a tutorial showing
you how to migrate a SQL Server database to Azure SQL Database using SQLPackage, see Migrate a SQL Server Database

Azure portal
This article provides directions for creating an Azure SQL database from a BACPAC file stored in Azure blob
storage using the Azure portal. Import using the Azure portal only supports importing a BACPAC file from Azure
blob storage.
To import a database using the Azure portal, open the page for your database and click Import on the toolbar.
Specify the *.bacpac filename, provide the Azure storage account and container for the bacpac, and provide the
credentials to connect to the source database.
To monitor the progress of the import operation, open the page for the logical server containing the database
being imported. Scroll down to Operations and then click Import/Export history.
Monitor the progress of the import operation
1. Click SQL servers.
2. Click the server you are restoring to.
3. In the SQL server blade, in the Operations area, click Import/Export history:
4. To verify the database is live on the server, click SQL databases and verify the new database is Online.

SQLPackage
To import a SQL database using the SqlPackage command-line utility, see Import parameters and properties. The
SQLPackage utility ships with the latest versions of SQL Server Management Studio and SQL Server Data Tools for
Visual Studio, or you can download the latest version of SqlPackage directly from the Microsoft download center.
We recommend the use of the SQLPackage utility for scale and performance in most production environments.
For a SQL Server Customer Advisory Team blog about migrating using BACPAC files, see Migrating from SQL
Server to Azure SQL Database using BACPAC Files.
See the following SQLPackage command for a sample script for how to import the AdventureWorks2008R2
database from local storage to an Azure SQL Database logical server, called mynewserver20170403 in this
example. This script shows the creation of a new database called myMigratedDatabase, with a service tier of
Premium, and a Service Objective of P6. Change these values as appropriate to your environment.

SqlPackage.exe /a:import /tcs:"Data Source=mynewserver20170403.database.windows.net;Initial Catalog=myMigratedDatabase;User

Id=ServerAdmin;Password=<change_to_your_password>" /sf:AdventureWorks2008R2.bacpac /p:DatabaseEdition=Premium
/p:DatabaseServiceObjective=P6
IMPORTANT
An Azure SQL Database logical server listens on port 1433. If you are attempting to connect to an Azure SQL Database
logical server from within a corporate firewall, this port must be open in the corporate firewall for you to successfully
connect.

PowerShell
Use the New-AzureRmSqlDatabaseImport cmdlet to submit an import database request to the Azure SQL
Database service. Depending on the size of your database, the import operation may take some time to complete.

$importRequest = New-AzureRmSqlDatabaseImport -ResourceGroupName "myResourceGroup" `

-ServerName $servername `
-DatabaseName "MyImportSample" `
-DatabaseMaxSizeBytes "262144000" `
-StorageKeyType "StorageAccessKey" `
-StorageKey $(Get-AzureRmStorageAccountKey -ResourceGroupName "myResourceGroup" -StorageAccountName
$storageaccountname).Value[0] `
-StorageUri "http://$storageaccountname.blob.core.windows.net/importsample/sample.bacpac" `
-Edition "Standard" `
-ServiceObjectiveName "P6" `
-AdministratorLogin "ServerAdmin" `
-AdministratorLoginPassword $(ConvertTo-SecureString -String "ASecureP@assw0rd" -AsPlainText -Force)

To check the status of the import request, use the Get-AzureRmSqlDatabaseImportExportStatus cmdlet. Running
this immediately after the request usually returns Status: InProgress. When you see Status: Succeeded the
import is complete.

$importStatus = Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $importRequest.OperationStatusLink

[Console]::Write("Importing")
while ($importStatus.Status -eq "InProgress")
{
$importStatus = Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $importRequest.OperationStatusLink
[Console]::Write(".")
Start-Sleep -s 10
}
[Console]::WriteLine("")
$importStatus

Next steps
To learn how to connect to and query an imported SQL Database, see Connect to SQL Database with SQL
Server Management Studio and perform a sample T-SQL query.
For a SQL Server Customer Advisory Team blog about migrating using BACPAC files, see Migrating from SQL
Server to Azure SQL Database using BACPAC Files.
For a discussion of the entire SQL Server database migration process, including performance
recommendations, see Migrate a SQL Server database to Azure SQL Database.
Export an Azure SQL database to a BACPAC file
4/10/2017 • 5 min to read • Edit Online

This article discusses exporting an Azure SQL database to a BACPAC file. This article discusses using the following
methods:
The Azure portal
the SqlPackage command-line utility
the New-AzureRmSqlDatabaseExport cmdlet
the Export a Data-tier Application wizard in SQL Server Management Studio.

IMPORTANT
Azure SQL Database Automated Export is now in preview and will be retired on March 1, 2017. Starting December 1, 2016,
you will no longer be able to configure automated export on any SQL database. All your existing automated export jobs will
continue to work until March 1, 2017. After December 1, 2016, you can use long-term backup retention or Azure
Automation to archive SQL databases periodically using PowerShell periodically according to a schedule of your choice. For a
sample script, you can download the sample PowerShell script from Github.

Overview
When you need to export a database for archiving or for moving to another platform, you can export the database
schema and data to a BACPAC file. A BACPAC file is a ZIP file with an extension of BACPAC containing the
metadata and data from a SQL Server database. A BACPAC file can be stored in Azure blob storage or in local
storage in an on-premises location and later imported back into Azure SQL Database or into a SQL Server on-
premises installation.

IMPORTANT
If you are exporting from SQL Server as a prelude to migration to Azure SQL Database, see Migrate a SQL Server database
to Azure SQL Database.

Considerations
For an export to be transactionally consistent, you must ensure either that no write activity is occurring during
the export, or that you are exporting from a transactionally consistent copy of your Azure SQL database.
If you are exporting to blob storage, the maximum size of a BACPAC file is 200 GB. To archive a larger BACPAC
file, export to local storage.
Exporting a BACPAC file to Azure premium storage using the methods discussed in this article is not
supported.
If the export operation from Azure SQL Database exceeds 20 hours, it may be canceled. To increase
performance during export, you can:
Temporarily increase your service level.
Cease all read and write activity during the export.
Use a clustered index with non-null values on all large tables. Without clustered indexes, an export may
fail if it takes longer than 6-12 hours. This is because the export service needs to complete a table scan
to try to export entire table. A good way to determine if your tables are optimized for export is to run
DBCC SHOW_STATISTICS and make sure that the RANGE_HI_KEY is not null and its value has good
distribution. For details, see DBCC SHOW_STATISTICS.

NOTE
BACPACs are not intended to be used for backup and restore operations. Azure SQL Database automatically creates
backups for every user database. For details, see Business Continuity Overview and SQL Database backups.

Azure portal
To export a database using the Azure portal, open the page for your database and click Export on the toolbar.
Specify the *.bacpac filename, provide the Azure storage account and container for the export, and provide the
credentials to connect to the source database.

To monitor the progress of the export operation, open the page for the logical server containing the database
being exported. Scroll down to Operations and then click Import/Export history.

SQLPackage utility
To export a SQL database using the SqlPackage command-line utility, see Export parameters and properties. The
SQLPackage utility ships with the latest versions of SQL Server Management Studio and SQL Server Data Tools
for Visual Studio, or you can download the latest version of SqlPackage directly from the Microsoft download
center.
We recommend the use of the SQLPackage utility for scale and performance in most production environments.
For a SQL Server Customer Advisory Team blog about migrating using BACPAC files, see Migrating from SQL
Server to Azure SQL Database using BACPAC Files.
SQL Server Management Studio
The newest versions of SQL Server Management Studio also provide a wizard to export an Azure SQL Database to
a bacpac file. See the Export a Data-tier Application.

PowerShell
Use the New-AzureRmSqlDatabaseImport cmdlet to submit an export database request to the Azure SQL
Database service. Depending on the size of your database, the export operation may take some time to complete.

$exportRequest = New-AzureRmSqlDatabaseExport -ResourceGroupName $ResourceGroupName -ServerName $ServerName `

-DatabaseName $DatabaseName -StorageKeytype $StorageKeytype -StorageKey $StorageKey -StorageUri $BacpacUri `
-AdministratorLogin $creds.UserName -AdministratorLoginPassword $creds.Password

To check the status of the export request, use the Get-AzureRmSqlDatabaseImportExportStatus cmdlet. Running
this immediately after the request usually returns Status: InProgress. When you see Status: Succeeded the
export is complete.

$importStatus = Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $importRequest.OperationStatusLink

[Console]::Write("Exporting")
while ($importStatus.Status -eq "InProgress")
{
$importStatus = Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $importRequest.OperationStatusLink
[Console]::Write(".")
Start-Sleep -s 10
}
[Console]::WriteLine("")
$importStatus

Export SQL database example

The following example exports an existing Azure SQL database to a BACPAC and then shows how to check the
status of the export operation.
To run the example, there are a few variables you need to replace with the specific values for your database and
storage account. In the Azure portal, browse to your storage account to get the storage account name, blob
container name, and key value. You can find the key by clicking Access keys on your storage account blade.
Replace the following VARIABLE-VALUES with values for your specific Azure resources. The database name is the
existing database you want to export.
$subscriptionId = "YOUR AZURE SUBSCRIPTION ID"

# Database to export
$DatabaseName = "DATABASE-NAME"
$ResourceGroupName = "RESOURCE-GROUP-NAME"
$ServerName = "SERVER-NAME
$serverAdmin = "ADMIN-NAME"
$serverPassword = "ADMIN-PASSWORD"
$securePassword = ConvertTo-SecureString -String $serverPassword -AsPlainText -Force
$creds = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $serverAdmin, $securePassword

# Generate a unique filename for the BACPAC

$bacpacFilename = $DatabaseName + (Get-Date).ToString("yyyyMMddHHmm") + ".bacpac"

# Storage account info for the BACPAC

$BaseStorageUri = "https://STORAGE-NAME.blob.core.windows.net/BLOB-CONTAINER-NAME/"
$BacpacUri = $BaseStorageUri + $bacpacFilename
$StorageKeytype = "StorageAccessKey"
$StorageKey = "YOUR STORAGE KEY"

$exportRequest = New-AzureRmSqlDatabaseExport -ResourceGroupName $ResourceGroupName -ServerName $ServerName `

-DatabaseName $DatabaseName -StorageKeytype $StorageKeytype -StorageKey $StorageKey -StorageUri $BacpacUri `
-AdministratorLogin $creds.UserName -AdministratorLoginPassword $creds.Password

$exportRequest

# Check status of the export

Get-AzureRmSqlDatabaseImportExportStatus -OperationStatusLink $exportRequest.OperationStatusLink

Next steps
To learn about long-term backup retention of an Azure SQL database backup as an alternative to exported a
database for archive purposes, see Long-term backup retention.
For a SQL Server Customer Advisory Team blog about migrating using BACPAC files, see Migrating from SQL
Server to Azure SQL Database using BACPAC Files.
To learn about importing a BACPAC to a SQL Server database, see Import a BACPCAC to a SQL Server
database.
To learn about exporting a BACPAC from a SQL Server database, see Export a Data-tier Application and
Migrate your first database.
Monitoring database performance in Azure SQL
Database
2/17/2017 • 2 min to read • Edit Online

Monitoring the performance of a SQL database in Azure starts with monitoring the resource utilization relative to
the level of database performance you choose. Monitoring helps you determine whether your database has excess
capacity or is having trouble because resources are maxed out, and then decide whether it's time to adjust the
performance level and service tier of your database. You can monitor your database using graphical tools in the
Azure portal or using SQL dynamic management views.

Monitor databases using the Azure portal

In the Azure portal, you can monitor a single database’s utilization by selecting your database and clicking the
Monitoring chart. This brings up a Metric window that you can change by clicking the Edit chart button. Add the
following metrics:
CPU percentage
DTU percentage
Data IO percentage
Database size percentage
Once you’ve added these metrics, you can continue to view them in the Monitoring chart with more details on the
Metric window. All four metrics show the average utilization percentage relative to the DTU of your database. See
the service tiers article for details about DTUs.

You can also configure alerts on the performance metrics. Click the Add alert button in the Metric window. Follow
the wizard to configure your alert. You have the option to alert if the metrics exceed a certain threshold or if the
metric falls below a certain threshold.
For example, if you expect the workload on your database to grow, you can choose to configure an email alert
whenever your database reaches 80% on any of the performance metrics. You can use this as an early warning to
figure out when you might have to switch to the next higher performance level.
The performance metrics can also help you determine if you are able to downgrade to a lower performance level.
Assume you are using a Standard S2 database and all performance metrics show that the database on average
does not use more than 10% at any given time. It is likely that the database will work well in Standard S1. However,
be aware of workloads that spike or fluctuate before making the decision to move to a lower performance level.

Monitor databases using DMVs

The same metrics that are exposed in the portal are also available through system views: sys.resource_stats in the
logical master database of your server, and sys.dm_db_resource_stats in the user database. Use
sys.resource_stats if you need to monitor less granular data across a longer period of time. Use
sys.dm_db_resource_stats if you need to monitor more granular data within a smaller time frame. For more
information, see Azure SQL Database Performance Guidance.

NOTE
sys.dm_db_resource_stats returns an empty result set when used in Web and Business edition databases, which are retired.

For elastic pools, you can monitor individual databases in the pool with the techniques described in this section. But
you can also monitor the pool as a whole. For information, see Monitor and manage an elastic pool.
SQL Database Advisor
2/1/2017 • 3 min to read • Edit Online

Azure SQL Database learns and adapts with your application and provides customized recommendations enabling
you to maximize the performance of your SQL databases. The SQL Database Advisor provides recommendations
for creating and dropping indexes, parameterizing queries, and fixing schema issues. The advisor assesses
performance by analyzing your SQL database's usage history. The recommendations that are best suited for
running your database’s typical workload are recommended.
The following recommendations are available for Azure SQL Database servers. Currently you can set the create
and drop index recommendations to be applied automatically, see Automatic index management for details.

Create Index recommendations

Create Index recommendations appear when the SQL Database service detects a missing index that if created,
can benefit your databases workload (non-clustered indexes only).

Drop Index recommendations

Drop Index recommendations appear when the SQL Database service detects duplicate indexes (currently in
preview and applies to duplicate indexes only).

Parameterize queries recommendations

Parameterize queries recommendations appear when you have one or more queries that are constantly being
recompiled but end up with the same query execution plan. This condition opens up an opportunity to apply
forced parameterization, which will allow query plans to be cached and reused in the future improving
performance and reducing resource usage.
Every query issued against SQL Server initially needs to be compiled to generate an execution plan. Each
generated plan is added to the plan cache and subsequent executions of the same query can reuse this plan from
the cache, eliminating the need for additional compilation.
Applications that send queries, which include non-parameterized values, can lead to a performance overhead,
where for every such query with different parameter values the execution plan is compiled again. In many cases
the same queries with different parameter values generate the same execution plans, but these plans are still
separately added to the plan cache. Recompiling execution plans use database resources, increase the query
duration time and overflow the plan cache causing plans to be evicted from the cache. This behavior of SQL
Server can be altered by setting the forced parameterization option on the database.
To help you estimate the impact of this recommendation, you are provided with a comparison between the actual
CPU usage and the projected CPU usage (as if the recommendation was applied). In addition to CPU savings, your
query duration decreases for the time spent in compilation. There will also be much less overhead on plan cache,
allowing majority of the plans to stay in cache and be reused. You can apply this recommendation quickly and
easily by clicking on the Apply command.
Once you apply this recommendation, it will enable forced parameterization within minutes on your database and
it starts the monitoring process which approximately lasts for 24 hours. After this period, you will be able to see
the validation report that shows CPU usage of your database 24 hours before and after the recommendation has
been applied. SQL Database Advisor has a safety mechanism that automatically reverts the applied
recommendation in case a performance regression has been detected.
Fix schema issues recommendations
Fix schema issues recommendations appear when the SQL Database service notices an anomaly in the number
of schema-related SQL errors happening on your Azure SQL Database. This recommendation typically appears
when your database encounters multiple schema-related errors (invalid column name, invalid object name, etc.)
within an hour.
“Schema issues” are a class of syntax errors in SQL Server that happen when the definition of the SQL query and
the definition of the database schema are not aligned. For example, one of the columns expected by the query
may be missing in the target table, or vice versa.
“Fix schema issue” recommendation appears when Azure SQL Database service notices an anomaly in the number
of schema-related SQL errors happening on your Azure SQL Database. The following table shows the errors that
are related to schema issues:

SQL ERROR CODE MESSAGE

201 Procedure or function '' expects parameter '', which was not
supplied.

207 Invalid column name '*'.

208 Invalid object name '*'.

213 Column name or number of supplied values does not match

table definition.

2812 Could not find stored procedure '*'.

8144 Procedure or function * has too many arguments specified.

Next steps
Monitor your recommendations and continue to apply them to refine performance. Database workloads are
dynamic and change continuously. SQL Database advisor continues to monitor and provide recommendations
that can potentially improve your database's performance.
See SQL Database Advisor in the Azure portal for steps on how to use SQL Database Advisor in the Azure
portal.
See Query Performance Insights to learn about and view the performance impact of your top queries.

Additional resources
Query Store
CREATE INDEX
Role-based access control
Azure SQL Database Query Performance Insight
1/31/2017 • 7 min to read • Edit Online

Managing and tuning the performance of relational databases is a challenging task that requires significant
expertise and time investment. Query Performance Insight allows you to spend less time troubleshooting
database performance by providing the following:
Deeper insight into your databases resource (DTU) consumption.
The top queries by CPU/Duration/Execution count, which can potentially be tuned for improved performance.
The ability to drill down into the details of a query, view its text and history of resource utilization.
Performance tuning annotations that show actions performed by SQL Azure Database Advisor

Prerequisites
Query Performance Insight requires that Query Store is active on your database. If Query Store is not running,
the portal prompts you to turn it on.

Permissions
The following role-based access control permissions are required to use Query Performance Insight:
Reader, Owner, Contributor, SQL DB Contributor, or SQL Server Contributor permissions are required to
view the top resource consuming queries and charts.
Owner, Contributor, SQL DB Contributor, or SQL Server Contributor permissions are required to view
query text.

Using Query Performance Insight

Query Performance Insight is easy to use:
Open Azure portal and find database that you want to examine.
From left-hand side menu, under support and troubleshooting, select “Query Performance Insight”.
On the first tab, review the list of top resource-consuming queries.
Select an individual query to view its details.
Open SQL Azure Database Advisor and check if any recommendations are available.
Use sliders or zoom icons to change observed interval.
NOTE
A couple hours of data needs to be captured by Query Store for SQL Database to provide query performance insights. If
the database has no activity or Query Store was not active during a certain time period, the charts will be empty when
displaying that time period. You may enable Query Store at any time if it is not running.

Review top CPU consuming queries

In the portal do the following:
1. Browse to a SQL database and click All settings > Support + Troubleshooting > Query performance
insight.

The top queries view opens and the top CPU consuming queries are listed.
2. Click around the chart for details.
The top line shows overall DTU% for the database, while the bars show CPU% consumed by the selected
queries during the selected interval (for example, if Past week is selected each bar represents one day).
The bottom grid represents aggregated information for the visible queries.
Query ID - unique identifier of query inside database.
CPU per query during observable interval (depends on aggregation function).
Duration per query (depends on aggregation function).
Total number of executions for a particular query.
Select or clear individual queries to include or exclude them from the chart using checkboxes.
3. If your data becomes stale, click the Refresh button.
4. You can use sliders and zoom buttons to change observation interval and investigate spikes:

5. Optionally, if you want a different view, you can select Custom tab and set:
Metric (CPU, duration, execution count)
Time interval (Last 24 hours, Past week, Past month).
Number of queries.
Aggregation function.
Viewing individual query details
To view query details:
1. Click any query in the list of top queries.

2. The details view opens and the queries CPU consumption/Duration/Execution count is broken down over time.
3. Click around the chart for details.
Top chart shows line with overall database DTU%, and the bars are CPU% consumed by the selected
query.
Second chart shows total duration by the selected query.
Bottom chart shows total number of executions by the selected query.
4. Optionally, use sliders, zoom buttons or click Settings to customize how query data is displayed, or to pick a
different time period.

Review top queries per duration

In the recent update of Query Performance Insight, we introduced two new metrics that can help you identify
potential bottlenecks: duration and execution count.
Long-running queries have the greatest potential for locking resources longer, blocking other users, and limiting
scalability. They are also the best candidates for optimization.
To identify long running queries:
1. Open Custom tab in Query Performance Insight for selected database
2. Change metrics to be duration
3. Select number of queries and observation interval
4. Select aggregation function
Sum adds up all query execution time during whole observation interval.
Max finds queries which execution time was maximum at whole observation interval.
Avg finds average execution time of all query executions and show you the top out of these
averages.
Review top queries per execution count
High number of executions might not be affecting database itself and resources usage can be low, but overall
application might get slow.
In some cases, very high execution count may lead to increase of network round trips. Round trips significantly
affect performance. They are subject to network latency and to downstream server latency.
For example, many data-driven Web sites heavily access the database for every user request. While connection
pooling helps, the increased network traffic and processing load on the database server can adversely affect
performance. General advice is to keep round trips to an absolute minimum.
To identify frequently executed queries (“chatty”) queries:
1. Open Custom tab in Query Performance Insight for selected database
2. Change metrics to be execution count
3. Select number of queries and observation interval
Understanding performance tuning annotations
While exploring your workload in Query Performance Insight, you might notice icons with vertical line on top of
the chart.
These icons are annotations; they represent performance affecting actions performed by SQL Azure Database
Advisor. By hovering annotation, you get basic information about the action:

If you want to know more or apply advisor recommendation, click the icon. It will open details of action. If it’s an
active recommendation you can apply it straight away using command.
Multiple annotations.
It’s possible, that because of zoom level, annotations that are close to each other will get collapsed into one. This
will be represented by special icon, clicking it will open new blade where list of grouped annotations will be
shown. Correlating queries and performance tuning actions can help to better understand your workload.

Optimizing the Query Store configuration for Query Performance

Insight
During your use of Query Performance Insight, you might encounter the following Query Store messages:
"Query Store is not properly configured on this database. Click here to learn more."
"Query Store is not properly configured on this database. Click here to change settings."
These messages usually appear when Query Store is not able to collect new data.
First case happens when Query Store is in Read-Only state and parameters are set optimally. You can fix this by
increasing size of Query Store or clearing Query Store.

Second case happens when Query Store is Off or parameters aren’t set optimally.
You can change the Retention and Capture policy and enable Query Store by executing commands below or
directly from portal:
Recommended retention and capture policy
There are two types of retention policies:
Size based - if set to AUTO it will clean data automatically when near max size is reached.
Time based - by default we will set it to 30 days, which means, if Query Store will run out of space, it will
delete query information older than 30 days
Capture policy could be set to:
All - Captures all queries.
Auto - Infrequent queries and queries with insignificant compile and execution duration are ignored.
Thresholds for execution count, compile and runtime duration are internally determined. This is the default
option.
None - Query Store stops capturing new queries, however runtime stats for already captured queries are still
collected.
We recommend setting all policies to AUTO and clean policy to 30 days:

ALTER DATABASE [YourDB]

SET QUERY_STORE (SIZE_BASED_CLEANUP_MODE = AUTO);

ALTER DATABASE [YourDB]

SET QUERY_STORE (CLEANUP_POLICY = (STALE_QUERY_THRESHOLD_DAYS = 30));

ALTER DATABASE [YourDB]

SET QUERY_STORE (QUERY_CAPTURE_MODE = AUTO);

Increase size of Query Store. This could be performed by connecting to a database and issuing following query:

ALTER DATABASE [YourDB]

SET QUERY_STORE (MAX_STORAGE_SIZE_MB = 1024);

Applying these settings will eventually make Query Store collecting new queries, however if you don’t want to
wait you can clear Query Store.
NOTE
Executing following query will delete all current information in the Query Store.

ALTER DATABASE [YourDB] SET QUERY_STORE CLEAR;

Summary
Query Performance Insight helps you understand the impact of your query workload and how it relates to
database resource consumption. With this feature, you will learn about the top consuming queries, and easily
identify the ones to fix before they become a problem.

Next steps
For additional recommendations about improving the performance of your SQL database, click
Recommendations on the Query Performance Insight blade.
Operating the Query Store in Azure SQL Database
2/2/2017 • 2 min to read • Edit Online

Query Store in Azure is a fully managed database feature that continuously collects and presents detailed historic
information about all queries. You can think about Query Store as similar to an airplane's flight data recorder that
significantly simplifies query performance troubleshooting both for cloud and on-premises customers. This article
explains specific aspects of operating Query Store in Azure. Using this pre-collected query data, you can quickly
diagnose and resolve performance problems and thus spend more time focusing on their business.
Query Store has been globally available in Azure SQL Database since November, 2015. Query Store is the
foundation for performance analysis and tuning features, such as SQL Database Advisor and Performance
Dashboard. At the moment of publishing this article, Query Store is running in more than 200,000 user databases
in Azure, collecting query-related information for several months, without interruption.

IMPORTANT
Microsoft is in the process of activating Query Store for all Azure SQL databases (existing and new).

Optimal Query Store Configuration

This section describes optimal configuration defaults that are designed to ensure reliable operation of the Query
Store and dependent features, such as SQL Database Advisor and Performance Dashboard. Default configuration is
optimized for continuous data collection, that is minimal time spent in OFF/READ_ONLY states.

CONFIGURATION DESCRIPTION DEFAULT COMMENT

MAX_STORAGE_SIZE_MB Specifies the limit for the 100 Enforced for new databases
data space that Query Store
can take inside z customer
database

INTERVAL_LENGTH_MINUTE Defines size of time window 60 Enforced for new databases

S during which collected
runtime statistics for query
plans are aggregated and
persisted. Every active query
plan has at most one row for
a period of time defined with
this configuration

STALE_QUERY_THRESHOLD_ Time-based cleanup policy 30 Enforced for new databases

DAYS that controls the retention and databases with previous
period of persisted runtime default (367)
statistics and inactive queries

SIZE_BASED_CLEANUP_MO Specifies whether automatic AUTO Enforced for all databases

DE data cleanup takes place
when Query Store data size
approaches the limit
CONFIGURATION DESCRIPTION DEFAULT COMMENT

QUERY_CAPTURE_MODE Specifies whether all queries AUTO Enforced for all databases
or only a subset of queries
are tracked

FLUSH_INTERVAL_SECONDS Specifies maximum period 900 Enforced for new databases

during which captured
runtime statistics are kept in
memory, before flushing to
disk

IMPORTANT
These defaults are automatically applied in the final stage of Query Store activation in all Azure SQL databases (see preceding
important note). After this light up, Azure SQL Database won’t be changing configuration values set by customers, unless
they negatively impact primary workload or reliable operations of the Query Store.

If you want to stay with your custom settings, use ALTER DATABASE with Query Store options to revert
configuration to the previous state. Check out Best Practices with the Query Store in order to learn how top chose
optimal configuration parameters.

Next steps
SQL Database Performance Insight

Additional resources
For more information check out the following articles:
A flight data recorder for your database
Monitoring Performance By Using the Query Store
Query Store Usage Scenarios
Monitoring Performance By Using the Query Store
Monitoring Azure SQL Database using dynamic
management views
1/17/2017 • 3 min to read • Edit Online

Microsoft Azure SQL Database enables a subset of dynamic management views to diagnose performance
problems, which might be caused by blocked or long-running queries, resource bottlenecks, poor query plans, and
so on. This topic provides information on how to detect common performance problems by using dynamic
management views.
SQL Database partially supports three categories of dynamic management views:
Database-related dynamic management views.
Execution-related dynamic management views.
Transaction-related dynamic management views.
For detailed information on dynamic management views, see Dynamic Management Views and Functions
(Transact-SQL) in SQL Server Books Online.

Permissions
In SQL Database, querying a dynamic management view requires VIEW DATABASE STATE permissions. The
VIEW DATABASE STATE permission returns information about all objects within the current database. To grant
the VIEW DATABASE STATE permission to a specific database user, run the following query:
GRANT VIEW DATABASE STATE TO database_user;

In an instance of on-premises SQL Server, dynamic management views return server state information. In SQL
Database, they return information regarding your current logical database only.

Calculating database size

The following query returns the size of your database (in megabytes):

-- Calculates the size of the database.

SELECT SUM(reserved_page_count)*8.0/1024
FROM sys.dm_db_partition_stats;
GO

The following query returns the size of individual objects (in megabytes) in your database:

-- Calculates the size of individual database objects.

SELECT sys.objects.name, SUM(reserved_page_count) * 8.0 / 1024
FROM sys.dm_db_partition_stats, sys.objects
WHERE sys.dm_db_partition_stats.object_id = sys.objects.object_id
GROUP BY sys.objects.name;
GO

Monitoring connections
You can use the sys.dm_exec_connections view to retrieve information about the connections established to a
specific Azure SQL Database server and the details of each connection. In addition, the sys.dm_exec_sessions view
is helpful when retrieving information about all active user connections and internal tasks. The following query
retrieves information on the current connection:

SELECT
c.session_id, c.net_transport, c.encrypt_option,
c.auth_scheme, s.host_name, s.program_name,
s.client_interface_name, s.login_name, s.nt_domain,
s.nt_user_name, s.original_login_name, c.connect_time,
s.login_time
FROM sys.dm_exec_connections AS c
JOIN sys.dm_exec_sessions AS s
ON c.session_id = s.session_id
WHERE c.session_id = @@SPID;

NOTE
When executing the sys.dm_exec_requests and sys.dm_exec_sessions views, if you have VIEW DATABASE STATE
permission on the database, you see all executing sessions on the database; otherwise, you see only the current session.

Monitoring query performance

Slow or long running queries can consume significant system resources. This section demonstrates how to use
dynamic management views to detect a few common query performance problems. An older but still helpful
reference for troubleshooting, is the Troubleshooting Performance Problems in SQL Server 2008 article on
Microsoft TechNet.
Finding top N queries
The following example returns information about the top five queries ranked by average CPU time. This example
aggregates the queries according to their query hash, so that logically equivalent queries are grouped by their
cumulative resource consumption.

SELECT TOP 5 query_stats.query_hash AS "Query Hash",

SUM(query_stats.total_worker_time) / SUM(query_stats.execution_count) AS "Avg CPU Time",
MIN(query_stats.statement_text) AS "Statement Text"
FROM
(SELECT QS.*,
SUBSTRING(ST.text, (QS.statement_start_offset/2) + 1,
((CASE statement_end_offset
WHEN -1 THEN DATALENGTH(ST.text)
ELSE QS.statement_end_offset END
- QS.statement_start_offset)/2) + 1) AS statement_text
FROM sys.dm_exec_query_stats AS QS
CROSS APPLY sys.dm_exec_sql_text(QS.sql_handle) as ST) as query_stats
GROUP BY query_stats.query_hash
ORDER BY 2 DESC;

Monitoring blocked queries

Slow or long-running queries can contribute to excessive resource consumption and be the consequence of
blocked queries. The cause of the blocking can be poor application design, bad query plans, the lack of useful
indexes, and so on. You can use the sys.dm_tran_locks view to get information about the current locking activity in
your Azure SQL Database. For example code, see sys.dm_tran_locks (Transact-SQL) in SQL Server Books Online.
Monitoring query plans
An inefficient query plan also may increase CPU consumption. The following example uses the
sys.dm_exec_query_stats view to determine which query uses the most cumulative CPU.
SELECT
highest_cpu_queries.plan_handle,
highest_cpu_queries.total_worker_time,
q.dbid,
q.objectid,
q.number,
q.encrypted,
q.[text]
FROM
(SELECT TOP 50
qs.plan_handle,
qs.total_worker_time
FROM
sys.dm_exec_query_stats qs
ORDER BY qs.total_worker_time desc) AS highest_cpu_queries
CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS q
ORDER BY highest_cpu_queries.total_worker_time DESC;

See also
Introduction to SQL Database
Azure SQL Database and performance for single
databases
4/19/2017 • 30 min to read • Edit Online

Azure SQL Database offers four service tiers: Basic, Standard, Premium, and Premium RS. Each service tier strictly
isolates the resources that your SQL database can use, and guarantees predictable performance for that service
level. In this article, we offer guidance that can help you choose the service tier for your application. We also
discuss ways that you can tune your application to get the most from Azure SQL Database.

NOTE
This article focuses on performance guidance for single databases in Azure SQL Database. For performance guidance related
to elastic pools, see Price and performance considerations for elastic pools. Note, though, that you can apply many of the
tuning recommendations in this article to databases in an elastic pool, and get similar performance benefits.

Why service tiers?

Although each database workload can differ, the purpose of service tiers is to provide performance predictability
at various performance levels. Customers with large-scale database resource requirements can work in a more
dedicated computing environment.
Common service tier use cases
Basic
You're just getting started with Azure SQL Database. Applications that are in development often don't
need high-performance levels. Basic databases are an ideal environment for database development, at a low
price point.
You have a database with a single user. Applications that associate a single user with a database typically
don’t have high concurrency and performance requirements. These applications are candidates for the Basic
service tier.
Standard
Your database has multiple concurrent requests. Applications that service more than one user at a time
usually need higher performance levels. For example, websites that get moderate traffic or departmental
applications that require more resources are good candidates for the Standard service tier.
Premium
Most Premium service tier use cases have one or more of these characteristics:
High peak load. An application that requires substantial CPU, memory, or input/output (I/O) to complete its
operations requires a dedicated, high-performance level. For example, a database operation known to consume
several CPU cores for an extended time is a candidate for the Premium service tier.
Many concurrent requests. Some database applications service many concurrent requests, for example, when
serving a website that has a high traffic volume. Basic and Standard service tiers limit the number of concurrent
requests per database. Applications that require more connections would need to choose an appropriate
reservation size to handle the maximum number of needed requests.
Low latency. Some applications need to guarantee a response from the database in minimal time. If a
specific stored procedure is called as part of a broader customer operation, you might have a requirement
to have a return from that call in no more than 20 milliseconds, 99 percent of the time. This type of
application benefits from the Premium service tier, to make sure that the required computing power is
available.
Premium RS. Designed for customers that have IO-intensive workloads but do not require the highest
availability guarantees. Examples include testing high-performance workloads, or an analytical workload
where the database is not the system of record.
The service level that you need for your SQL database depends on the peak load requirements for each resource
dimension. Some applications use a trivial amount of a single resource, but have significant requirements for other
resources.

Service tier capabilities and limits

Each service tier and performance level is associated with different limits and performance characteristics. This
table describes these characteristics for a single database.
Basic service tier
PERFORMANCE LEVEL BASIC

Max DTUs 5

Max database size* 2 GB

Max in-memory OLTP storage N/A

Max concurrent workers 30

Max concurrent logins 30

Max concurrent sessions 300

Standard service tier

PERFORMANCE LEVEL S0 S1 S2 S3

Max DTUs 10 20 50 100

Max database size* 250 GB 250 GB 250 GB 250 GB

Max in-memory OLTP N/A N/A N/A N/A

storage

Max concurrent 60 90 120 200

workers

Max concurrent 60 90 120 200

logins

Max concurrent 600 900 1200 2400

sessions

Premium service tier

PERFORMANCE
LEVEL P1 P2 P4 P6 P11 P15

Max DTUs 125 250 500 1000 1750 4000

Max database 500 GB 500 GB 500 GB 500 GB 4 TB* 4 TB*

size*

Max in- 1 GB 2 GB 4 GB 8 GB 14 GB 32 GB
memory OLTP
storage

Max 200 400 800 1600 2400 6400

concurrent
workers

Max 200 400 800 1600 2400 6400

concurrent
logins

Max 30000 30000 30000 30000 30000 30000

concurrent
sessions

Premium RS service tier

PERFORMANCE LEVEL PRS1 PRS2 PRS4 PRS6

Max DTUs 125 250 500 1000

Max database size* 500 GB 500 GB 500 GB 500 GB

Max in-memory OLTP 1 GB 2 GB 4 GB 8 GB

storage

Max concurrent 200 400 800 1600

workers

Max concurrent 200 400 800 1600

logins

Max concurrent 30000 30000 30000 30000

sessions

* Max database size refers to the maximum size of the data in the database.

Maximum In-Memory OLTP storage

You can use the sys.dm_db_resource_stats view to monitor your Azure In-Memory storage use. For more
information about monitoring, see Monitor In-Memory OLTP storage.
Maximum concurrent requests
To see the number of concurrent requests, run this Transact-SQL query on your SQL database:

SELECT COUNT(*) AS [Concurrent_Requests]

FROM sys.dm_exec_requests R

To analyze the workload of an on-premises SQL Server database, modify this query to filter on the specific
database you want to analyze. For example, if you have an on-premises database named MyDatabase, this
Transact-SQL query returns the count of concurrent requests in that database:

SELECT COUNT(*) AS [Concurrent_Requests]

FROM sys.dm_exec_requests R
INNER JOIN sys.databases D ON D.database_id = R.database_id
AND D.name = 'MyDatabase'

This is just a snapshot at a single point in time. To get a better understanding of your workload and concurrent
request requirements, you'll need to collect many samples over time.
Maximum concurrent logins
You can analyze your user and application patterns to get an idea of the frequency of logins. You also can run real-
world loads in a test environment to make sure that you're not hitting this or other limits we discuss in this article.
There isn’t a single query or dynamic management view (DMV) that can show you concurrent login counts or
history.
If multiple clients use the same connection string, the service authenticates each login. If 10 users simultaneously
connect to a database by using the same username and password, there would be 10 concurrent logins. This limit
applies only to the duration of the login and authentication. If the same 10 users connect to the database
sequentially, the number of concurrent logins would never be greater than 1.

NOTE
Currently, this limit does not apply to databases in elastic pools.

Maximum sessions
To see the number of current active sessions, run this Transact-SQL query on your SQL database:

SELECT COUNT(*) AS [Sessions]

FROM sys.dm_exec_connections

If you're analyzing an on-premises SQL Server workload, modify the query to focus on a specific database. This
query helps you determine possible session needs for the database if you are considering moving it to Azure SQL
Database.

SELECT COUNT(*) AS [Sessions]

FROM sys.dm_exec_connections C
INNER JOIN sys.dm_exec_sessions S ON (S.session_id = C.session_id)
INNER JOIN sys.databases D ON (D.database_id = S.database_id)
WHERE D.name = 'MyDatabase'

Again, these queries return a point-in-time count. If you collect multiple samples over time, you’ll have the best
understanding of your session use.
For SQL Database analysis, you can get historical statistics on sessions by querying the sys.resource_stats view and
reviewing the active_session_count column.

Monitor resource use

You can monitor resource usage using SQL Database Query Performance Insight and Query Store.
You can also monitor usage using these two views:
sys.dm_db_resource_stats
sys.resource_stats
sys.dm_db_resource_stats
You can use the sys.dm_db_resource_stats view in every SQL database. The sys.dm_db_resource_stats view
shows recent resource use data relative to the service tier. Average percentages for CPU, data I/O, log writes, and
memory are recorded every 15 seconds and are maintained for 1 hour.
Because this view provides a more granular look at resource use, use sys.dm_db_resource_stats first for any
current-state analysis or troubleshooting. For example, this query shows the average and maximum resource use
for the current database over the past hour:

SELECT
AVG(avg_cpu_percent) AS 'Average CPU use in percent',
MAX(avg_cpu_percent) AS 'Maximum CPU use in percent',
AVG(avg_data_io_percent) AS 'Average data I/O in percent',
MAX(avg_data_io_percent) AS 'Maximum data I/O in percent',
AVG(avg_log_write_percent) AS 'Average log write use in percent',
MAX(avg_log_write_percent) AS 'Maximum log write use in percent',
AVG(avg_memory_usage_percent) AS 'Average memory use in percent',
MAX(avg_memory_usage_percent) AS 'Maximum memory use in percent'
FROM sys.dm_db_resource_stats;

For other queries, see the examples in sys.dm_db_resource_stats.

sys.resource_stats
The sys.resource_stats view in the master database has additional information that can help you monitor the
performance of your SQL database at its specific service tier and performance level. The data is collected every 5
minutes and is maintained for approximately 14 days. This view is useful for a longer-term historical analysis of
how your SQL database uses resources.
The following graph shows the CPU resource use for a Premium database with the P2 performance level for each
hour in a week. This graph starts on a Monday, shows 5 work days, and then shows a weekend, when much less
happens on the application.
From the data, this database currently has a peak CPU load of just over 50 percent CPU use relative to the P2
performance level (midday on Tuesday). If CPU is the dominant factor in the application’s resource profile, then
you might decide that P2 is the right performance level to guarantee that the workload always fits. If you expect an
application to grow over time, it's a good idea to have an extra resource buffer so that the application doesn't ever
reach the performance-level limit. If you increase the performance level, you can help avoid customer-visible
errors that might occur when a database doesn't have enough power to process requests effectively, especially in
latency-sensitive environments. An example is a database that supports an application that paints webpages based
on the results of database calls.
Other application types might interpret the same graph differently. For example, if an application tries to process
payroll data each day and has the same chart, this kind of "batch job" model might do fine at a P1 performance
level. The P1 performance level has 100 DTUs compared to 200 DTUs at the P2 performance level. The P1
performance level provides half the performance of the P2 performance level. So, 50 percent of CPU use in P2
equals 100 percent CPU use in P1. If the application does not have timeouts, it might not matter if a job takes 2
hours or 2.5 hours to finish, if it gets done today. An application in this category probably can use a P1
performance level. You can take advantage of the fact that there are periods of time during the day when resource
use is lower, so that any "big peak" might spill over into one of the troughs later in the day. The P1 performance
level might be good for that kind of application (and save money), as long as the jobs can finish on time each day.
Azure SQL Database exposes consumed resource information for each active database in the sys.resource_stats
view of the master database in each server. The data in the table is aggregated for 5-minute intervals. With the
Basic, Standard, Premium, and Premium RS service tiers, the data can take more than 5 minutes to appear in the
table, so this data is more useful for historical analysis rather than near-real-time analysis. Query the
sys.resource_stats view to see the recent history of a database and to validate whether the reservation you chose
delivered the performance you want when needed.

NOTE
You must be connected to the master database of your logical SQL database server to query sys.resource_stats in the
following examples.

This example shows you how the data in this view is exposed:

SELECT TOP 10 *
FROM sys.resource_stats
WHERE database_name = 'resource1'
ORDER BY start_time DESC

The next example shows you different ways that you can use the sys.resource_stats catalog view to get
information about how your SQL database uses resources:
1. To look at the past week’s resource use for the database userdb1, you can run this query:

SELECT *
FROM sys.resource_stats
WHERE database_name = 'userdb1' AND
start_time > DATEADD(day, -7, GETDATE())
ORDER BY start_time DESC;
2. To evaluate how well your workload fits the performance level, you need to drill down into each aspect of
the resource metrics: CPU, reads, writes, number of workers, and number of sessions. Here's a revised
query using sys.resource_stats to report the average and maximum values of these resource metrics:

SELECT
avg(avg_cpu_percent) AS 'Average CPU use in percent',
max(avg_cpu_percent) AS 'Maximum CPU use in percent',
avg(avg_data_io_percent) AS 'Average physical data I/O use in percent',
max(avg_data_io_percent) AS 'Maximum physical data I/O use in percent',
avg(avg_log_write_percent) AS 'Average log write use in percent',
max(avg_log_write_percent) AS 'Maximum log write use in percent',
avg(max_session_percent) AS 'Average % of sessions',
max(max_session_percent) AS 'Maximum % of sessions',
avg(max_worker_percent) AS 'Average % of workers',
max(max_worker_percent) AS 'Maximum % of workers'
FROM sys.resource_stats
WHERE database_name = 'userdb1' AND start_time > DATEADD(day, -7, GETDATE());

3. With this information about the average and maximum values of each resource metric, you can assess how
well your workload fits into the performance level you chose. Usually, average values from
sys.resource_stats give you a good baseline to use against the target size. It should be your primary
measurement stick. For an example, you might be using the Standard service tier with S2 performance
level. The average use percentages for CPU and I/O reads and writes are below 40 percent, the average
number of workers is below 50, and the average number of sessions is below 200. Your workload might fit
into the S1 performance level. It's easy to see whether your database fits in the worker and session limits.
To see whether a database fits into a lower performance level with regards to CPU, reads, and writes, divide
the DTU number of the lower performance level by the DTU number of your current performance level, and
then multiply the result by 100:
S1 DTU / S2 DTU * 100 = 20 / 50 * 100 = 40
The result is the relative performance difference between the two performance levels in percentage. If your
resource use doesn't exceed this amount, your workload might fit into the lower performance level.
However, you need to look at all ranges of resource use values, and determine, by percentage, how often
your database workload would fit into the lower performance level. The following query outputs the fit
percentage per resource dimension, based on the threshold of 40 percent that we calculated in this
example:

SELECT
(COUNT(database_name) - SUM(CASE WHEN avg_cpu_percent >= 40 THEN 1 ELSE 0 END) * 1.0) / COUNT(database_name) AS
'CPU Fit Percent'
,(COUNT(database_name) - SUM(CASE WHEN avg_log_write_percent >= 40 THEN 1 ELSE 0 END) * 1.0) / COUNT(database_name)
AS 'Log Write Fit Percent'
,(COUNT(database_name) - SUM(CASE WHEN avg_data_io_percent >= 40 THEN 1 ELSE 0 END) * 1.0) / COUNT(database_name)
AS 'Physical Data IO Fit Percent'
FROM sys.resource_stats
WHERE database_name = 'userdb1' AND start_time > DATEADD(day, -7, GETDATE());

Based on your database service level objective (SLO), you can decide whether your workload fits into the
lower performance level. If your database workload SLO is 99.9 percent and the preceding query returns
values greater than 99.9 percent for all three resource dimensions, your workload likely fits into the lower
performance level.
Looking at the fit percentage also gives you insight into whether you should move to the next higher
performance level to meet your SLO. For example, userdb1 shows the following CPU use for the past week:
AVERAGE CPU PERCENT MAXIMUM CPU PERCENT

24.5 100.00

The average CPU is about a quarter of the limit of the performance level, which would fit well into the
performance level of the database. But, the maximum value shows that the database reaches the limit of the
performance level. Do you need to move to the next higher performance level? Look at how many times
your workload reaches 100 percent, and then compare it to your database workload SLO.

SELECT
(COUNT(database_name) - SUM(CASE WHEN avg_cpu_percent >= 100 THEN 1 ELSE 0 END) * 1.0) / COUNT(database_name) AS
'CPU fit percent'
,(COUNT(database_name) - SUM(CASE WHEN avg_log_write_percent >= 100 THEN 1 ELSE 0 END) * 1.0) / COUNT(database_name)
AS 'Log write fit percent'
,(COUNT(database_name) - SUM(CASE WHEN avg_data_io_percent >= 100 THEN 1 ELSE 0 END) * 1.0) / COUNT(database_name)
AS 'Physical data I/O fit percent'
FROM sys.resource_stats
WHERE database_name = 'userdb1' AND start_time > DATEADD(day, -7, GETDATE());

If this query returns a value less than 99.9 percent for any of the three resource dimensions, consider either
moving to the next higher performance level or use application-tuning techniques to reduce the load on the
SQL database.
4. This exercise also considers your projected workload increase in the future.

Tune your application

In traditional on-premises SQL Server, the process of initial capacity planning often is separated from the process
of running an application in production. Hardware and product licenses are purchased first, and performance
tuning is done afterward. When you use Azure SQL Database, it's a good idea to interweave the process of running
an application and tuning it. With the model of paying for capacity on demand, you can tune your application to
use the minimum resources needed now, instead of overprovisioning on hardware based on guesses of future
growth plans for an application, which often are incorrect. Some customers might choose not to tune an
application, and instead choose to overprovision hardware resources. This approach might be a good idea if you
don't want to change a key application during a busy period. But, tuning an application can minimize resource
requirements and lower monthly bills when you use the service tiers in Azure SQL Database.
Application characteristics
Although Azure SQL Database service tiers are designed to improve performance stability and predictability for an
application, some best practices can help you tune your application to better take advantage of the resources at a
performance level. Although many applications have significant performance gains simply by switching to a
higher performance level or service tier, some applications need additional tuning to benefit from a higher level of
service. For increased performance, consider additional application tuning for applications that have these
characteristics:
Applications that have slow performance because of "chatty" behavior. Chatty applications make
excessive data access operations that are sensitive to network latency. You might need to modify these kinds of
applications to reduce the number of data access operations to the SQL database. For example, you might
improve application performance by using techniques like batching ad-hoc queries or moving the queries to
stored procedures. For more information, see Batch queries.
Databases with an intensive workload that can't be supported by an entire single machine. Databases
that exceed the resources of the highest Premium performance level might benefit from scaling out the
workload. For more information, see Cross-database sharding and Functional partitioning.
Applications that have suboptimal queries. Applications, especially those in the data access layer, that have
poorly tuned queries might not benefit from a higher performance level. This includes queries that lack a
WHERE clause, have missing indexes, or have outdated statistics. These applications benefit from standard
query performance-tuning techniques. For more information, see Missing indexes and Query tuning and
hinting.
Applications that have suboptimal data access design. Applications that have inherent data access
concurrency issues, for example deadlocking, might not benefit from a higher performance level. Consider
reducing round trips against the Azure SQL Database by caching data on the client side with the Azure Caching
service or another caching technology. See Application tier caching.

Tuning techniques
In this section, we look at some techniques that you can use to tune Azure SQL Database to gain the best
performance for your application and run it at the lowest possible performance level. Some of these techniques
match traditional SQL Server tuning best practices, but others are specific to Azure SQL Database. In some cases,
you can examine the consumed resources for a database to find areas to further tune and extend traditional SQL
Server techniques to work in Azure SQL Database.
Azure portal tools
The following tools in the Azure portal can help you analyze and fix performance issues with your SQL database:
Query Performance Insight
SQL Database Advisor
The Azure portal has more information about both of these tools and how to use them. To efficiently diagnose and
correct problems, we recommend that you first try the tools in the Azure portal. We recommend that you use the
manual tuning approaches that we discuss next, for missing indexes and query tuning, in special cases.
Missing indexes
A common problem in OLTP database performance relates to the physical database design. Often, database
schemas are designed and shipped without testing at scale (either in load or in data volume). Unfortunately, the
performance of a query plan might be acceptable on a small scale but degrade substantially under production-
level data volumes. The most common source of this issue is the lack of appropriate indexes to satisfy filters or
other restrictions in a query. Often, missing indexes manifests as a table scan when an index seek could suffice.
In this example, the selected query plan uses a scan when a seek would suffice:

DROP TABLE dbo.missingindex;

CREATE TABLE dbo.missingindex (col1 INT IDENTITY PRIMARY KEY, col2 INT);
DECLARE @a int = 0;
SET NOCOUNT ON;
BEGIN TRANSACTION
WHILE @a < 20000
BEGIN
INSERT INTO dbo.missingindex(col2) VALUES (@a);
SET @a += 1;
END
COMMIT TRANSACTION;
GO
SELECT m1.col1
FROM dbo.missingindex m1 INNER JOIN dbo.missingindex m2 ON(m1.col1=m2.col1)
WHERE m1.col2 = 4;
Azure SQL Database can help you find and fix common missing index conditions. DMVs that are built into Azure
SQL Database look at query compilations in which an index would significantly reduce the estimated cost to run a
query. During query execution, SQL Database tracks how often each query plan is executed, and tracks the
estimated gap between the executing query plan and the imagined one where that index existed. You can use
these DMVs to quickly guess which changes to your physical database design might improve overall workload
cost for a database and its real workload.
You can use this query to evaluate potential missing indexes:

SELECT CONVERT (varchar, getdate(), 126) AS runtime,

mig.index_group_handle, mid.index_handle,
CONVERT (decimal (28,1), migs.avg_total_user_cost * migs.avg_user_impact *
(migs.user_seeks + migs.user_scans)) AS improvement_measure,
'CREATE INDEX missing_index_' + CONVERT (varchar, mig.index_group_handle) + '_' +
CONVERT (varchar, mid.index_handle) + ' ON ' + mid.statement + '
(' + ISNULL (mid.equality_columns,'')
+ CASE WHEN mid.equality_columns IS NOT NULL
AND mid.inequality_columns IS NOT NULL
THEN ',' ELSE '' END + ISNULL (mid.inequality_columns, '')
+ ')'
+ ISNULL (' INCLUDE (' + mid.included_columns + ')', '') AS create_index_statement,
migs.*,
mid.database_id,
mid.[object_id]
FROM sys.dm_db_missing_index_groups AS mig
INNER JOIN sys.dm_db_missing_index_group_stats AS migs
ON migs.group_handle = mig.index_group_handle
INNER JOIN sys.dm_db_missing_index_details AS mid
ON mig.index_handle = mid.index_handle
ORDER BY migs.avg_total_user_cost * migs.avg_user_impact * (migs.user_seeks + migs.user_scans) DESC

In this example, the query resulted in this suggestion:

CREATE INDEX missing_index_5006_5005 ON [dbo].[missingindex] ([col2])

After it's created, that same SELECT statement picks a different plan, which uses a seek instead of a scan, and then
executes the plan more efficiently:

The key insight is that the I/O capacity of a shared, commodity system is more limited than that of a dedicated
server machine. There's a premium on minimizing unnecessary I/O to take maximum advantage of the system in
the DTU of each performance level of the Azure SQL Database service tiers. Appropriate physical database design
choices can significantly improve the latency for individual queries, improve the throughput of concurrent
requests handled per scale unit, and minimize the costs required to satisfy the query. For more information about
the missing index DMVs, see sys.dm_db_missing_index_details.
Query tuning and hinting
The query optimizer in Azure SQL Database is similar to the traditional SQL Server query optimizer. Most of the
best practices for tuning queries and understanding the reasoning model limitations for the query optimizer also
apply to Azure SQL Database. If you tune queries in Azure SQL Database, you might get the additional benefit of
reducing aggregate resource demands. Your application might be able to run at a lower cost than an untuned
equivalent because it can run at a lower performance level.
An example that is common in SQL Server and which also applies to Azure SQL Database is how the query
optimizer "sniffs" parameters. During compilation, the query optimizer evaluates the current value of a parameter
to determine whether it can generate a more optimal query plan. Although this strategy often can lead to a query
plan that is significantly faster than a plan compiled without known parameter values, currently it works
imperfectly both in SQL Server and in Azure SQL Database. Sometimes the parameter is not sniffed, and
sometimes the parameter is sniffed but the generated plan is suboptimal for the full set of parameter values in a
workload. Microsoft includes query hints (directives) so that you can specify intent more deliberately and override
the default behavior of parameter sniffing. Often, if you use hints, you can fix cases in which the default SQL Server
or Azure SQL Database behavior is imperfect for a specific customer workload.
The next example demonstrates how the query processor can generate a plan that is suboptimal both for
performance and resource requirements. This example also shows that if you use a query hint, you can reduce
query run time and resource requirements for your SQL database:

DROP TABLE psptest1;

CREATE TABLE psptest1(col1 int primary key identity, col2 int, col3 binary(200));

DECLARE @a int = 0;
SET NOCOUNT ON;
BEGIN TRANSACTION
WHILE @a < 20000
BEGIN
INSERT INTO psptest1(col2) values (1);
INSERT INTO psptest1(col2) values (@a);
SET @a += 1;
END
COMMIT TRANSACTION
CREATE INDEX i1 on psptest1(col2);
GO

CREATE PROCEDURE psp1 (@param1 int)

AS
BEGIN
INSERT INTO t1 SELECT * FROM psptest1
WHERE col2 = @param1
ORDER BY col2;
END
GO

CREATE PROCEDURE psp2 (@param2 int)

AS
BEGIN
INSERT INTO t1 SELECT * FROM psptest1 WHERE col2 = @param2
ORDER BY col2
OPTION (OPTIMIZE FOR (@param2 UNKNOWN))
END
GO

CREATE TABLE t1 (col1 int primary key, col2 int, col3 binary(200));
GO

The setup code creates a table that has skewed data distribution. The optimal query plan differs based on which
parameter is selected. Unfortunately, the plan caching behavior doesn't always recompile the query based on the
most common parameter value. So, it's possible for a suboptimal plan to be cached and used for many values,
even when a different plan might be a better plan choice on average. Then the query plan creates two stored
procedures that are identical, except that one has a special query hint.
Example, part 1

-- Prime Procedure Cache with scan plan

EXEC psp1 @param1=1;
TRUNCATE TABLE t1;

-- Iterate multiple times to show the performance difference

DECLARE @i int = 0;
WHILE @i < 1000
BEGIN
EXEC psp1 @param1=2;
TRUNCATE TABLE t1;
SET @i += 1;
END

Example, part 2
(We recommend that you wait at least 10 minutes before you begin part 2 of the example, so that the results are
distinct in the resulting telemetry data.)

EXEC psp2 @param2=1;

TRUNCATE TABLE t1;

DECLARE @i int = 0;
WHILE @i < 1000
BEGIN
EXEC psp2 @param2=2;
TRUNCATE TABLE t1;
SET @i += 1;
END

Each part of this example attempts to run a parameterized insert statement 1,000 times (to generate a sufficient
load to use as a test data set). When it executes stored procedures, the query processor examines the parameter
value that is passed to the procedure during its first compilation (parameter "sniffing"). The processor caches the
resulting plan and uses it for later invocations, even if the parameter value is different. The optimal plan might not
be used in all cases. Sometimes you need to guide the optimizer to pick a plan that is better for the average case
rather than the specific case from when the query was first compiled. In this example, the initial plan generates a
"scan" plan that reads all rows to find each value that matches the parameter:

Because we executed the procedure by using the value 1, the resulting plan was optimal for the value 1 but was
suboptimal for all other values in the table. The result likely isn't what you would want if you were to pick each
plan randomly, because the plan performs more slowly and uses more resources.
If you run the test with SET STATISTICS IO set to ON , the logical scan work in this example is done behind the
scenes. You can see that there are 1,148 reads done by the plan (which is inefficient, if the average case is to return
just one row):
The second part of the example uses a query hint to tell the optimizer to use a specific value during the
compilation process. In this case, it forces the query processor to ignore the value that is passed as the parameter,
and instead to assume UNKNOWN . This refers to a value that has the average frequency in the table (ignoring
skew). The resulting plan is a seek-based plan that is faster and uses fewer resources, on average, than the plan in
part 1 of this example:

You can see the effect in the sys.resource_stats table (there is a delay from the time that you execute the test and
when the data populates the table). For this example, part 1 executed during the 22:25:00 time window, and part 2
executed at 22:35:00. The earlier time window used more resources in that time window than the later one
(because of plan efficiency improvements).

SELECT TOP 1000 *

FROM sys.resource_stats
WHERE database_name = 'resource1'
ORDER BY start_time DESC

NOTE
Although the volume in this example is intentionally small, the effect of suboptimal parameters can be substantial, especially
on larger databases. The difference, in extreme cases, can be between seconds for fast cases and hours for slow cases.

You can examine sys.resource_stats to determine whether the resource for a test uses more or fewer resources
than another test. When you compare data, separate the timing of tests so that they are not in the same 5-minute
window in the sys.resource_stats view. The goal of the exercise is to minimize the total amount of resources used,
and not to minimize the peak resources. Generally, optimizing a piece of code for latency also reduces resource
consumption. Make sure that the changes you make to an application are necessary, and that the changes don't
negatively affect the customer experience for someone who might be using query hints in the application.
If a workload has a set of repeating queries, often it makes sense to capture and validate the optimality of your
plan choices because it drives the minimum resource size unit required to host the database. After you validate it,
occasionally reexamine the plans to help you make sure that they have not degraded. You can learn more about
query hints (Transact-SQL).
Cross-database sharding
Because Azure SQL Database runs on commodity hardware, the capacity limits for a single database are lower
than for a traditional on-premises SQL Server installation. Some customers use sharding techniques to spread
database operations over multiple databases when the operations don't fit inside the limits of a single database in
Azure SQL Database. Most customers who use sharding techniques in Azure SQL Database split their data on a
single dimension across multiple databases. For this approach, you need to understand that OLTP applications
often perform transactions that apply to only one row or to a small group of rows in the schema.

NOTE
SQL Database now provides a library to assist with sharding. For more information, see Elastic Database client library
overview.

For example, if a database has customer name, order, and order details (like the traditional example Northwind
database that ships with SQL Server), you could split this data into multiple databases by grouping a customer
with the related order and order detail information. You can guarantee that the customer's data stays in a single
database. The application would split different customers across databases, effectively spreading the load across
multiple databases. With sharding, customers not only can avoid the maximum database size limit, but Azure SQL
Database also can process workloads that are significantly larger than the limits of the different performance
levels, as long as each individual database fits into its DTU.
Although database sharding doesn't reduce the aggregate resource capacity for a solution, it's highly effective at
supporting very large solutions that are spread over multiple databases. Each database can run at a different
performance level to support very large, "effective" databases with high resource requirements.
Functional partitioning
SQL Server users often combine many functions in a single database. For example, if an application has logic to
manage inventory for a store, that database might have logic associated with inventory, tracking purchase orders,
stored procedures, and indexed or materialized views that manage end-of-month reporting. This technique makes
it easier to administer the database for operations like backup, but it also requires you to size the hardware to
handle the peak load across all functions of an application.
If you use a scale-out architecture in Azure SQL Database, it's a good idea to split different functions of an
application into different databases. By using this technique, each application scales independently. As an
application becomes busier (and the load on the database increases), the administrator can choose independent
performance levels for each function in the application. At the limit, with this architecture, an application can be
larger than a single commodity machine can handle because the load is spread across multiple machines.
Batch queries
For applications that access data by using high-volume, frequent, ad hoc querying, a substantial amount of
response time is spent on network communication between the application tier and the Azure SQL Database tier.
Even when both the application and Azure SQL Database are in the same data center, the network latency between
the two might be magnified by a large number of data access operations. To reduce the network round trips for
the data access operations, consider using the option to either batch the ad hoc queries, or to compile them as
stored procedures. If you batch the ad hoc queries, you can send multiple queries as one large batch in a single trip
to Azure SQL Database. If you compile ad hoc queries in a stored procedure, you could achieve the same result as
if you batch them. Using a stored procedure also gives you the benefit of increasing the chances of caching the
query plans in Azure SQL Database so you can use the stored procedure again.
Some applications are write-intensive. Sometimes you can reduce the total I/O load on a database by considering
how to batch writes together. Often, this is as simple as using explicit transactions instead of auto-commit
transactions in stored procedures and ad hoc batches. For an evaluation of different techniques you can use, see
Batching techniques for SQL Database applications in Azure. Experiment with your own workload to find the right
model for batching. Be sure to understand that a model might have slightly different transactional consistency
guarantees. Finding the right workload that minimizes resource use requires finding the right combination of
consistency and performance trade-offs.
Application-tier caching
Some database applications have read-heavy workloads. Caching layers might reduce the load on the database
and might potentially reduce the performance level required to support a database by using Azure SQL Database.
With Azure Redis Cache, if you have a read-heavy workload, you can read the data once (or perhaps once per
application-tier machine, depending on how it is configured), and then store that data outside your SQL database.
This is a way to reduce database load (CPU and read I/O), but there is an effect on transactional consistency
because the data being read from the cache might be out of sync with the data in the database. Although in many
applications some level of inconsistency is acceptable, that's not true for all workloads. You should fully
understand any application requirements before you implement an application-tier caching strategy.

Next steps
For more information about service tiers, see SQL Database options and performance
For more information about elastic pools, see What is an Azure elastic pool?
For information about performance and elastic pools, see When to consider an elastic pool
How to use batching to improve SQL Database
application performance
4/19/2017 • 25 min to read • Edit Online

Batching operations to Azure SQL Database significantly improves the performance and scalability of your
applications. In order to understand the benefits, the first part of this article covers some sample test results that
compare sequential and batched requests to a SQL Database. The remainder of the article shows the techniques,
scenarios, and considerations to help you to use batching successfully in your Azure applications.

Why is batching important for SQL Database?

Batching calls to a remote service is a well-known strategy for increasing performance and scalability. There are
fixed processing costs to any interactions with a remote service, such as serialization, network transfer, and
deserialization. Packaging many separate transactions into a single batch minimizes these costs.
In this paper, we want to examine various SQL Database batching strategies and scenarios. Although these
strategies are also important for on-premises applications that use SQL Server, there are several reasons for
highlighting the use of batching for SQL Database:
There is potentially greater network latency in accessing SQL Database, especially if you are accessing SQL
Database from outside the same Microsoft Azure datacenter.
The multitenant characteristics of SQL Database means that the efficiency of the data access layer correlates to
the overall scalability of the database. SQL Database must prevent any single tenant/user from monopolizing
database resources to the detriment of other tenants. In response to usage in excess of predefined quotas, SQL
Database can reduce throughput or respond with throttling exceptions. Efficiencies, such as batching, enable you
to do more work on SQL Database before reaching these limits.
Batching is also effective for architectures that use multiple databases (sharding). The efficiency of your
interaction with each database unit is still a key factor in your overall scalability.
One of the benefits of using SQL Database is that you don’t have to manage the servers that host the database.
However, this managed infrastructure also means that you have to think differently about database optimizations.
You can no longer look to improve the database hardware or network infrastructure. Microsoft Azure controls
those environments. The main area that you can control is how your application interacts with SQL Database.
Batching is one of these optimizations.
The first part of the paper examines various batching techniques for .NET applications that use SQL Database. The
last two sections cover batching guidelines and scenarios.

Batching strategies
Note about timing results in this topic

NOTE
Results are not benchmarks but are meant to show relative performance. Timings are based on an average of at least 10
test runs. Operations are inserts into an empty table. These tests were measured awhile ago, and they do not necessarily
correspond to throughput that you might experience today. The relative benefit of the batching technique should be similar.

Transactions
It seems strange to begin a review of batching by discussing transactions. But the use of client-side transactions has
a subtle server-side batching effect that improves performance. And transactions can be added with only a few
lines of code, so they provide a fast way to improve performance of sequential operations.
Consider the following C# code that contains a sequence of insert and update operations on a simple table.

List<string> dbOperations = new List<string>();

dbOperations.Add("update MyTable set mytext = 'updated text' where id = 1");
dbOperations.Add("update MyTable set mytext = 'updated text' where id = 2");
dbOperations.Add("update MyTable set mytext = 'updated text' where id = 3");
dbOperations.Add("insert MyTable values ('new value',1)");
dbOperations.Add("insert MyTable values ('new value',2)");
dbOperations.Add("insert MyTable values ('new value',3)");

The following ADO.NET code sequentially performs these operations.

using (SqlConnection connection = new SqlConnection(CloudConfigurationManager.GetSetting("Sql.ConnectionString")))

{
conn.Open();

foreach(string commandString in dbOperations)

{
SqlCommand cmd = new SqlCommand(commandString, conn);
cmd.ExecuteNonQuery();
}
}

The best way to optimize this code is to implement some form of client-side batching of these calls. But there is a
simple way to increase the performance of this code by simply wrapping the sequence of calls in a transaction.
Here is the same code that uses a transaction.

using (SqlConnection connection = new SqlConnection(CloudConfigurationManager.GetSetting("Sql.ConnectionString")))

{
conn.Open();
SqlTransaction transaction = conn.BeginTransaction();

foreach (string commandString in dbOperations)

{
SqlCommand cmd = new SqlCommand(commandString, conn, transaction);
cmd.ExecuteNonQuery();
}

transaction.Commit();
}

Transactions are actually being used in both of these examples. In the first example, each individual call is an
implicit transaction. In the second example, an explicit transaction wraps all of the calls. Per the documentation for
the write-ahead transaction log, log records are flushed to the disk when the transaction commits. So by including
more calls in a transaction, the write to the transaction log can delay until the transaction is committed. In effect,
you are enabling batching for the writes to the server’s transaction log.
The following table shows some ad-hoc testing results. The tests performed the same sequential inserts with and
without transactions. For more perspective, the first set of tests ran remotely from a laptop to the database in
Microsoft Azure. The second set of tests ran from a cloud service and database that both resided within the same
Microsoft Azure datacenter (West US). The following table shows the duration in milliseconds of sequential inserts
with and without transactions.
On-Premises to Azure:
OPERATIONS NO TRANSACTION (MS) TRANSACTION (MS)

1 130 402

10 1208 1226

100 12662 10395

1000 128852 102917

Azure to Azure (same datacenter):

OPERATIONS NO TRANSACTION (MS) TRANSACTION (MS)

1 21 26

10 220 56

100 2145 341

1000 21479 2756

NOTE
Results are not benchmarks. See the note about timing results in this topic.

Based on the previous test results, wrapping a single operation in a transaction actually decreases performance. But
as you increase the number of operations within a single transaction, the performance improvement becomes
more marked. The performance difference is also more noticeable when all operations occur within the Microsoft
Azure datacenter. The increased latency of using SQL Database from outside the Microsoft Azure datacenter
overshadows the performance gain of using transactions.
Although the use of transactions can increase performance, continue to observe best practices for transactions and
connections. Keep the transaction as short as possible, and close the database connection after the work completes.
The using statement in the previous example assures that the connection is closed when the subsequent code block
completes.
The previous example demonstrates that you can add a local transaction to any ADO.NET code with two lines.
Transactions offer a quick way to improve the performance of code that makes sequential insert, update, and delete
operations. However, for the fastest performance, consider changing the code further to take advantage of client-
side batching, such as table-valued parameters.
For more information about transactions in ADO.NET, see Local Transactions in ADO.NET.
Table -valued parameters
Table-valued parameters support user-defined table types as parameters in Transact-SQL statements, stored
procedures, and functions. This client-side batching technique allows you to send multiple rows of data within the
table-valued parameter. To use table-valued parameters, first define a table type. The following Transact-SQL
statement creates a table type named MyTableType.

CREATE TYPE MyTableType AS TABLE

( mytext TEXT,
num INT );
In code, you create a DataTable with the exact same names and types of the table type. Pass this DataTable in a
parameter in a text query or stored procedure call. The following example shows this technique:

using (SqlConnection connection = new SqlConnection(CloudConfigurationManager.GetSetting("Sql.ConnectionString")))

{
connection.Open();

DataTable table = new DataTable();

// Add columns and rows. The following is a simple example.
table.Columns.Add("mytext", typeof(string));
table.Columns.Add("num", typeof(int));
for (var i = 0; i < 10; i++)
{
table.Rows.Add(DateTime.Now.ToString(), DateTime.Now.Millisecond);
}

SqlCommand cmd = new SqlCommand(

"INSERT INTO MyTable(mytext, num) SELECT mytext, num FROM @TestTvp",
connection);

cmd.Parameters.Add(
new SqlParameter()
{
ParameterName = "@TestTvp",
SqlDbType = SqlDbType.Structured,
TypeName = "MyTableType",
Value = table,
});

cmd.ExecuteNonQuery();
}

In the previous example, the SqlCommand object inserts rows from a table-valued parameter, @TestTvp. The
previously created DataTable object is assigned to this parameter with the SqlCommand.Parameters.Add
method. Batching the inserts in one call significantly increases the performance over sequential inserts.
To improve the previous example further, use a stored procedure instead of a text-based command. The following
Transact-SQL command creates a stored procedure that takes the SimpleTestTableType table-valued parameter.

CREATE PROCEDURE [dbo].[sp_InsertRows]

@TestTvp as MyTableType READONLY
AS
BEGIN
INSERT INTO MyTable(mytext, num)
SELECT mytext, num FROM @TestTvp
END
GO

Then change the SqlCommand object declaration in the previous code example to the following.

SqlCommand cmd = new SqlCommand("sp_InsertRows", connection);

cmd.CommandType = CommandType.StoredProcedure;

In most cases, table-valued parameters have equivalent or better performance than other batching techniques.
Table-valued parameters are often preferable, because they are more flexible than other options. For example,
other techniques, such as SQL bulk copy, only permit the insertion of new rows. But with table-valued parameters,
you can use logic in the stored procedure to determine which rows are updates and which are inserts. The table
type can also be modified to contain an “Operation” column that indicates whether the specified row should be
inserted, updated, or deleted.
The following table shows ad-hoc test results for the use of table-valued parameters in milliseconds.

OPERATIONS ON-PREMISES TO AZURE (MS) AZURE SAME DATACENTER (MS)

1 124 32

10 131 25

100 338 51

1000 2615 382

10000 23830 3586

NOTE
Results are not benchmarks. See the note about timing results in this topic.

The performance gain from batching is immediately apparent. In the previous sequential test, 1000 operations took
129 seconds outside the datacenter and 21 seconds from within the datacenter. But with table-valued parameters,
1000 operations take only 2.6 seconds outside the datacenter and 0.4 seconds within the datacenter.
For more information on table-valued parameters, see Table-Valued Parameters.
SQL bulk copy
SQL bulk copy is another way to insert large amounts of data into a target database. .NET applications can use the
SqlBulkCopy class to perform bulk insert operations. SqlBulkCopy is similar in function to the command-line
tool, Bcp.exe, or the Transact-SQL statement, BULK INSERT. The following code example shows how to bulk copy
the rows in the source DataTable, table, to the destination table in SQL Server, MyTable.

using (SqlConnection connection = new SqlConnection(CloudConfigurationManager.GetSetting("Sql.ConnectionString")))

{
connection.Open();

using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connection))

{
bulkCopy.DestinationTableName = "MyTable";
bulkCopy.ColumnMappings.Add("mytext", "mytext");
bulkCopy.ColumnMappings.Add("num", "num");
bulkCopy.WriteToServer(table);
}
}

There are some cases where bulk copy is preferred over table-valued parameters. See the comparison table of
Table-Valued parameters versus BULK INSERT operations in the topic Table-Valued Parameters.
The following ad-hoc test results show the performance of batching with SqlBulkCopy in milliseconds.

OPERATIONS ON-PREMISES TO AZURE (MS) AZURE SAME DATACENTER (MS)

1 433 57

10 441 32

100 636 53
OPERATIONS ON-PREMISES TO AZURE (MS) AZURE SAME DATACENTER (MS)

1000 2535 341

10000 21605 2737

NOTE
Results are not benchmarks. See the note about timing results in this topic.

In smaller batch sizes, the use table-valued parameters outperformed the SqlBulkCopy class. However,
SqlBulkCopy performed 12-31% faster than table-valued parameters for the tests of 1,000 and 10,000 rows. Like
table-valued parameters, SqlBulkCopy is a good option for batched inserts, especially when compared to the
performance of non-batched operations.
For more information on bulk copy in ADO.NET, see Bulk Copy Operations in SQL Server.
Multiple -row Parameterized INSERT statements
One alternative for small batches is to construct a large parameterized INSERT statement that inserts multiple rows.
The following code example demonstrates this technique.

using (SqlConnection connection = new SqlConnection(CloudConfigurationManager.GetSetting("Sql.ConnectionString")))

{
connection.Open();

string insertCommand = "INSERT INTO [MyTable] ( mytext, num ) " +

"VALUES (@p1, @p2), (@p3, @p4), (@p5, @p6), (@p7, @p8), (@p9, @p10)";

SqlCommand cmd = new SqlCommand(insertCommand, connection);

for (int i = 1; i <= 10; i += 2)

{
cmd.Parameters.Add(new SqlParameter("@p" + i.ToString(), "test"));
cmd.Parameters.Add(new SqlParameter("@p" + (i+1).ToString(), i));
}

cmd.ExecuteNonQuery();
}

This example is meant to show the basic concept. A more realistic scenario would loop through the required entities
to construct the query string and the command parameters simultaneously. You are limited to a total of 2100 query
parameters, so this limits the total number of rows that can be processed in this manner.
The following ad-hoc test results show the performance of this type of insert statement in milliseconds.

OPERATIONS TABLE-VALUED PARAMETERS (MS) SINGLE-STATEMENT INSERT (MS)

1 32 20

10 30 25

100 33 51
NOTE
Results are not benchmarks. See the note about timing results in this topic.

This approach can be slightly faster for batches that are less than 100 rows. Although the improvement is small,
this technique is another option that might work well in your specific application scenario.
DataAdapter
The DataAdapter class allows you to modify a DataSet object and then submit the changes as INSERT, UPDATE,
and DELETE operations. If you are using the DataAdapter in this manner, it is important to note that separate calls
are made for each distinct operation. To improve performance, use the UpdateBatchSize property to the number
of operations that should be batched at a time. For more information, see Performing Batch Operations Using
DataAdapters.
Entity framework
Entity Framework does not currently support batching. Different developers in the community have attempted to
demonstrate workarounds, such as override the SaveChanges method. But the solutions are typically complex and
customized to the application and data model. The Entity Framework codeplex project currently has a discussion
page on this feature request. To view this discussion, see Design Meeting Notes - August 2, 2012.
XML
For completeness, we feel that it is important to talk about XML as a batching strategy. However, the use of XML
has no advantages over other methods and several disadvantages. The approach is similar to table-valued
parameters, but an XML file or string is passed to a stored procedure instead of a user-defined table. The stored
procedure parses the commands in the stored procedure.
There are several disadvantages to this approach:
Working with XML can be cumbersome and error prone.
Parsing the XML on the database can be CPU-intensive.
In most cases, this method is slower than table-valued parameters.
For these reasons, the use of XML for batch queries is not recommended.

Batching considerations
The following sections provide more guidance for the use of batching in SQL Database applications.
Tradeoffs
Depending on your architecture, batching can involve a tradeoff between performance and resiliency. For example,
consider the scenario where your role unexpectedly goes down. If you lose one row of data, the impact is smaller
than the impact of losing a large batch of unsubmitted rows. There is a greater risk when you buffer rows before
sending them to the database in a specified time window.
Because of this tradeoff, evaluate the type of operations that you batch. Batch more aggressively (larger batches
and longer time windows) with data that is less critical.
Batch size
In our tests, there was typically no advantage to breaking large batches into smaller chunks. In fact, this subdivision
often resulted in slower performance than submitting a single large batch. For example, consider a scenario where
you want to insert 1000 rows. The following table shows how long it takes to use table-valued parameters to insert
1000 rows when divided into smaller batches.
BATCH SIZE ITERATIONS TABLE-VALUED PARAMETERS (MS)

1000 1 347

500 2 355

100 10 465

50 20 630

NOTE
Results are not benchmarks. See the note about timing results in this topic.

You can see that the best performance for 1000 rows is to submit them all at once. In other tests (not shown here)
there was a small performance gain to break a 10000 row batch into two batches of 5000. But the table schema for
these tests is relatively simple, so you should perform tests on your specific data and batch sizes to verify these
findings.
Another factor to consider is that if the total batch becomes too large, SQL Database might throttle and refuse to
commit the batch. For the best results, test your specific scenario to determine if there is an ideal batch size. Make
the batch size configurable at runtime to enable quick adjustments based on performance or errors.
Finally, balance the size of the batch with the risks associated with batching. If there are transient errors or the role
fails, consider the consequences of retrying the operation or of losing the data in the batch.
Parallel processing
What if you took the approach of reducing the batch size but used multiple threads to execute the work? Again, our
tests showed that several smaller multithreaded batches typically performed worse than a single larger batch. The
following test attempts to insert 1000 rows in one or more parallel batches. This test shows how more
simultaneous batches actually decreased performance.

BATCH SIZE [ITERATIONS] TWO THREADS (MS) FOUR THREADS (MS) SIX THREADS (MS)

1000 [1] 277 315 266

500 [2] 548 278 256

250 [4] 405 329 265

100 [10] 488 439 391

NOTE
Results are not benchmarks. See the note about timing results in this topic.

There are several potential reasons for the degradation in performance due to parallelism:
There are multiple simultaneous network calls instead of one.
Multiple operations against a single table can result in contention and blocking.
There are overheads associated with multithreading.
The expense of opening multiple connections outweighs the benefit of parallel processing.
If you target different tables or databases, it is possible to see some performance gain with this strategy. Database
sharding or federations would be a scenario for this approach. Sharding uses multiple databases and routes
different data to each database. If each small batch is going to a different database, then performing the operations
in parallel can be more efficient. However, the performance gain is not significant enough to use as the basis for a
decision to use database sharding in your solution.
In some designs, parallel execution of smaller batches can result in improved throughput of requests in a system
under load. In this case, even though it is quicker to process a single larger batch, processing multiple batches in
parallel might be more efficient.
If you do use parallel execution, consider controlling the maximum number of worker threads. A smaller number
might result in less contention and a faster execution time. Also, consider the additional load that this places on the
target database both in connections and transactions.
Related performance factors
Typical guidance on database performance also affects batching. For example, insert performance is reduced for
tables that have a large primary key or many nonclustered indexes.
If table-valued parameters use a stored procedure, you can use the command SET NOCOUNT ON at the beginning
of the procedure. This statement suppresses the return of the count of the affected rows in the procedure. However,
in our tests, the use of SET NOCOUNT ON either had no effect or decreased performance. The test stored
procedure was simple with a single INSERT command from the table-valued parameter. It is possible that more
complex stored procedures would benefit from this statement. But don’t assume that adding SET NOCOUNT ON
to your stored procedure automatically improves performance. To understand the effect, test your stored
procedure with and without the SET NOCOUNT ON statement.

Batching scenarios
The following sections describe how to use table-valued parameters in three application scenarios. The first
scenario shows how buffering and batching can work together. The second scenario improves performance by
performing master-detail operations in a single stored procedure call. The final scenario shows how to use table-
valued parameters in an “UPSERT” operation.
Buffering
Although there are some scenarios that are obvious candidate for batching, there are many scenarios that could
take advantage of batching by delayed processing. However, delayed processing also carries a greater risk that the
data is lost in the event of an unexpected failure. It is important to understand this risk and consider the
consequences.
For example, consider a web application that tracks the navigation history of each user. On each page request, the
application could make a database call to record the user’s page view. But higher performance and scalability can
be achieved by buffering the users’ navigation activities and then sending this data to the database in batches. You
can trigger the database update by elapsed time and/or buffer size. For example, a rule could specify that the batch
should be processed after 20 seconds or when the buffer reaches 1000 items.
The following code example uses Reactive Extensions - Rx to process buffered events raised by a monitoring class.
When the buffer fills or a timeout is reached, the batch of user data is sent to the database with a table-valued
parameter.
The following NavHistoryData class models the user navigation details. It contains basic information such as the
user identifier, the URL accessed, and the access time.
public class NavHistoryData
{
public NavHistoryData(int userId, string url, DateTime accessTime)
{ UserId = userId; URL = url; AccessTime = accessTime; }
public int UserId { get; set; }
public string URL { get; set; }
public DateTime AccessTime { get; set; }
}

The NavHistoryDataMonitor class is responsible for buffering the user navigation data to the database. It contains a
method, RecordUserNavigationEntry, which responds by raising an OnAdded event. The following code shows the
constructor logic that uses Rx to create an observable collection based on the event. It then subscribes to this
observable collection with the Buffer method. The overload specifies that the buffer should be sent every 20
seconds or 1000 entries.

public NavHistoryDataMonitor()
{
var observableData =
Observable.FromEventPattern<NavHistoryDataEventArgs>(this, "OnAdded");

observableData.Buffer(TimeSpan.FromSeconds(20), 1000).Subscribe(Handler);
}

The handler converts all of the buffered items into a table-valued type and then passes this type to a stored
procedure that processes the batch. The following code shows the complete definition for both the
NavHistoryDataEventArgs and the NavHistoryDataMonitor classes.
public class NavHistoryDataEventArgs : System.EventArgs
{
public NavHistoryDataEventArgs(NavHistoryData data) { Data = data; }
public NavHistoryData Data { get; set; }
}

public class NavHistoryDataMonitor

{
public event EventHandler<NavHistoryDataEventArgs> OnAdded;

public NavHistoryDataMonitor()
{
var observableData =
Observable.FromEventPattern<NavHistoryDataEventArgs>(this, "OnAdded");

observableData.Buffer(TimeSpan.FromSeconds(20), 1000).Subscribe(Handler);
}

public void RecordUserNavigationEntry(NavHistoryData data)

{
if (OnAdded != null)
OnAdded(this, new NavHistoryDataEventArgs(data));
}

protected void Handler(IList<EventPattern<NavHistoryDataEventArgs>> items)

{
DataTable navHistoryBatch = new DataTable("NavigationHistoryBatch");
navHistoryBatch.Columns.Add("UserId", typeof(int));
navHistoryBatch.Columns.Add("URL", typeof(string));
navHistoryBatch.Columns.Add("AccessTime", typeof(DateTime));
foreach (EventPattern<NavHistoryDataEventArgs> item in items)
{
NavHistoryData data = item.EventArgs.Data;
navHistoryBatch.Rows.Add(data.UserId, data.URL, data.AccessTime);
}

using (SqlConnection connection = new SqlConnection(CloudConfigurationManager.GetSetting("Sql.ConnectionString")))

{
connection.Open();

SqlCommand cmd = new SqlCommand("sp_RecordUserNavigation", connection);

cmd.CommandType = CommandType.StoredProcedure;

cmd.Parameters.Add(
new SqlParameter()
{
ParameterName = "@NavHistoryBatch",
SqlDbType = SqlDbType.Structured,
TypeName = "NavigationHistoryTableType",
Value = navHistoryBatch,
});

cmd.ExecuteNonQuery();
}
}
}

To use this buffering class, the application creates a static NavHistoryDataMonitor object. Each time a user accesses
a page, the application calls the NavHistoryDataMonitor.RecordUserNavigationEntry method. The buffering logic
proceeds to take care of sending these entries to the database in batches.
Master detail
Table-valued parameters are useful for simple INSERT scenarios. However, it can be more challenging to batch
inserts that involve more than one table. The “master/detail” scenario is a good example. The master table identifies
the primary entity. One or more detail tables store more data about the entity. In this scenario, foreign key
relationships enforce the relationship of details to a unique master entity. Consider a simplified version of a
PurchaseOrder table and its associated OrderDetail table. The following Transact-SQL creates the PurchaseOrder
table with four columns: OrderID, OrderDate, CustomerID, and Status.

CREATE TABLE [dbo].[PurchaseOrder](

[OrderID] [int] IDENTITY(1,1) NOT NULL,
[OrderDate] [datetime] NOT NULL,
[CustomerID] [int] NOT NULL,
[Status] [nvarchar](50) NOT NULL,
CONSTRAINT [PrimaryKey_PurchaseOrder]
PRIMARY KEY CLUSTERED ( [OrderID] ASC ))

Each order contains one or more product purchases. This information is captured in the PurchaseOrderDetail table.
The following Transact-SQL creates the PurchaseOrderDetail table with five columns: OrderID, OrderDetailID,
ProductID, UnitPrice, and OrderQty.

CREATE TABLE [dbo].[PurchaseOrderDetail](

[OrderID] [int] NOT NULL,
[OrderDetailID] [int] IDENTITY(1,1) NOT NULL,
[ProductID] [int] NOT NULL,
[UnitPrice] [money] NULL,
[OrderQty] [smallint] NULL,
CONSTRAINT [PrimaryKey_PurchaseOrderDetail] PRIMARY KEY CLUSTERED
( [OrderID] ASC, [OrderDetailID] ASC ))

The OrderID column in the PurchaseOrderDetail table must reference an order from the PurchaseOrder table. The
following definition of a foreign key enforces this constraint.

ALTER TABLE [dbo].[PurchaseOrderDetail] WITH CHECK ADD

CONSTRAINT [FK_OrderID_PurchaseOrder] FOREIGN KEY([OrderID])
REFERENCES [dbo].[PurchaseOrder] ([OrderID])

In order to use table-valued parameters, you must have one user-defined table type for each target table.

CREATE TYPE PurchaseOrderTableType AS TABLE

( OrderID INT,
OrderDate DATETIME,
CustomerID INT,
Status NVARCHAR(50) );
GO

CREATE TYPE PurchaseOrderDetailTableType AS TABLE

( OrderID INT,
ProductID INT,
UnitPrice MONEY,
OrderQty SMALLINT );
GO

Then define a stored procedure that accepts tables of these types. This procedure allows an application to locally
batch a set of orders and order details in a single call. The following Transact-SQL provides the complete stored
procedure declaration for this purchase order example.
CREATE PROCEDURE sp_InsertOrdersBatch (
@orders as PurchaseOrderTableType READONLY,
@details as PurchaseOrderDetailTableType READONLY )
AS
SET NOCOUNT ON;

-- Table that connects the order identifiers in the @orders

-- table with the actual order identifiers in the PurchaseOrder table
DECLARE @IdentityLink AS TABLE (
SubmittedKey int,
ActualKey int,
RowNumber int identity(1,1)
);

-- Add new orders to the PurchaseOrder table, storing the actual

-- order identifiers in the @IdentityLink table
INSERT INTO PurchaseOrder ([OrderDate], [CustomerID], [Status])
OUTPUT inserted.OrderID INTO @IdentityLink (ActualKey)
SELECT [OrderDate], [CustomerID], [Status] FROM @orders ORDER BY OrderID;

-- Match the passed-in order identifiers with the actual identifiers

-- and complete the @IdentityLink table for use with inserting the details
WITH OrderedRows As (
SELECT OrderID, ROW_NUMBER () OVER (ORDER BY OrderID) As RowNumber
FROM @orders
)
UPDATE @IdentityLink SET SubmittedKey = M.OrderID
FROM @IdentityLink L JOIN OrderedRows M ON L.RowNumber = M.RowNumber;

-- Insert the order details into the PurchaseOrderDetail table,

-- using the actual order identifiers of the master table, PurchaseOrder
INSERT INTO PurchaseOrderDetail (
[OrderID],
[ProductID],
[UnitPrice],
[OrderQty] )
SELECT L.ActualKey, D.ProductID, D.UnitPrice, D.OrderQty
FROM @details D
JOIN @IdentityLink L ON L.SubmittedKey = D.OrderID;
GO

In this example, the locally defined @IdentityLink table stores the actual OrderID values from the newly inserted
rows. These order identifiers are different from the temporary OrderID values in the @orders and @details table-
valued parameters. For this reason, the @IdentityLink table then connects the OrderID values from the @orders
parameter to the real OrderID values for the new rows in the PurchaseOrder table. After this step, the @IdentityLink
table can facilitate inserting the order details with the actual OrderID that satisfies the foreign key constraint.
This stored procedure can be used from code or from other Transact-SQL calls. See the table-valued parameters
section of this paper for a code example. The following Transact-SQL shows how to call the sp_InsertOrdersBatch.
declare @orders as PurchaseOrderTableType
declare @details as PurchaseOrderDetailTableType

INSERT @orders
([OrderID], [OrderDate], [CustomerID], [Status])
VALUES(1, '1/1/2013', 1125, 'Complete'),
(2, '1/13/2013', 348, 'Processing'),
(3, '1/12/2013', 2504, 'Shipped')

INSERT @details
([OrderID], [ProductID], [UnitPrice], [OrderQty])
VALUES(1, 10, $11.50, 1),
(1, 12, $1.58, 1),
(2, 23, $2.57, 2),
(3, 4, $10.00, 1)

exec sp_InsertOrdersBatch @orders, @details

This solution allows each batch to use a set of OrderID values that begin at 1. These temporary OrderID values
describe the relationships in the batch, but the actual OrderID values are determined at the time of the insert
operation. You can run the same statements in the previous example repeatedly and generate unique orders in the
database. For this reason, consider adding more code or database logic that prevents duplicate orders when using
this batching technique.
This example demonstrates that even more complex database operations, such as master-detail operations, can be
batched using table-valued parameters.
UPSERT
Another batching scenario involves simultaneously updating existing rows and inserting new rows. This operation
is sometimes referred to as an “UPSERT” (update + insert) operation. Rather than making separate calls to INSERT
and UPDATE, the MERGE statement is best suited to this task. The MERGE statement can perform both insert and
update operations in a single call.
Table-valued parameters can be used with the MERGE statement to perform updates and inserts. For example,
consider a simplified Employee table that contains the following columns: EmployeeID, FirstName, LastName,
SocialSecurityNumber:

CREATE TABLE [dbo].[Employee](

[EmployeeID] [int] IDENTITY(1,1) NOT NULL,
[FirstName] [nvarchar](50) NOT NULL,
[LastName] [nvarchar](50) NOT NULL,
[SocialSecurityNumber] [nvarchar](50) NOT NULL,
CONSTRAINT [PrimaryKey_Employee] PRIMARY KEY CLUSTERED
([EmployeeID] ASC ))

In this example, you can use the fact that the SocialSecurityNumber is unique to perform a MERGE of multiple
employees. First, create the user-defined table type:

CREATE TYPE EmployeeTableType AS TABLE

( Employee_ID INT,
FirstName NVARCHAR(50),
LastName NVARCHAR(50),
SocialSecurityNumber NVARCHAR(50) );
GO

Next, create a stored procedure or write code that uses the MERGE statement to perform the update and insert. The
following example uses the MERGE statement on a table-valued parameter, @employees, of type
EmployeeTableType. The contents of the @employees table are not shown here.
MERGE Employee AS target
USING (SELECT [FirstName], [LastName], [SocialSecurityNumber] FROM @employees)
AS source ([FirstName], [LastName], [SocialSecurityNumber])
ON (target.[SocialSecurityNumber] = source.[SocialSecurityNumber])
WHEN MATCHED THEN
UPDATE SET
target.FirstName = source.FirstName,
target.LastName = source.LastName
WHEN NOT MATCHED THEN
INSERT ([FirstName], [LastName], [SocialSecurityNumber])
VALUES (source.[FirstName], source.[LastName], source.[SocialSecurityNumber]);

For more information, see the documentation and examples for the MERGE statement. Although the same work
could be performed in a multiple-step stored procedure call with separate INSERT and UPDATE operations, the
MERGE statement is more efficient. Database code can also construct Transact-SQL calls that use the MERGE
statement directly without requiring two database calls for INSERT and UPDATE.

Recommendation summary
The following list provides a summary of the batching recommendations discussed in this topic:
Use buffering and batching to increase the performance and scalability of SQL Database applications.
Understand the tradeoffs between batching/buffering and resiliency. During a role failure, the risk of losing an
unprocessed batch of business-critical data might outweigh the performance benefit of batching.
Attempt to keep all calls to the database within a single datacenter to reduce latency.
If you choose a single batching technique, table-valued parameters offer the best performance and flexibility.
For the fastest insert performance, follow these general guidelines but test your scenario:
For < 100 rows, use a single parameterized INSERT command.
For < 1000 rows, use table-valued parameters.
For >= 1000 rows, use SqlBulkCopy.
For update and delete operations, use table-valued parameters with stored procedure logic that determines the
correct operation on each row in the table parameter.
Batch size guidelines:
Use the largest batch sizes that make sense for your application and business requirements.
Balance the performance gain of large batches with the risks of temporary or catastrophic failures. What
is the consequence of retries or loss of the data in the batch?
Test the largest batch size to verify that SQL Database does not reject it.
Create configuration settings that control batching, such as the batch size or the buffering time window.
These settings provide flexibility. You can change the batching behavior in production without
redeploying the cloud service.
Avoid parallel execution of batches that operate on a single table in one database. If you do choose to divide a
single batch across multiple worker threads, run tests to determine the ideal number of threads. After an
unspecified threshold, more threads will decrease performance rather than increase it.
Consider buffering on size and time as a way of implementing batching for more scenarios.

Next steps
This article focused on how database design and coding techniques related to batching can improve your
application performance and scalability. But this is just one factor in your overall strategy. For more ways to
improve performance and scalability, see Azure SQL Database performance guidance for single databases and
Price and performance considerations for an elastic pool.
Extended events in SQL Database
4/10/2017 • 5 min to read • Edit Online

This topic explains how the implementation of extended events in Azure SQL Database is slightly different
compared to extended events in Microsoft SQL Server.
SQL Database gained the extended events feature in the second half of calendar 2015.
SQL Server has had extended events since 2008.
The feature set of extended events on SQL Database is a robust subset of the features on SQL Server.
XEvents is an informal nickname that is sometimes used for 'extended events' in blogs and other informal
locations.
Additional information about extended events, for Azure SQL Database and Microsoft SQL Server, is available at:
Quick Start: Extended events in SQL Server
Extended Events

Prerequisites
This topic assumes you already have some knowledge of:
Azure SQL Database service.
Extended events in Microsoft SQL Server.
The bulk of our documentation about extended events applies to both SQL Server and SQL Database.
Prior exposure to the following items is helpful when choosing the Event File as the target:
Azure Storage service
PowerShell
Using Azure PowerShell with Azure Storage - Provides comprehensive information about PowerShell
and the Azure Storage service.

Code samples
Related topics provide two code samples:
Ring Buffer target code for extended events in SQL Database
Short simple Transact-SQL script.
We emphasize in the code sample topic that, when you are done with a Ring Buffer target, you should
release its resources by executing an alter-drop ALTER EVENT SESSION ... ON DATABASE DROP TARGET ...;
statement. Later you can add another instance of Ring Buffer by
ALTER EVENT SESSION ... ON DATABASE ADD TARGET ... .
Event File target code for extended events in SQL Database
Phase 1 is PowerShell to create an Azure Storage container.
Phase 2 is Transact-SQL that uses the Azure Storage container.

Transact-SQL differences
When you execute the CREATE EVENT SESSION command on SQL Server, you use the ON SERVER clause.
But on SQL Database you use the ON DATABASE clause instead.
The ON DATABASE clause also applies to the ALTER EVENT SESSION and DROP EVENT SESSION Transact-
SQL commands.
A best practice is to include the event session option of STARTUP_STATE = ON in your CREATE EVENT
SESSION or ALTER EVENT SESSION statements.
The = ON value supports an automatic restart after a reconfiguration of the logical database due to a
failover.

New catalog views

The extended events feature is supported by several catalog views. Catalog views tell you about metadata or
definitions of user-created event sessions in the current database. The views do not return information about
instances of active event sessions.

NAME OF
CATALOG VIEW DESCRIPTION

sys.database_event_session_actions Returns a row for each action on each event of an event

session.

sys.database_event_session_events Returns a row for each event in an event session.

sys.database_event_session_fields Returns a row for each customize-able column that was

explicitly set on events and targets.

sys.database_event_session_targets Returns a row for each event target for an event session.

sys.database_event_sessions Returns a row for each event session in the SQL Database
database.

In Microsoft SQL Server, similar catalog views have names that include .server_ instead of .database_. The name
pattern is like sys.server_event_%.

New dynamic management views (DMVs)

Azure SQL Database has dynamic management views (DMVs) that support extended events. DMVs tell you about
active event sessions.

NAME OF DMV DESCRIPTION

sys.dm_xe_database_session_event_actions Returns information about event session actions.

sys.dm_xe_database_session_events Returns information about session events.

sys.dm_xe_database_session_object_columns Shows the configuration values for objects that are bound to
a session.

sys.dm_xe_database_session_targets Returns information about session targets.

sys.dm_xe_database_sessions Returns a row for each event session that is scoped to the
current database.
In Microsoft SQL Server, similar catalog views are named without the _database portion of the name, such as:
sys.dm_xe_sessions, instead of name
sys.dm_xe_database_sessions.
DMVs common to both
For extended events there are additional DMVs that are common to both Azure SQL Database and Microsoft SQL
Server:
sys.dm_xe_map_values
sys.dm_xe_object_columns
sys.dm_xe_objects
sys.dm_xe_packages

Find the available extended events, actions, and targets

You can run a simple SQL SELECT to obtain a list of the available events, actions, and target.

SELECT
o.object_type,
p.name AS [package_name],
o.name AS [db_object_name],
o.description AS [db_obj_description]
FROM
sys.dm_xe_objects AS o
INNER JOIN sys.dm_xe_packages AS p ON p.guid = o.package_guid
WHERE
o.object_type in
(
'action', 'event', 'target'
)
ORDER BY
o.object_type,
p.name,
o.name;

Targets for your SQL Database event sessions

Here are targets that can capture results from your event sessions on SQL Database:
Ring Buffer target - Briefly holds event data in memory.
Event Counter target - Counts all events that occur during an extended events session.
Event File target - Writes complete buffers to an Azure Storage container.
The Event Tracing for Windows (ETW) API is not available for extended events on SQL Database.

Restrictions
There are a couple of security-related differences befitting the cloud environment of SQL Database:
Extended events are founded on the single-tenant isolation model. An event session in one database cannot
access data or events from another database.
You cannot issue a CREATE EVENT SESSION statement in the context of the master database.

Permission model
You must have Control permission on the database to issue a CREATE EVENT SESSION statement. The database
owner (dbo) has Control permission.
Storage container authorizations
The SAS token you generate for your Azure Storage container must specify rwl for the permissions. The rwl value
provides the following permissions:
Read
Write
List

Performance considerations
There are scenarios where intensive use of extended events can accumulate more active memory than is healthy
for the overall system. Therefore the Azure SQL Database system dynamically sets and adjusts limits on the
amount of active memory that can be accumulated by an event session. Many factors go into the dynamic
calculation.
If you receive an error message that says a memory maximum was enforced, some corrective actions you can take
are:
Run fewer concurrent event sessions.
Through your CREATE and ALTER statements for event sessions, reduce the amount of memory you specify
on the MAX_MEMORY clause.
Network latency
The Event File target might experience network latency or failures while persisting data to Azure Storage blobs.
Other events in SQL Database might be delayed while they wait for the network communication to complete. This
delay can slow your workload.
To mitigate this performance risk, avoid setting the EVENT_RETENTION_MODE option to NO_EVENT_LOSS
in your event session definitions.

Related links
Using Azure PowerShell with Azure Storage.
Azure Storage Cmdlets
Using Azure PowerShell with Azure Storage - Provides comprehensive information about PowerShell and the
Azure Storage service.
How to use Blob storage from .NET
CREATE CREDENTIAL (Transact-SQL)
CREATE EVENT SESSION (Transact-SQL)
Jonathan Kehayias' blog posts about extended events in Microsoft SQL Server
The Azure Service Updates webpage, narrowed by parameter to Azure SQL Database:
https://azure.microsoft.com/updates/?service=sql-database
Other code sample topics for extended events are available at the following links. However, you must routinely
check any sample to see whether the sample targets Microsoft SQL Server versus Azure SQL Database. Then you
can decide whether minor changes are needed to run the sample.
Improved query performance with compatibility Level
130 in Azure SQL Database
4/10/2017 • 17 min to read • Edit Online

Azure SQL Database is running transparently hundreds of thousands of databases at many different compatibility
levels, preserving and guaranteeing the backward compatibility to the corresponding version of Microsoft SQL
Server for all its customers!
In this article, we explore the benefits of running your Azure SQL Database at compatibility level 130, and using the
benefits of the new query optimizer and query processor features. We also address the possible side effects on the
query performance for the existing SQL applications.
As a reminder of history, the alignment of SQL versions to default compatibility levels are as follows:
100: in SQL Server 2008 and Azure SQL Database.
110: in SQL Server 2012 and Azure SQL Database.
120: in SQL Server 2014 and Azure SQL Database.
130: in SQL Server 2016 and Azure SQL Database.

IMPORTANT
The default compatibility level is 130 for newly created databases.
Databases created before mid-June 2016 have the compatibility level in place at their creation time. Databases that migrated
from an earlier version of Azure SQL Database have a compatibility level of either 100 or 110. The compatibility level of the
master database for a server created prior to mid-June 2016 has the compatibility level in place when the server was created
and cannot be changed. A different compatibility level between the master database and a user database is not something
that impacts functionality or performance for user databases.

About compatibility level 130

First, if you want to know the current compatibility level of your database, execute the following Transact-SQL
statement.

SELECT compatibility_level
FROM sys.databases
WHERE name = '<YOUR DATABASE_NAME>’;

Before this change to level 130 happens for newly created databases, let’s review what this change is all about
through some basic query examples, and see how anyone can benefit from it.
Query processing in relational databases can be very complex and require a deep background in computer science
and mathematics to understand the inherent design choices and behaviors. In this document, the content has been
intentionally simplified to ensure that anyone with some minimum technical background can understand the
impact of the compatibility level change and determine how it can benefit applications.
Let’s have a quick look at what the compatibility level 130 brings at the table. You can find more details at ALTER
DATABASE Compatibility Level (Transact-SQL), but here is a short summary:
The Insert operation of an Insert-select statement can be multi-threaded or can have a parallel plan, while before
this operation was single-threaded.
Memory Optimized table and table variables queries can now have parallel plans, while before this operation
was also single-threaded.
Statistics for Memory Optimized table can now be sampled and are auto-updated. See What's New in Database
Engine: In-Memory OLTP for more details.
Batch mode v/s Row Mode changes with Column Store indexes
Sorts on a table with a Column Store index are now in batch mode.
Windowing aggregates now operate in batch mode such as T-SQL LAG/LEAD statements.
Queries on Column Store tables with Multiple distinct clauses operate in Batch mode.
Queries running under DOP=1 or with a serial plan also execute in Batch Mode.
Last, Cardinality Estimation improvements come with compatibility level 120, but for those of you running at a
lower Compatibility level (that is 100, or 110), the move to compatibility level 130 also brings these
improvements, and these can also benefit the query performance of your applications.

Practicing compatibility level 130

First let’s get some tables, indexes, and random data created to practice some of these new capabilities. The T-SQL
script examples can be executed under SQL Server 2016, or under Azure SQL Database. However, when creating an
Azure SQL database, make sure you choose at the minimum a P2 database because you need at least a couple of
cores to allow multi-threading and therefore benefit from these features.

-- Create a Premium P2 Database in Azure SQL Database

CREATE DATABASE MyTestDB

(EDITION=’Premium’, SERVICE_OBJECTIVE=’P2′);
GO

-- Create 2 tables with a column store index on

-- the second one (only available on Premium databases)

CREATE TABLE T_source

(Color varchar(10), c1 bigint, c2 bigint);

CREATE TABLE T_target

(c1 bigint, c2 bigint);

CREATE CLUSTERED COLUMNSTORE INDEX CCI ON T_target;

-- Insert few rows.

INSERT T_source VALUES

(‘Blue’, RAND() * 100000, RAND() * 100000),
(‘Yellow’, RAND() * 100000, RAND() * 100000),
(‘Red’, RAND() * 100000, RAND() * 100000),
(‘Green’, RAND() * 100000, RAND() * 100000),
(‘Black’, RAND() * 100000, RAND() * 100000);

GO 200

INSERT T_source SELECT * FROM T_source;

GO 10

Now, let’s have a look to some of the Query Processing features coming with compatibility level 130.

Parallel INSERT
Executing the following T-SQL statements executes the INSERT operation under compatibility level 120 and 130,
which respectively executes the INSERT operation in a single threaded model (120), and in a multi-threaded model
(130).

-- Parallel INSERT … SELECT … in heap or CCI

-- is available under 130 only

SET STATISTICS XML ON;

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 120;
GO

-- The INSERT part is in serial

INSERT t_target WITH (tablock)

SELECT C1, COUNT(C2) * 10 * RAND()
FROM T_source
GROUP BY C1
OPTION (RECOMPILE);

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 130
GO

-- The INSERT part is in parallel

INSERT t_target WITH (tablock)

SELECT C1, COUNT(C2) * 10 * RAND()
FROM T_source
GROUP BY C1
OPTION (RECOMPILE);

SET STATISTICS XML OFF;

By requesting the actual the query plan, looking at its graphical representation or its XML content, you can
determine which Cardinality Estimation function is at play. Looking at the plans side by side on figure 1, we can
clearly see that the Column Store INSERT execution goes from serial in 120 to parallel in 130. Also, note that the
change of the iterator icon in the 130 plan showing two parallel arrows, illustrating the fact that now the iterator
execution is indeed parallel. If you have large INSERT operations to complete, the parallel execution, linked to the
number of core you have at your disposal for the database, performs better; up to a 100 times faster depending on
your situation!
Figure 1: INSERT operation changes from serial to parallel with compatibility level 130.
SERIAL Batch Mode
Similarly, moving to compatibility level 130 when processing rows of data enables batch mode processing. First,
batch mode operations are only available when you have a column store index in place. Second, a batch typically
represents ~900 rows, and uses a code logic optimized for multicore CPU, higher memory throughput and directly
leverages the compressed data of the Column Store whenever possible. Under these conditions, SQL Server 2016
can process ~900 rows at once, instead of 1 row at the time, and as a consequence, the overall overhead cost of the
operation is now shared by the entire batch, reducing the overall cost by row. This shared number of operations
combined with the column store compression basically reduces the latency involved in a SELECT batch mode
operation. You can find more details about the column store and batch mode at Columnstore Indexes Guide.
-- Serial batch mode execution

SET STATISTICS XML ON;

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 120;
GO

-- The scan and aggregate are in row mode

SELECT C1, COUNT (C2)

FROM T_target
GROUP BY C1
OPTION (MAXDOP 1, RECOMPILE);
GO

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 130;
GO

-- The scan and aggregate are in batch mode,

-- and force MAXDOP to 1 to show that batch mode
-- also now works in serial mode.

SELECT C1, COUNT(C2)

FROM T_target
GROUP BY C1
OPTION (MAXDOP 1, RECOMPILE);
GO

SET STATISTICS XML OFF;

By observing the query plans side by side on figure 2, we can see that the processing mode has changed with the
compatibility level, and as a consequence, when executing the queries in both compatibility level altogether, we can
see that most of the processing time is spent in row mode (86%) compared to the batch mode (14%), where 2
batches have been processed. Increase the dataset, increase the benefit.
Figure 2: SELECT operation changes from serial to batch mode with compatibility level 130.

Batch mode on Sort Execution

Similar to the preceding example, but applied to a sort operation, the transition from row mode (compatibility level
120) to batch mode (compatibility level 130) improves the performance of the SORT operation for the same
reasons.

-- Batch mode on sort execution

SET STATISTICS XML ON;

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 120;
GO

-- The scan and aggregate are in row mode

SELECT C1, COUNT(C2)

FROM T_target
GROUP BY C1
ORDER BY C1
OPTION (MAXDOP 1, RECOMPILE);
GO

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 130;
GO

-- The scan and aggregate are in batch mode,

-- and force MAXDOP to 1 to show that batch mode
-- also now works in serial mode.

SELECT C1, COUNT(C2)

FROM T_target
GROUP BY C1
ORDER BY C1
OPTION (MAXDOP 1, RECOMPILE);
GO

SET STATISTICS XML OFF;

Visible side by side on figure 3, we can see that the sort operation in row mode represents 81% of the cost, while
the batch mode only represents 19% of the cost (respectively 81% and 56% on the sort itself).
Figure 3: SORT operation changes from row to batch mode with compatibility level 130.
Obviously, these samples only contain tens of thousands of rows, which are nothing when looking at the data
available in most SQL Servers these days. Project these against millions of rows instead, and this can translate in
several minutes of execution spared every day pending the nature of your workload.

Cardinality Estimation (CE) improvements

Introduced with SQL Server 2014, any database running at a compatibility level 120 or higher makes use of the
new Cardinality Estimation functionality. Essentially, cardinality estimation is the logic used to determine how SQL
server executes a query based on its estimated cost. The estimation is calculated using input from statistics
associated with objects involved in that query. Practically, at a high-level, Cardinality Estimation functions are row
count estimates along with information about the distribution of the values, distinct value counts, and duplicate
counts contained in the tables and objects referenced in the query. Getting these estimates wrong, can lead to
unnecessary disk I/O due to insufficient memory grants (that is TempDB spills), or to a selection of a serial plan
execution over a parallel plan execution, to name a few. Conclusion, incorrect estimates can lead to an overall
performance degradation of the query execution. On the other side, better estimates, more accurate estimates,
leads to better query executions!
As mentioned before, query optimizations and estimates are a complex matter, but if you want to learn more about
query plans and cardinality estimator, you can refer to the document at Optimizing Your Query Plans with the SQL
Server 2014 Cardinality Estimator for a deeper dive.

Which Cardinality Estimation do you currently use?

To determine under which Cardinality Estimation your queries are running, use the following query samples. This
first example runs under compatibility level 110, implying the use of the old Cardinality Estimation functions.
-- Old CE

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 110;
GO

SET STATISTICS XML ON;

SELECT [c1]
FROM [dbo].[T_target]
WHERE [c1] > 20000;
GO

SET STATISTICS XML OFF;

Once execution is complete, click the XML link, and look at the properties of the first iterator. Note the property
name called CardinalityEstimationModelVersion currently set on 70. It does not mean that the database
compatibility level is set to the SQL Server 7.0 version (it is set on 110 as visible in the preceding T-SQL
statements), but the value 70 represents the legacy Cardinality Estimation functionality available since SQL Server
7.0, which had no major revisions until SQL Server 2014 (which comes with a compatibility level of 120).
Figure 4: The CardinalityEstimationModelVersion is set to 70 when using a compatibility level of 110 or lower.

Alternatively, you can change the compatibility level to 130, and disable the use of the new Cardinality Estimation
function by using the LEGACY_CARDINALITY_ESTIMATION set to ON with ALTER DATABASE SCOPED
CONFIGURATION. This is the same as using 110 from a Cardinality Estimation function point of view, while using
the latest query processing compatibility level. Doing so, you can benefit from the new query processing features
coming with the latest compatibility level (that is batch mode), but still rely on the old Cardinality Estimation
functionality if necessary.

-- Old CE

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 130;
GO

ALTER DATABASE
SCOPED CONFIGURATION
SET LEGACY_CARDINALITY_ESTIMATION = ON;
GO

SET STATISTICS XML ON;

SELECT [c1]
FROM [dbo].[T_target]
WHERE [c1] > 20000;
GO

SET STATISTICS XML OFF;

Changing the compatibility level 120 or 130 enables the new Cardinality Estimation functionality. In such a case, the
default CardinalityEstimationModelVersion is set to 120 or 130.

-- New CE

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 130;
GO

ALTER DATABASE
SCOPED CONFIGURATION
SET LEGACY_CARDINALITY_ESTIMATION = OFF;
GO

SET STATISTICS XML ON;

SELECT [c1]
FROM [dbo].[T_target]
WHERE [c1] > 20000;
GO

SET STATISTICS XML OFF;

Figure 5: The CardinalityEstimationModelVersion is set to 130 when using a compatibility level of 130.

Witnessing the Cardinality Estimation differences

Now, let’s run a slightly more complex query involving an INNER JOIN with a WHERE clause with some predicates,
and let’s look at the row count estimate from the old Cardinality Estimation function first.
-- Old CE row estimate with INNER JOIN and WHERE clause

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 130;
GO

ALTER DATABASE
SCOPED CONFIGURATION
SET LEGACY_CARDINALITY_ESTIMATION = ON;
GO

SET STATISTICS XML ON;

SELECT T.[c2]
FROM
[dbo].[T_source] S
INNER JOIN [dbo].[T_target] T ON T.c1=S.c1
WHERE
S.[Color] = ‘Red’ AND
S.[c2] > 2000 AND
T.[c2] > 2000
OPTION (RECOMPILE);
GO

SET STATISTICS XML OFF;

Executing this query effectively returns 200,704 rows, while the row estimate with the old Cardinality Estimation
functionality claims 194,284 rows. Obviously, as said before, these row count results also depend how often you
ran the previous samples, which populates the sample tables over and over again at each run. Obviously, the
predicates in your query also have an influence on the actual estimation aside from the table shape, data content,
and how this data correlate with each other.
Figure 6: The row count estimate is 194,284 or 6,000 rows off from the 200,704 rows expected.

In the same way, let’s now execute the same query with the new Cardinality Estimation functionality.
-- New CE row estimate with INNER JOIN and WHERE clause

ALTER DATABASE MyTestDB

SET COMPATIBILITY_LEVEL = 130;
GO

ALTER DATABASE
SCOPED CONFIGURATION
SET LEGACY_CARDINALITY_ESTIMATION = OFF;
GO

SET STATISTICS XML ON;

SELECT T.[c2]
FROM
[dbo].[T_source] S
INNER JOIN [dbo].[T_target] T ON T.c1=S.c1
WHERE
S.[Color] = ‘Red’ AND
S.[c2] > 2000 AND
T.[c2] > 2000
OPTION (RECOMPILE);
GO

SET STATISTICS XML OFF;

We now see that the row estimate is 202,877, or much closer and higher than the old Cardinality Estimation.
Figure 7: The row count estimate is now 202,877, instead of 194,284.

In reality, the result set is 200,704 rows (but all of it depends how often you did run the queries of the previous
samples, but more importantly, because the T-SQL uses the RAND() statement, the actual values returned can vary
from one run to the next). Therefore, in this particular example, the new Cardinality Estimation does a better job at
estimating the number of rows because 202,877 is much closer to 200,704, than 194,284! Last, if you change the
WHERE clause predicates to equality (rather than “>” for instance), this could make the estimates between the old
and new Cardinality function even more different, depending on how many matches you can get.
Obviously, in this case, being ~6000 rows off from actual count does not represent a lot of data in some situations.
Now, transpose this to millions of rows across several tables and more complex queries, and at times the estimate
can be off by millions of rows, and therefore, the risk of picking-up the wrong execution plan, or requesting
insufficient memory grants leading to TempDB spills, and so more I/O, are much higher.
If you have the opportunity, practice this comparison with your most typical queries and datasets, and see for
yourself by how the old and new estimates are affected, while some could become more off from the reality, or
some others closer to the actual row counts returned in the result sets. All of it depends on the shape of your
queries, the Azure SQL database characteristics, the nature and the size of your datasets, and the statistics available
about them. If you just created your Azure SQL Database instance, the query optimizer has to build its knowledge
from scratch instead of reusing statistics made of the previous query runs. So, the estimates are very contextual and
almost specific to every server and application situation. It is an important aspect to keep in mind!

Some considerations to take into account

Although most workloads would benefit from the compatibility level 130, before adopting the compatibility level
for your production environment, you basically have 3 options:
1. You move to compatibility level 130, and see how things perform. In case you notice some regressions, you set
the compatibility level back to its original level, or keep 130, and only reverse the Cardinality Estimation back to
the legacy mode (As preciously explained, this alone could address the issue).
2. You thoroughly test your existing applications under similar production load, fine-tune, and validate the
performance before going to production. In case of issues, same as the preceding, you can always go back to the
original compatibility level, or reverse the Cardinality Estimation back to the legacy mode.
3. As a final option, and the most recent way to address these questions, is to use Query Store. That’s today’s
recommended option! To assist the analysis of your queries under compatibility level 120 or lower versus 130,
we cannot encourage you enough to use Query Store. Query Store is available with the latest version of Azure
SQL Database, and it’s designed to help you with query performance troubleshooting. Think of the Query Store
as a flight data recorder for your database collecting and presenting detailed historic information about all
queries. This greatly simplifies performance forensics by reducing the time to diagnose and resolve issues. You
can find more information at Query Store: A flight data recorder for your database.
At the high level, if you already have a set of databases running at compatibility level 120 or lower, and plan to
move some of them to 130, or because your workload automatically provision new databases that are set by
default to 130, please consider the followings:
Before changing to the new compatibility level in production, enable Query Store. You can refer to Change the
Database Compatibility Mode and Use the Query Store for more information.
Next, test all critical workloads using representative data and queries of a production-like environment, and
compare the performance experienced and as reported by Query Store. If you experience some regressions, you
can identify the regressed queries with the Query Store and use the plan forcing option from Query Store (aka
plan pinning). In such a case, you definitively stay with the compatibility level 130, and use the former query
plan as suggested by the Query Store.
If you want to use the new features and capabilities of Azure SQL Database (which is running SQL Server 2016),
but are sensitive to changes brought by the compatibility level 130, as a last resort, you could consider forcing
the compatibility level back to the level that suits your workload by using an ALTER DATABASE statement. But
first, be aware that the Query Store plan pinning option is your best option because not using 130 is basically
staying at the functionality level of an older SQL Server version.
If you have multitenant applications spanning multiple databases, it may be necessary to update the
provisioning logic of your databases to ensure a consistent compatibility level across all databases; old and
newly provisioned ones. Your application workload performance could be sensitive to the fact that some
databases are running at different compatibility levels, and therefore, compatibility level consistency across any
database could be required in order to provide the same experience to your customers all across the board. It is
not a mandate; it really depends on how your application is affected by the compatibility level.
Last, regarding the Cardinality Estimation, it is recommended to test your production workload under the new
conditions to determine if your application benefits from the Cardinality Estimation improvements.

Conclusion
Using Azure SQL Database to benefit from all SQL Server 2016 enhancements can clearly improve your query
executions. Of course, like any new feature, a proper evaluation must be done to determine the exact conditions
under which your database workload operates the best. Experience shows that most workload are expected to at
least run transparently under compatibility level 130, while using new query processing functions, and new
Cardinality Estimation. That said, realistically, there are always some exceptions and doing proper due diligence is
an important assessment to determine how much you can benefit from these enhancements. And again, the Query
Store can be of a great help in doing this work!

References
What’s New in Database Engine
Blog: Query Store: A flight data recorder for your database, by Borko Novakovic, June 8 2016
ALTER DATABASE Compatibility Level (Transact-SQL)
ALTER DATABASE SCOPED CONFIGURATION
Compatibility Level 130 for Azure SQL Database
Optimizing Your Query Plans with the SQL Server 2014 Cardinality Estimator
Columnstore Indexes Guide
Blog: Improved Query Performance with Compatibility Level 130 in Azure SQL Database, by Alain Lissoir, May 6
2016
Create and manage an elastic pool with the Azure
portal
4/19/2017 • 11 min to read • Edit Online

This topic shows you how to create and manage scalable elastic pools with the Azure portal. You can also create
and manage an Azure elastic pool with PowerShell, the REST API, or C#. You can also create and move databases
into and out of elastic pools using Transact-SQL.

Create an elastic pool

There are two ways you can create an elastic pool. You can do it from scratch if you know the pool setup you
want, or start with a recommendation from the service. SQL Database has built-in intelligence that recommends
an elastic pool setup if it's more cost-efficient for you based on the past usage telemetry for your databases.
You can create multiple pools on a server, but you can't add databases from different servers into the same pool.

NOTE
Elastic pools are generally available (GA) in all Azure regions except West India where it is currently in preview. GA of
elastic pools in this region will occur as soon as possible.

Step 1: Create an elastic pool

Creating an elastic pool from an existing server blade in the portal is the easiest way to move existing databases
into an elastic pool.

NOTE
You can also create an elastic pool by searching SQL elastic pool in the Marketplace or clicking +Add in the SQL
elastic pools browse blade. You are able to specify a new or existing server through this pool provisioning workflow.

1. In the Azure portal, click More services > SQL servers, and then click the server that contains the databases
you want to add to an elastic pool.
2. Click New pool.

-OR-
You may see a message saying there are recommended elastic pools for the server. Click the message to
see the recommended pools based on historical database usage telemetry, and then click the tier to see
more details and customize the pool. See Understand pool recommendations later in this topic for how
the recommendation is made.
3. The elastic pool blade appears, which is where you specify the settings for your pool. If you clicked New
pool in the previous step, the pricing tier is Standard by default and no databases is selected. You can
create an empty pool, or specify a set of existing databases from that server to move into the pool. If you
are creating a recommended pool, the recommended pricing tier, performance settings, and list of
databases are prepopulated, but you can still change them.

4. Specify a name for the elastic pool, or leave it as the default.

Step 2: Choose a pricing tier
The pool's pricing tier determines the features available to the elastics in the pool, and the maximum number of
eDTUs (eDTU MAX), and storage (GBs) available to each database. For details, see Service Tiers.
To change the pricing tier for the pool, click Pricing tier, click the pricing tier you want, and then click Select.

IMPORTANT
After you choose the pricing tier and commit your changes by clicking OK in the last step, you won't be able to change
the pricing tier of the pool. To change the pricing tier for an existing elastic pool, create an elastic pool in the desired
pricing tier and migrate the databases to this new pool.

Step 3: Configure the pool

After setting the pricing tier, click Configure pool where you add databases, set pool eDTUs and storage (pool
GBs), and where you set the min and max eDTUs for the elastics in the pool.
1. Click Configure pool
2. Select the databases you want to add to the pool. This step is optional while creating the pool. Databases
can be added after the pool has been created. To add databases, click Add database, click the databases
that you want to add, and then click the Select button.
If the databases you're working with have enough historical usage telemetry, the Estimated eDTU and
GB usage graph and the Actual eDTU usage bar chart update to help you make configuration decisions.
Also, the service may give you a recommendation message to help you right-size the pool. See Dynamic
Recommendations.
3. Use the controls on the Configure pool page to explore settings and configure your pool. See Elastic
pools limits for more detail about limits for each service tier, and see Price and performance
considerations for elastic pools for detailed guidance on right-sizing an elastic pool. For more information
about pool settings, see Elastic pool properties.
4. Click Select in the Configure Pool blade after changing settings.
5. Click OK to create the pool.

Understand elastic pool recommendations

The SQL Database service evaluates usage history and recommends one or more pools when it is more cost-
effective than using single databases. Each recommendation is configured with a unique subset of the server's
databases that best fit the pool.

The pool recommendation comprises:

A pricing tier for the pool (Basic, Standard, Premium, or Premium RS)
Appropriate POOL eDTUs (also called Max eDTUs per pool)
The eDTU MAX and eDTU Min per database
The list of recommended databases for the pool

IMPORTANT
The service takes the last 30 days of telemetry into account when recommending pools. For a database to be considered
as a candidate for an elastic pool, it must exist for at least 7 days. Databases that are already in an elastic pool are not
considered as candidates for elastic pool recommendations.

The service evaluates resource needs and cost effectiveness of moving the single databases in each service tier
into pools of the same tier. For example, all Standard databases on a server are assessed for their fit into a
Standard Elastic Pool. This means the service does not make cross-tier recommendations such as moving a
Standard database into a Premium pool.
After adding databases to the pool, recommendations are dynamically generated based on the historical usage
of the databases you have selected. These recommendations are shown in the eDTU and GB usage chart and in a
recommendation banner at the top of the Configure pool blade. These recommendations are intended to assist
you in creating an elastic pool optimized for your specific databases.

Manage and monitor an elastic pool

You can use the Azure portal to monitor and manage an elastic pool and the databases in the pool. From the
portal, you can monitor the utilization of an elastic pool and the databases within that pool. You can also make a
set of changes to your elastic pool and submit all changes at the same time. These changes include adding or
removing databases, changing your elastic pool settings, or changing your database settings.
The following graphic shows an example elastic pool. The view includes:
Charts for monitoring resource usage of both the elastic pool and the databases contained in the pool.
The Configure pool button to make changes to the elastic pool.
The Create database button that creates a database and adds it to the current elastic pool.
Elastic jobs that help you manage large numbers of databases by running Transact SQL scripts against all
databases in a list.
You can go to a particular pool to see its resource utilization. By default, the pool is configured to show storage
and eDTU usage for the last hour. The chart can be configured to show different metrics over various time
windows.
1. Select an elastic pool to work with.
2. Under Elastic Pool Monitoring is a chart labeled Resource utilization. Click the chart.
The Metric blade opens, showing a detailed view of the specified metrics over the specified time window.
To customize the chart display
You can edit the chart and the metric blade to display other metrics such as CPU percentage, data IO percentage,
and log IO percentage used.
1. On the metric blade, click Edit.

2. In the Edit Chart blade, select a time range (past hour, today, or past week), or click custom to select any
date range in the last two weeks. Select the chart type (bar or line), then select the resources to monitor.

NOTE
Only metrics with the same unit of measure can be displayed in the chart at the same time. For example, if you
select "eDTU percentage" then you can only select other metrics with percentage as the unit of measure.
3. Then click OK.

Manage and monitor databases in an elastic pool

Individual databases can also be monitored for potential trouble.
1. Under Elastic Database Monitoring, there is a chart that displays metrics for five databases. By default,
the chart displays the top 5 databases in the pool by average eDTU usage in the past hour. Click the chart.
2. The Database Resource Utilization blade appears. This provides a detailed view of the database usage
in the pool. Using the grid in the lower part of the blade, you can select any databases in the pool to
display its usage in the chart (up to 5 databases). You can also customize the metrics and time window
displayed in the chart by clicking Edit chart.
To customize the view
1. In the Database resource utilization blade, click Edit chart.

2. In the Edit chart blade, select a time range (past hour or past 24 hours), or click custom to select a
different day in the past 2 weeks to display.
3. Click the Compare databases by dropdown to select a different metric to use when comparing
databases.

To select databases to monitor

In the database list in the Database Resource Utilization blade, you can find particular databases by looking
through the pages in the list or by typing in the name of a database. Use the checkbox to select the database.
Add an alert to an elastic pool resource
You can add rules to an elastic pool that send email to people or alert strings to URL endpoints when the elastic
pool hits a utilization threshold that you set up.

IMPORTANT
Resource utilization monitoring for elastic pools has a lag of at least 20 minutes. Setting alerts of less than 30 minutes for
elastic pools is not currently supported. Any alerts set for Elastic Pools with a period (parameter called “-WindowSize” in
PowerShell API) of less than 30 minutes may not be triggered. Make sure that any alerts you define for Elastic Pools use a
period (WindowSize) of 30 minutes or more.

To add an alert to any resource:

1. Click the Resource utilization chart to open the Metric blade, click Add alert, and then fill out the
information in the Add an alert rule blade (Resource is automatically set up to be the pool you're working
with).
2. Type a Name and Description that identifies the alert to you and the recipients.
3. Choose a Metric that you want to alert from the list.
The chart dynamically shows resource utilization for that metric to help you choose a threshold.
4. Choose a Condition (greater than, less than, etc.) and a Threshold.
5. Click OK.

Move a database into an elastic pool

You can add or remove databases from an existing pool. The databases can be in other pools. However, you can
only add databases that are on the same logical server.
1. In the blade for the pool, under Elastic databases click Configure pool.

2. In the Configure pool blade, click Add to pool.

3. In the Add databases blade, select the database or databases to add to the pool. Then click Select.

The Configure pool blade now lists the database you selected to be added, with its status set to
Pending.

4. In the Configure pool blade, click Save.

Move a database out of an elastic pool

1. In the Configure pool blade, select the database or databases to remove.

2. Click Remove from pool.

The Configure pool blade now lists the database you selected to be removed with its status set to
Pending.
3. In the Configure pool blade, click Save.

Change performance settings of an elastic pool

As you monitor the resource utilization of an elastic pool, you may discover that some adjustments are needed.
Maybe the pool needs a change in the performance or storage limits. Possibly you want to change the database
settings in the pool. You can change the setup of the pool at any time to get the best balance of performance and
cost. See When should an elastic pool be used? for more information.
To change the eDTUs or storage limits per pool, and eDTUs per database:
1. Open the Configure pool blade.
Under elastic pool settings, use either slider to change the pool settings.
2. When the setting changes, the display shows the estimated monthly cost of the change.

Latency of elastic pool operations

Next steps
To understand what an elastic pool is, see SQL Database elastic pool.
For guidance on using elastic pools, see Price and performance considerations for elastic pools.
To use elastic jobs to run Transact-SQL scripts against any number of databases in the pool, see Elastic jobs
overview.
To query across any number of databases in the pool, see Elastic query overview.
For transactions any number of databases in the pool, see Elastic transactions.
Create and manage an elastic pool with PowerShell
4/14/2017 • 11 min to read • Edit Online

This topic shows you how to create and manage scalable elastic pools with PowerShell. You can also create and
manage an Azure elastic pool the Azure portal, the REST API, or C#. You can also create and move databases into
and out of elastic pools using Transact-SQL.

Start your PowerShell session

First, you should have the latest Azure PowerShell installed and running. For detailed information, see How to
install and configure Azure PowerShell.

NOTE
Many new features of SQL Database are only supported when you are using the Azure Resource Manager deployment
model, so examples use the Azure SQL Database PowerShell cmdlets for Resource Manager. The service management
(classic) deployment model Azure SQL Database Service Management cmdlets are supported for backward compatibility,
but we recommend you use the Resource Manager cmdlets.

Run the Add-AzureRmAccount cmdlet, and you will be presented with a sign-in screen to enter your credentials.
Use the same credentials that you use to sign in to the Azure portal.

Add-AzureRmAccount

If you have multiple subscriptions, use the Set-AzureRmContext cmdlet to select which subscription your
PowerShell session should use. To see what subscription the current PowerShell session is using, run Get-
AzureRmContext. To see all your subscriptions, run Get-AzureRmSubscription.

Set-AzureRmContext -SubscriptionId '4cac86b0-1e56-bbbb-aaaa-000000000000'

Create an elastic pool

The New-AzureRmSqlElasticPool cmdlet creates an elastic pool. The values for eDTU per pool, min, and max DTUs
are constrained by the service tier value (Basic, Standard, Premium, or Premium RS). See eDTU and storage limits
for elastic pools and pooled databases.

New-AzureRmSqlElasticPool -ResourceGroupName "resourcegroup1" -ServerName "server1" -ElasticPoolName "elasticpool1" -Edition

"Standard" -Dtu 400 -DatabaseDtuMin 10 -DatabaseDtuMax 100

Create a pooled database in an elastic pool

Use the New-AzureRmSqlDatabase cmdlet and set the ElasticPoolName parameter to the target pool. To move
an existing database into an elastic pool, see Move a database into an elastic pool.

New-AzureRmSqlDatabase -ResourceGroupName "resourcegroup1" -ServerName "server1" -DatabaseName "database1" -ElasticPoolName

"elasticpool1"
Complete script
This script creates an Azure resource group and a server. When prompted, supply an administrator username and
password for the new server (not your Azure credentials).

$subscriptionId = '<your Azure subscription id>'

$resourceGroupName = '<resource group name>'
$location = '<datacenter location>'
$serverName = '<server name>'
$poolName = '<pool name>'
$databaseName = '<database name>'

New-AzureRmResourceGroup -Name $resourceGroupName -Location $location

New-AzureRmSqlServer -ResourceGroupName $resourceGroupName -ServerName $serverName -Location $location -ServerVersion "12.0"
New-AzureRmSqlServerFirewallRule -ResourceGroupName $resourceGroupName -ServerName $serverName -FirewallRuleName "rule1" -
StartIpAddress "192.168.0.198" -EndIpAddress "192.168.0.199"

New-AzureRmSqlElasticPool -ResourceGroupName $resourceGroupName -ServerName $serverName -ElasticPoolName $poolName -Edition

"Standard" -Dtu 400 -DatabaseDtuMin 10 -DatabaseDtuMax 100

New-AzureRmSqlDatabase -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName -

ElasticPoolName $poolName -MaxSizeBytes 10GB

Create an elastic pool and add multiple pooled databases

Creation of many databases in an elastic pool can take time when done using the portal or PowerShell cmdlets
that create only a single database at a time. To automate creation into an elastic pool, see
CreateOrUpdateElasticPoolAndPopulate .

Move a database into an elastic pool

You can move a database into or out of an elastic pool with the Set-AzureRmSqlDatabase.

Set-AzureRmSqlDatabase -ResourceGroupName "resourcegroup1" -ServerName "server1" -DatabaseName "database1" -ElasticPoolName

"elasticpool1"

Change performance settings of an elastic pool

When performance suffers, you can change the settings of the pool to accommodate growth. Use the Set-
AzureRmSqlElasticPool cmdlet. Set the -Dtu parameter to the eDTUs per pool. See eDTU and storage limits for
possible values.

Set-AzureRmSqlElasticPool -ResourceGroupName “resourcegroup1” -ServerName “server1” -ElasticPoolName “elasticpool1” -Dtu 1200 -

DatabaseDtuMax 100 -DatabaseDtuMin 50

Get the status of pool operations

Creating an elastic pool can take time. To track the status of pool operations including creation and updates, use
the Get-AzureRmSqlElasticPoolActivity cmdlet.

Get-AzureRmSqlElasticPoolActivity -ResourceGroupName “resourcegroup1” -ServerName “server1” -ElasticPoolName “elasticpool1”

Get the status of moving a database into and out of an elastic pool
Moving a database can take time. Track a move status using the Get-AzureRmSqlDatabaseActivity cmdlet.

Get-AzureRmSqlDatabaseActivity -ResourceGroupName "resourcegroup1" -ServerName "server1" -DatabaseName "database1" -

ElasticPoolName "elasticpool1"

Get resource usage data for an elastic pool

Metrics that can be retrieved as a percentage of the resource pool limit:

METRIC NAME DESCRIPTION

cpu_percent Average compute utilization in percentage of the limit of the

pool.

physical_data_read_percent Average I/O utilization in percentage based on the limit of the

pool.

log_write_percent Average write resource utilization in percentage of the limit of

the pool.

DTU_consumption_percent Average eDTU utilization in percentage of eDTU limit for the

pool

storage_percent Average storage utilization in percentage of the storage limit

of the pool.

workers_percent Maximum concurrent workers (requests) in percentage based

on the limit of the pool.

sessions_percent Maximum concurrent sessions in percentage based on the

limit of the pool.

eDTU_limit Current max elastic pool DTU setting for this elastic pool
during this interval.

storage_limit Current max elastic pool storage limit setting for this elastic
pool in megabytes during this interval.

eDTU_used Average eDTUs used by the pool in this interval.

storage_used Average storage used by the pool in this interval in bytes

Metrics granularity/retention periods:

Data is returned at 5-minute granularity.
Data retention is 35 days.
This cmdlet and API limits the number of rows that can be retrieved in one call to 1000 rows (about 3 days of data
at 5-minute granularity). But this command can be called multiple times with different start/end time intervals to
retrieve more data
To retrieve the metrics:
$metrics = (Get-AzureRmMetric -ResourceId
/subscriptions/<subscriptionId>/resourceGroups/FabrikamData01/providers/Microsoft.Sql/servers/fabrikamsqldb02/elasticPools/franchisepool
-TimeGrain ([TimeSpan]::FromMinutes(5)) -StartTime "4/18/2015" -EndTime "4/21/2015")

Get resource usage data for a database in an elastic pool

These APIs are the same as the APIs used for monitoring the resource utilization of a single database, except for
the following semantic difference: metrics retrieved are expressed as a percentage of the per database max eDTUs
(or equivalent cap for the underlying metric like CPU or IO) set for that pool. For example, 50% utilization of any
of these metrics indicates that the specific resource consumption is at 50% of the per database cap limit for that
resource in the parent pool.
To retrieve the metrics:

$metrics = (Get-AzureRmMetric -ResourceId

/subscriptions/<subscriptionId>/resourceGroups/FabrikamData01/providers/Microsoft.Sql/servers/fabrikamsqldb02/databases/myDB -
TimeGrain ([TimeSpan]::FromMinutes(5)) -StartTime "4/18/2015" -EndTime "4/21/2015")

Add an alert to an elastic pool resource

You can add alert rules to an elastic pool to send email notifications or alert strings to URL endpoints when the
elastic pool hits a utilization threshold that you set up. Use the Add-AzureRmMetricAlertRule cmdlet.

IMPORTANT
Resource utilization monitoring for elastic pools has a lag of at least 20 minutes. Setting alerts of less than 30 minutes for
elastic pools is not currently supported. Any alerts set for elastic pools with a period (parameter called “-WindowSize” in
PowerShell API) of less than 30 minutes may not be triggered. Make sure that any alerts you define for elastic pools use a
period (WindowSize) of 30 minutes or more.

This example adds an alert for getting notified when an elastic pool’s eDTU consumption goes above certain
threshold.

# Set up your resource ID configurations

$subscriptionId = '<Azure subscription id>' # Azure subscription ID
$location = '<location' # Azure region
$resourceGroupName = '<resource group name>' # Resource Group
$serverName = '<server name>' # server name
$poolName = '<elastic pool name>' # pool name

#$Target Resource ID
$ResourceID = '/subscriptions/' + $subscriptionId + '/resourceGroups/' +$resourceGroupName + '/providers/Microsoft.Sql/servers/' +
$serverName + '/elasticpools/' + $poolName

# Create an email action

$actionEmail = New-AzureRmAlertRuleEmail -SendToServiceOwners -CustomEmail JohnDoe@contoso.com

# create a unique rule name

$alertName = $poolName + "- DTU consumption rule"

# Create an alert rule for DTU_consumption_percent

Add-AzureRMMetricAlertRule -Name $alertName -Location $location -ResourceGroup $resourceGroupName -TargetResourceId $ResourceID -
MetricName "DTU_consumption_percent" -Operator GreaterThan -Threshold 80 -TimeAggregationOperator Average -WindowSize 00:60:00 -
Actions $actionEmail
Add alerts to all databases in an elastic pool
You can add alert rules to all database in an elastic pool to send email notifications or alert strings to URL
endpoints when a resource hits a utilization threshold set up by the alert.

IMPORTANT
Resource utilization monitoring for elastic pools has a lag of at least 20 minutes. Setting alerts of less than 30 minutes for
elastic pools is not currently supported. Any alerts set for elastic pools with a period (parameter called “-WindowSize” in
PowerShell API) of less than 30 minutes may not be triggered. Make sure that any alerts you define for elastic pools use a
period (WindowSize) of 30 minutes or more.

This example adds an alert to each of the databases in an elastic pool for getting notified when that database’s
DTU consumption goes above certain threshold.

# Set up your resource ID configurations

# Get the list of databases in this pool.

$dbList = Get-AzureRmSqlElasticPoolDatabase -ResourceGroupName $resourceGroupName -ServerName $serverName -ElasticPoolName
$poolName

# Create an email action

$actionEmail = New-AzureRmAlertRuleEmail -SendToServiceOwners -CustomEmail JohnDoe@contoso.com

# Get resource usage metrics for a database in an elastic pool for the specified time interval.
foreach ($db in $dbList)
{
$dbResourceId = '/subscriptions/' + $subscriptionId + '/resourceGroups/' + $resourceGroupName + '/providers/Microsoft.Sql/servers/' +
$serverName + '/databases/' + $db.DatabaseName

# create a unique rule name

$alertName = $db.DatabaseName + "- DTU consumption rule"

# Create an alert rule for DTU_consumption_percent

Add-AzureRMMetricAlertRule -Name $alertName -Location $location -ResourceGroup $resourceGroupName -TargetResourceId
$dbResourceId -MetricName "dtu_consumption_percent" -Operator GreaterThan -Threshold 80 -TimeAggregationOperator Average -
WindowSize 00:60:00 -Actions $actionEmail

# drop the alert rule

#Remove-AzureRmAlertRule -ResourceGroup $resourceGroupName -Name $alertName
}

Collect and monitor resource usage data across multiple pools in a

subscription
When you have many databases in a subscription, it is cumbersome to monitor each elastic pool separately.
Instead, SQL database PowerShell cmdlets and T-SQL queries can be combined to collect resource usage data
from multiple pools and their databases for monitoring and analysis of resource usage. A sample implementation
of such a set of powershell scripts can be found in the GitHub SQL Server samples repository along with
documentation on what it does and how to use it.
To use this sample implementation, follow these steps.
1. Download the scripts and documentation:
2. Modify the scripts for your environment. Specify one or more servers on which elastic pools are hosted.
3. Specify a telemetry database where the collected metrics are to be stored.
4. Customize the script to specify the duration of the scripts' execution.
At a high level, the scripts do the following:
Enumerates all servers in a given Azure subscription (or a specified list of servers).
Runs a background job for each server. The job runs in a loop at regular intervals and collects telemetry data
for all the pools in the server. It then loads the collected data into the specified telemetry database.
Enumerates a list of databases in each pool to collect the database resource usage data. It then loads the
collected data into the telemetry database.
The collected metrics in the telemetry database can be analyzed to monitor the health of elastic pools and the
databases in it. The script also installs a pre-defined Table-Value function (TVF) in the telemetry database to help
aggregate the metrics for a specified time window. For example, results of the TVF can be used to show “top N
elastic pools with the maximum eDTU utilization in a given time window.” Optionally, use analytic tools like Excel
or Power BI to query and analyze the collected data.
Example: retrieve resource consumption metrics for an elastic pool and its databases
This example retrieves the consumption metrics for a given elastic pool and all its databases. Collected data is
formatted and written to a .csv formatted file. The file can be browsed with Excel.
$subscriptionId = '<Azure subscription id>' # Azure subscription ID
$resourceGroupName = '<resource group name>' # Resource Group
$serverName = <server name> # server name
$poolName = <elastic pool name> # pool name

# Login to Azure account and select the subscription.

# Get resource usage metrics for an elastic pool for the specified time interval.
$startTime = '4/27/2016 00:00:00' # start time in UTC
$endTime = '4/27/2016 01:00:00' # end time in UTC

# Construct the pool resource ID and retrive pool metrics at 5-minute granularity.
$poolResourceId = '/subscriptions/' + $subscriptionId + '/resourceGroups/' + $resourceGroupName + '/providers/Microsoft.Sql/servers/' +
$serverName + '/elasticPools/' + $poolName
$poolMetrics = (Get-AzureRmMetric -ResourceId $poolResourceId -TimeGrain ([TimeSpan]::FromMinutes(5)) -StartTime $startTime -EndTime
$endTime)

# Get the list of databases in this pool.

$dbList = Get-AzureRmSqlElasticPoolDatabase -ResourceGroupName $resourceGroupName -ServerName $serverName -ElasticPoolName
$poolName

# Get resource usage metrics for a database in an elastic pool for the specified time interval.
$dbMetrics = @()
foreach ($db in $dbList)
{
$dbResourceId = '/subscriptions/' + $subscriptionId + '/resourceGroups/' + $resourceGroupName + '/providers/Microsoft.Sql/servers/' +
$serverName + '/databases/' + $db.DatabaseName
$dbMetrics = $dbMetrics + (Get-AzureRmMetric -ResourceId $dbResourceId -TimeGrain ([TimeSpan]::FromMinutes(5)) -StartTime $startTime
-EndTime $endTime)
}

#Optionally you can format the metrics and output as .csv file using the following script block.
$command = {
param($metricList, $outputFile)

# Format metrics into a table.

$table = @()
foreach($metric in $metricList) {
foreach($metricValue in $metric.MetricValues) {
$sx = New-Object PSObject -Property @{
Timestamp = $metricValue.Timestamp.ToString()
MetricName = $metric.Name;
Average = $metricValue.Average;
ResourceID = $metric.ResourceId
}
$table = $table += $sx
}
}

# Output the metrics into a .csv file.

write-output $table | Export-csv -Path $outputFile -Append -NoTypeInformation
}

# Format and output pool metrics

Invoke-Command -ScriptBlock $command -ArgumentList $poolMetrics,c:\temp\poolmetrics.csv

# Format and output database metrics

Invoke-Command -ScriptBlock $command -ArgumentList $dbMetrics,c:\temp\dbmetrics.csv

Latency of elastic pool operations

Changing the min eDTUs per database or max eDTUs per database typically completes in 5 minutes or less.
Changing the eDTUs per pool depends on the total amount of space used by all databases in the pool.
Changes average 90 minutes or less per 100 GB. For example, if the total space used by all databases in the
pool is 200 GB, then the expected latency for changing the pool eDTU per pool is 3 hours or less.
For reference documentation about these PowerShell cmdlets, see:
Get-AzureRMSqlServerUpgrade
Start-AzureRMSqlServerUpgrade
Stop-AzureRMSqlServerUpgrade
The Stop- cmdlet means cancel, not pause. There is no way to resume an upgrade, other than starting again from
the beginning. The Stop- cmdlet cleans up and releases all appropriate resources.

Next steps
Create elastic jobs Elastic jobs let you run T-SQL scripts against any number of databases in the pool.
See Scaling out with Azure SQL Database: use elastic tools to scale out, move data, query, or create
transactions.
Monitor and manage an elastic pool with Transact-
SQL
3/10/2017 • 2 min to read • Edit Online

This topic shows you how to manage scalable elastic pools with Transact-SQL. You can also create and manage an
Azure elastic pool the Azure portal, PowerShell, the REST API, or C#. You can also create and move databases into
and out of elastic pools using Transact-SQL.
Use the Create Database (Azure SQL Database) and Alter Database(Azure SQL Database) commands to create and
move databases into and out of elastic pools. The elastic pool must exist before you can use these commands.
These commands affect only databases. Creation of new pools and the setting of pool properties (such as min and
max eDTUs) cannot be changed with T-SQL commands.

Create a pooled database in an elastic pool

Use the CREATE DATABASE command with the SERVICE_OBJECTIVE option.

CREATE DATABASE db1 ( SERVICE_OBJECTIVE = ELASTIC_POOL (name = [S3M100] ));

-- Create a database named db1 in an elastic named S3M100.

All databases in an elastic pool inherit the service tier of the elastic pool (Basic, Standard, Premium).

Move a database between elastic pools

Use the ALTER DATABASE command with the MODIFY and set SERVICE_OBJECTIVE option as ELASTIC_POOL. Set
the name to the name of the target pool.

ALTER DATABASE db1 MODIFY ( SERVICE_OBJECTIVE = ELASTIC_POOL (name = [PM125] ));

-- Move the database named db1 to an elastic named P1M125

Move a database into an elastic pool

Use the ALTER DATABASE command with the MODIFY and set SERVICE_OBJECTIVE option as ELASTIC_POOL. Set
the name to the name of the target pool.

ALTER DATABASE db1 MODIFY ( SERVICE_OBJECTIVE = ELASTIC_POOL (name = [S3100] ));

-- Move the database named db1 to an elastic named S3100.

Move a database out of an elastic pool

Use the ALTER DATABASE command and set the SERVICE_OBJECTIVE to one of the performance levels (such as
S0 or S1).

ALTER DATABASE db1 MODIFY ( SERVICE_OBJECTIVE = 'S1');

-- Changes the database into a stand-alone database with the service objective S1.
List databases in an elastic pool
Use the sys.database_service _objectives view to list all the databases in an elastic pool. Log in to the master
database to query the view.

SELECT d.name, slo.*

FROM sys.databases d
JOIN sys.database_service_objectives slo
ON d.database_id = slo.database_id
WHERE elastic_pool_name = 'MyElasticPool';

Get resource usage data for an elastic pool

Use the sys.elastic_pool _resource _stats view to examine the resource usage statistics of an elastic pool on a
logical server. Log in to the master database to query the view.

SELECT * FROM sys.elastic_pool_resource_stats

WHERE elastic_pool_name = 'MyElasticPool'
ORDER BY end_time DESC;

Get resource usage for a pooled database

Use the sys.dm_ db_ resource_stats view or sys.resource _stats view to examine the resource usage statistics of a
database in an elastic pool. This process is similar to querying resource usage for a single database.

Next steps
After creating an elastic pool, you can manage elastic databases in the pool by creating elastic jobs. Elastic jobs
facilitate running T-SQL scripts against any number of databases in the pool. For more information, see Elastic
database jobs overview.
See Scaling out with Azure SQL Database: use elastic database tools to scale out, move data, query, or create
transactions.
Create and manage an elastic pool with C#
4/19/2017 • 8 min to read • Edit Online

This topic shows you how to create and manage scalable elastic pools with C#. You can also create and manage
an Azure elastic pool with the Azure portal, PowerShell, or the REST API. You can also create and move databases
into and out of elastic pools using Transact-SQL.

NOTE
Many new features of SQL Database are only supported when you are using the Azure Resource Manager deployment
model, so you should always use the latest Azure SQL Database Management Library for .NET (docs | NuGet
Package). The older classic deployment model-based libraries are supported for backward compatibility only, so we
recommend you use the newer Resource Manager based libraries.

This topic describes how to use C# to create and manage an Azure SQL elastic pool with the Azure SQL Database
Library for .NET. To create a single SQL database, see Use C# to create a SQL database with the SQL Database
Library for .NET.
The Azure SQL Database Library for .NET provides an Azure Resource Manager-based API that wraps the
Resource Manager-based SQL Database REST API.

NOTE
Many new features of SQL Database are only supported when you are using the Azure Resource Manager deployment
model, so you should always use the latest Azure SQL Database Management Library for .NET (docs | NuGet
Package). The older classic deployment model libraries are supported for backward compatibility only, so we recommend
you use the newer Resource Manager based libraries.

Prerequisites
To complete the steps in this article, you need the following:
An Azure subscription. If you need an Azure subscription simply click FREE ACCOUNT at the top of this page,
and then come back to finish this article.
Visual Studio. For a free copy of Visual Studio, see the Visual Studio Downloads page.

Create a console app and install the required libraries

1. Start Visual Studio.
2. Click File > New > Project.
3. Create a C# Console Application and name it: SqlElasticPoolConsoleApp

Create a SQL database

To create a SQL database with C#, load the required management libraries (using the package manager console):
1. Click Tools > NuGet Package Manager > Package Manager Console.
2. Type Install-Package Microsoft.Azure.Management.Sql -Pre to install the Microsoft Azure SQL Management Library.
3. Type Install-Package Microsoft.Azure.Management.ResourceManager -Pre to install the Microsoft Azure Resource Manager
Library.
4. Type Install-Package Microsoft.Azure.Common.Authentication -Pre to install the Microsoft Azure Common Authentication
Library.

NOTE
The examples in this article use a synchronous form of each API request and block until completion of the REST call on the
underlying service. There are async methods available.

Create a SQL elastic pool - C# example

The following sample creates a resource group, server, firewall rule, elastic pool, and then creates a SQL database
in the pool. See, Create a service principal to access resources to get the
_subscriptionId, _tenantId, _applicationId, and _applicationSecret variables.

Replace the contents of Program.cs with the following, and update the {variables} with your app values (do not
include the {} ).

using Microsoft.Azure;
using Microsoft.Azure.Management.ResourceManager;
using Microsoft.Azure.Management.ResourceManager.Models;
using Microsoft.Azure.Management.Sql;
using Microsoft.Azure.Management.Sql.Models;
using Microsoft.IdentityModel.Clients.ActiveDirectory;
using System;

namespace SqlElasticPoolConsoleApp
{
class Program
{

// For details about these four (4) values, see

// https://azure.microsoft.com/documentation/articles/resource-group-authenticate-service-principal/
static string _subscriptionId = "{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}";
static string _tenantId = "{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}";
static string _applicationId = "{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}";
static string _applicationSecret = "{your-password}";

// Create management clients for the Azure resources your app needs to work with.
// This app works with Resource Groups, and Azure SQL Database.
static ResourceManagementClient _resourceMgmtClient;
static SqlManagementClient _sqlMgmtClient;

// Authentication token
static AuthenticationResult _token;

// Azure resource variables

static string _resourceGroupName = "{resource-group-name}";
static string _resourceGrouplocation = "{Azure-region}";

static string _serverlocation = _resourceGrouplocation;

static string _serverName = "{server-name}";
static string _serverAdmin = "{server-admin-login}";
static string _serverAdminPassword = "{server-admin-password}";

static string _firewallRuleName = "{firewall-rule-name}";

static string _startIpAddress = "{0.0.0.0}";
static string _endIpAddress = "{255.255.255.255}";

static string _poolName = "{pool-name}";

static string _poolEdition = "{Standard}";
static int _poolDtus = {100};
static int _databaseMinDtus = {10};
static int _databaseMinDtus = {10};
static int _databaseMaxDtus = {100};

static string _databaseName = "{elasticdbfromcsarticle}";

static void Main(string[] args)

{
// Authenticate:
_token = GetToken(_tenantId, _applicationId, _applicationSecret);
Console.WriteLine("Token acquired. Expires on:" + _token.ExpiresOn);

// Instantiate management clients:

_resourceMgmtClient = new ResourceManagementClient(new Microsoft.Rest.TokenCredentials(_token.AccessToken));
_sqlMgmtClient = new SqlManagementClient(new TokenCloudCredentials(_subscriptionId, _token.AccessToken));

Console.WriteLine("Resource group...");
ResourceGroup rg = CreateOrUpdateResourceGroup(_resourceMgmtClient, _subscriptionId, _resourceGroupName,
_resourceGrouplocation);
Console.WriteLine("Resource group: " + rg.Id);

Console.WriteLine("Server...");
ServerGetResponse sgr = CreateOrUpdateServer(_sqlMgmtClient, _resourceGroupName, _serverlocation, _serverName, _serverAdmin,
_serverAdminPassword);
Console.WriteLine("Server: " + sgr.Server.Id);

Console.WriteLine("Server firewall...");
FirewallRuleGetResponse fwr = CreateOrUpdateFirewallRule(_sqlMgmtClient, _resourceGroupName, _serverName, _firewallRuleName,
_startIpAddress, _endIpAddress);
Console.WriteLine("Server firewall: " + fwr.FirewallRule.Id);

Console.WriteLine("Elastic pool...");
ElasticPoolCreateOrUpdateResponse epr = CreateOrUpdateElasticDatabasePool(_sqlMgmtClient, _resourceGroupName, _serverName,
_poolName, _poolEdition, _poolDtus, _databaseMinDtus, _databaseMaxDtus);
Console.WriteLine("Elastic pool: " + epr.ElasticPool.Id);

Console.WriteLine("Database...");
DatabaseCreateOrUpdateResponse dbr = CreateOrUpdateDatabase(_sqlMgmtClient, _resourceGroupName, _serverName,
_databaseName, _poolName);
Console.WriteLine("Database: " + dbr.Database.Id);

Console.WriteLine("Press any key to continue...");

Console.ReadKey();
}

static ResourceGroup CreateOrUpdateResourceGroup(ResourceManagementClient resourceMgmtClient, string subscriptionId, string

resourceGroupName, string resourceGroupLocation)
{
ResourceGroup resourceGroupParameters = new ResourceGroup()
{
Location = resourceGroupLocation,
};
resourceMgmtClient.SubscriptionId = subscriptionId;
ResourceGroup resourceGroupResult = resourceMgmtClient.ResourceGroups.CreateOrUpdate(resourceGroupName,
resourceGroupParameters);
return resourceGroupResult;
}

static ServerGetResponse CreateOrUpdateServer(SqlManagementClient sqlMgmtClient, string resourceGroupName, string serverLocation,

string serverName, string serverAdmin, string serverAdminPassword)
{
ServerCreateOrUpdateParameters serverParameters = new ServerCreateOrUpdateParameters()
{
Location = serverLocation,
Properties = new ServerCreateOrUpdateProperties()
{
{
AdministratorLogin = serverAdmin,
AdministratorLoginPassword = serverAdminPassword,
Version = "12.0"
}
};
ServerGetResponse serverResult = sqlMgmtClient.Servers.CreateOrUpdate(resourceGroupName, serverName, serverParameters);
return serverResult;
}

static FirewallRuleGetResponse CreateOrUpdateFirewallRule(SqlManagementClient sqlMgmtClient, string resourceGroupName, string

serverName, string firewallRuleName, string startIpAddress, string endIpAddress)
{
FirewallRuleCreateOrUpdateParameters firewallParameters = new FirewallRuleCreateOrUpdateParameters()
{
Properties = new FirewallRuleCreateOrUpdateProperties()
{
StartIpAddress = startIpAddress,
EndIpAddress = endIpAddress
}
};
FirewallRuleGetResponse firewallResult = sqlMgmtClient.FirewallRules.CreateOrUpdate(resourceGroupName, serverName,
firewallRuleName, firewallParameters);
return firewallResult;
}

static ElasticPoolCreateOrUpdateResponse CreateOrUpdateElasticDatabasePool(SqlManagementClient sqlMgmtClient, string

resourceGroupName, string serverName, string poolName, string poolEdition, int poolDtus, int databaseMinDtus, int databaseMaxDtus)
{
// Retrieve the server that will host this elastic pool
Server currentServer = sqlMgmtClient.Servers.Get(resourceGroupName, serverName).Server;

// Create elastic pool: configure create or update parameters and properties explicitly
ElasticPoolCreateOrUpdateParameters newPoolParameters = new ElasticPoolCreateOrUpdateParameters()
{
Location = currentServer.Location,
Properties = new ElasticPoolCreateOrUpdateProperties()
{
Edition = poolEdition,
Dtu = poolDtus,
DatabaseDtuMin = databaseMinDtus,
DatabaseDtuMax = databaseMaxDtus
}
};

// Create the pool

var newPoolResponse = sqlMgmtClient.ElasticPools.CreateOrUpdate(resourceGroupName, serverName, poolName,
newPoolParameters);
return newPoolResponse;
}

static DatabaseCreateOrUpdateResponse CreateOrUpdateDatabase(SqlManagementClient sqlMgmtClient, string resourceGroupName,

string serverName, string databaseName, string poolName)
{
// Retrieve the server that will host this database
Server currentServer = sqlMgmtClient.Servers.Get(resourceGroupName, serverName).Server;

// Create a database: configure create or update parameters and properties explicitly

DatabaseCreateOrUpdateParameters newDatabaseParameters = new DatabaseCreateOrUpdateParameters()
{
Location = currentServer.Location,
Properties = new DatabaseCreateOrUpdateProperties()
{
CreateMode = DatabaseCreateMode.Default,
CreateMode = DatabaseCreateMode.Default,
ElasticPoolName = poolName
}
};
DatabaseCreateOrUpdateResponse dbResponse = sqlMgmtClient.Databases.CreateOrUpdate(resourceGroupName, serverName,
databaseName, newDatabaseParameters);
return dbResponse;
}

private static AuthenticationResult GetToken(string tenantId, string applicationId, string applicationSecret)

{
AuthenticationContext authContext = new AuthenticationContext("https://login.windows.net/" + tenantId);
_token = authContext.AcquireToken("https://management.core.windows.net/", new ClientCredential(applicationId, applicationSecret));
return _token;
}
}
}

Create a service principal to access resources

The following PowerShell script creates the Active Directory (AD) application and the service principal that we
need to authenticate our C# app. The script outputs values we need for the preceding C# sample. For detailed
information, see Use Azure PowerShell to create a service principal to access resources.

# Sign in to Azure.
Add-AzureRmAccount

# Provide these values for your new AAD app.

# $appName is the display name for your app, must be unique in your directory.
# $uri does not need to be a real uri.
# $secret is a password you create.

$appName = "{app-name}"
$uri = "http://{app-name}"
$secret = "{app-password}"

# Create a AAD app

$azureAdApplication = New-AzureRmADApplication -DisplayName $appName -HomePage $Uri -IdentifierUris $Uri -Password $secret

# Create a Service Principal for the app

$svcprincipal = New-AzureRmADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId

# To avoid a PrincipalNotFound error, I pause here for 15 seconds.

Start-Sleep -s 15

# Output the values we need for our C# application to successfully authenticate

Write-Output "Copy these values into the C# sample app"

Write-Output "_subscriptionId:" (Get-AzureRmContext).Subscription.SubscriptionId

Write-Output "_tenantId:" (Get-AzureRmContext).Tenant.TenantId
Write-Output "_applicationId:" $azureAdApplication.ApplicationId.Guid
Write-Output "_applicationSecret:" $secret
To complete the steps in this article, you need the following items:
An elastic pool. To create an elastic, see Create an elastic pool with C#.
Visual Studio. For a free copy of Visual Studio, see the Visual Studio Downloads page.

Move a database into an elastic pool

You can move a stand-alone database in or out of an elastic.

// Retrieve current database properties.

currentDatabase = sqlClient.Databases.Get("resourcegroup-name", "server-name", "Database1").Database;

// Configure create or update parameters with existing property values, override those to be changed.
DatabaseCreateOrUpdateParameters updatePooledDbParameters = new DatabaseCreateOrUpdateParameters()
{
Location = currentDatabase.Location,
Properties = new DatabaseCreateOrUpdateProperties()
{
Edition = "Standard",
RequestedServiceObjectiveName = "ElasticPool",
ElasticPoolName = "ElasticPool1",
MaxSizeBytes = currentDatabase.Properties.MaxSizeBytes,
Collation = currentDatabase.Properties.Collation,
}
};

// Update the database.

var dbUpdateResponse = sqlClient.Databases.CreateOrUpdate("resourcegroup-name", "server-name", "Database1",
updatePooledDbParameters);

List databases in an elastic pool

To retrieve all databases in an elastic, call the ListDatabases method.

//List databases in the elastic pool

DatabaseListResponse dbListInPool = sqlClient.ElasticPools.ListDatabases("resourcegroup-name", "server-name", "ElasticPool1");
Console.WriteLine("Databases in Elastic Pool {0}", "server-name.ElasticPool1");
foreach (Database db in dbListInPool)
{
Console.WriteLine(" Database {0}", db.Name);
}

Change performance settings of an elastic pool

Retrieve existing the pool properties. Modify the values and execute the CreateOrUpdate method.
var currentPool = sqlClient.ElasticPools.Get("resourcegroup-name", "server-name", "ElasticPool1").ElasticPool;

// Configure create or update parameters with existing property values, override those to be changed.
ElasticPoolCreateOrUpdateParameters updatePoolParameters = new ElasticPoolCreateOrUpdateParameters()
{
Location = currentPool.Location,
Properties = new ElasticPoolCreateOrUpdateProperties()
{
Edition = currentPool.Properties.Edition,
DatabaseDtuMax = 50, /* Setting DatabaseDtuMax to 50 limits the eDTUs that any one database can consume */
DatabaseDtuMin = 10, /* Setting DatabaseDtuMin above 0 limits the number of databases that can be stored in the pool */
Dtu = (int)currentPool.Properties.Dtu,
StorageMB = currentPool.Properties.StorageMB, /* For a Standard pool there is 1 GB of storage per eDTU. */
}
};

newPoolResponse = sqlClient.ElasticPools.CreateOrUpdate("resourcegroup-name", "server-name", "ElasticPool1", newPoolParameters);

Latency of elastic pool operations

Changing the min eDTUs per database or max eDTUs per database typically completes in five minutes or less.
Time to change the pool size (eDTUs) depends on the combined size of all databases in the pool. Changes
average 90 minutes or less per 100 GB. For example, if the total space of all databases in the pool is 200 GB,
then the expected latency for changing the pool eDTU per pool is 3 hours or less.

Additional Resources
For SQL error codes for SQL Database client applications, database connection error and other issues, see
Error messages.
Azure Resource Management APIs
For elastic pool guidance, see When should an elastic pool be used?
SQL Server authentication, access, and database-
level firewall rules
4/14/2017 • 12 min to read • Edit Online

In this tutorial, you learn how to use SQL Server Management Studio to work with SQL Server authentication,
logins, users, and database roles that grant access and permissions to Azure SQL Database servers and
databases. After you complete this tutorial, you will know how to:
Create logins and users based on SQL Server authentication
Add users to roles and grant permissions to roles
Use T-SQL to create a database-level and a server-level firewall rule
Connect as a user to a specific database using SSMS
View user permissions in the master database and in user databases
Time estimate: This tutorial takes approximately 45 minutes to complete (assuming you have already met the
prerequisites).

NOTE
This tutorial helps you to learn the content of these topics: SQL Database access and control, Logins, users, and database
roles, Principals, Database roles, and SQL Database firewall rules. For a tutorial about Azure Active Directory authentication,
see Getting started with Azure AD Authentication.

Prerequisites
An Azure account. You need an Azure account. You can open a free Azure account or Activate Visual
Studio subscriber benefits.
Azure create permissions. You must be able to connect to the Azure portal using an account that is a
member of either the subscription owner or contributor role. For more information on role-based access
control (RBAC), see Getting started with access management in the Azure portal.
SQL Server Management Studio. You can download and install the latest version of SQL Server
Management Studio (SSMS) at Download SQL Server Management Studio. Always use the latest version
of SSMS when connecting to Azure SQL Database as new capabilities are continually being released.
Base server and databases To install and configure a server and the two databases used in this tutorial,
click the Deploy to Azure button. Clicking the button opens the Deploy from a template blade; create a
new resource group, and provide the Admin Login Password for the new server that will be created:

Sign in to the Azure portal using your Azure account

The steps in this procedure show you how to connect to the Azure portal using your Azure account]
(https://account.windowsazure.com/Home/Index).
1. Open your browser of choice and connect to the Azure portal.
2. Sign in to the Azure portal.
3. On the Sign in page, provide the credentials for your subscription.

View logical server security information in the Azure portal

The steps in this procedure show you how to view information about the security configuration for your logical
server in the Azure portal.
1. Open the SQL Server blade for your server and view the information in the Overview page.

2. Make note of the name for server admin on your logical server.
3. If you do not remember the password, click Reset password to set a new password.
4. If you need to get the connection information for this server, click Properties.

View server admin permissions using SSMS

The steps in this procedure show you how to view information about the server admin account and its
permissions in the master database and in user databases.
1. Open SQL Server Management Studio and connect to your server as the server admin using SQL Server
Authentication and the Server admin account.

2. Click Connect.
3. In Object Explorer, expand Security, and then expand Logins to view the existing logins for your server -
the only login on a new server is the login for server admin account.

4. In Object Explorer, expand Databases, expand System databases, expand master, expand Security, and
then expand Users to view the user account that was created for the server admin login in this database.

NOTE
For information about the other user accounts that appear in the Users node, see Principals.

5. In Object Explorer, right-click master and then click New Query to open a query window connected to the
master database.
6. In the query window, execute the following query to return information about the user executing the
query.

SELECT USER;

7. In the query window, execute the following query to return information about the permissions of the
sqladmin user in the master database.

SELECT prm.permission_name
, prm.class_desc
, prm.state_desc
, p2.name as 'Database role'
, p3.name as 'Additional database role'
FROM sys.database_principals p
JOIN sys.database_permissions prm
ON p.principal_id = prm.grantee_principal_id
LEFT JOIN sys.database_principals p2
ON prm.major_id = p2.principal_id
LEFT JOIN sys.database_role_members r
ON p.principal_id = r.member_principal_id
LEFT JOIN sys.database_principals p3
ON r.role_principal_id = p3.principal_id
WHERE p.name = 'sqladmin';
NOTE
The server admin has permissions to connect to the master database, create logins and users, select information
from the sys.sql_logins table, and add users to the dbmanager and dbcreator database roles. These permissions are
in addition to permissions granted to the public role from which all users inherit permissions (such as permissions
to select information from certain tables). See Permissions for more information.

8. In Object Explorer, expand blankdb, expand Security, and then expand Users to view the user account
that was created for the server admin login in this database (and in each user database).

9. In Object Explorer, right-click blankdb and then click New Query.

10. In the query window, execute the following query to return information about the user executing the
query.

SELECT USER;
11. In the query window, execute the following query to return information about the permissions of the dbo
user.

SELECT prm.permission_name
, prm.class_desc
, prm.state_desc
, p2.name as 'Database role'
, p3.name as 'Additional database role'
FROM sys.database_principals AS p
JOIN sys.database_permissions AS prm
ON p.principal_id = prm.grantee_principal_id
LEFT JOIN sys.database_principals AS p2
ON prm.major_id = p2.principal_id
LEFT JOIN sys.database_role_members r
ON p.principal_id = r.member_principal_id
LEFT JOIN sys.database_principals AS p3
ON r.role_principal_id = p3.principal_id
WHERE p.name = 'dbo';

NOTE
The dbo user is a member of the public role and also a member of the db_owner fixed database role. See Database-
Level Roles for more information.

Create a new user with SELECT permissions

The steps in this procedure show you how to create a database-level user, test the default permissions of a new
user(through the public role), grant a user SELECT permissions, and view these modified permissions.

NOTE
Database-level users are also called contained users and increase the portability of your database. For information about
the benefits of portability, see Configure and manage Azure SQL Database security for geo-restore or failover to a
secondary server.

1. In Object Explorer, right-click sqldbtutorialdb and then click New Query.

2. Execute the following statement in this query window to create a user called user1 in the sqldbtutorialdb
database.

CREATE USER user1

WITH PASSWORD = 'p@ssw0rd';

3. In the query window, execute the following query to return information about the permissions of user1.

NOTE
A new user in a database only has the permissions inherited from the public role.

4. Execute the following queries using the EXECUTE AS USER statement to attempt to query the
SalesLT.ProductCategory table in the sqldbtutorialdb database as user1 with only the permissions
inherited from the public role.

EXECUTE AS USER = 'user1';

SELECT * FROM [SalesLT].[ProductCategory];
REVERT;

NOTE
By default, the public role does not grant SELECT permissions on user objects.

5. Execute the following statement to grant SELECT permissions on the SalesLT.ProductCategory table to
user1.

GRANT SELECT ON OBJECT::[SalesLT].[ProductCategory] to user1;

6. Execute the following queries to successfully query the SalesLT.ProductCategory table in the
sqldbtutorialdb database as user1.

EXECUTE AS USER = 'user1';

SELECT * FROM [SalesLT].[ProductCategory];
REVERT;
Create a database-level firewall rule using T-SQL
The steps in this procedure show you how to create a database-level firewall rule using the
sp_set_database_firewall_rule system stored procedure. A database-level firewall rule allows a server admin to
allow users through the Azure SQL Database firewall only for specific databases.

NOTE
Database-level firewall rules increase the portability of your database. For information about the benefits of portability, see
Configure and manage Azure SQL Database security for geo-restore or failover to a secondary server.

IMPORTANT
To test a database-level firewall rule, connect from another computer (or delete the server-level firewall rule in the Azure
portal.)

1. Open SQL Server Management Studio on a computer for which you do not have a server-level firewall
rule.
2. In the Connect to Server window, enter the server name and authentication information to connect using
SQL Server authentication with the user1 account.
3. Click Options in the Connect to server dialog box to specify the database to which you want to connect
and then type sqldbtutorialdb in the Connect to Database drop-down box on the Connection
Properties tab.

4. Click Connect.
A dialog box appears informing you that the computer from which you are attempting to connect to SQL
Database does not have a firewall rule enabling access to the database.

5. Copy the client IP address from this dialog box for use in step 8.
6. Click OK to close the error dialog box, but do not close the Connect to Server dialog box.
7. Switch back to a computer for which you have already created a server-level firewall rule.
8. Connect to the sqldbtutorialdb database in SSMS as server admin and execute the following statement to
create a database-level firewall using the IP address (or address range) from step 5.

EXEC sp_set_database_firewall_rule @name = N'sqldbtutorialdbFirewallRule',

@start_ip_address = 'x.x.x.x', @end_ip_address = 'x.x.x.x';
9. Switch computers again and click Connect in the Connect to Server dialog box to connect to
sqldbtutorialdb as user1.

NOTE
After you create the database-level firewall rule, it may take up to 5 minutes to become active.

10. After you connect successfully, expand Databases in Object Explorer. Notice that user1 can only view the
sqldbtutorialdb database.

11. Expand sqldbtutorialdb, and then expand Tables. Notice that user1 only has permission to view a single
table, the SalesLT.ProductCategory table.

Create a new user as db_owner and a database-level firewall rule

The steps in this procedure show you how to create a user in another database with db_owner database role
permissions and create a database-level firewall for this other database. This new user with db_owner role
membership will only be able to connect to and manage this single database.
1. Switch to your computer with a connection to SQL Database using the server admin account.
2. Open a query window connected to the blankdb database and execute the following statement to create a
user called blankdbadmin in the blankdb database.

CREATE USER blankdbadmin

WITH PASSWORD = 'p@ssw0rd';

3. In the same query window, execute the following statement to add the blankdbadmin user to the
db_owner database role. This user can now perform all actions necessary to manage the blankdb database.

ALTER ROLE db_owner ADD MEMBER blankdbadmin;

4. In the same query window, execute the following statement to create a database-level firewall by executing
sp_set_database_firewall_rule using the IP address from step 4 in the previous procedure (or a range of IP
addresses for users of this database):

EXEC sp_set_database_firewall_rule @name = N'blankdbFirewallRule',

@start_ip_address = 'x.x.x.x', @end_ip_address = 'x.x.x.x';

5. Switch computers (to one for which you have created a database-level firewall rule) and connect to the
blankdb database using the blankdbadmin user account.
6. Open a query window to the blankdb database and execute the following statement to create a user called
blankdbuser1 in the blankdb database.

CREATE USER blankdbuser1

WITH PASSWORD = 'p@ssw0rd';

7. As necessary for your learning environment, create an additional database-level firewall rule for this user.
However, if you created the database-level firewall rule using an IP address range, this may not be
necessary.

Grant dbmanager permissions and create a server-level firewall rule

The steps in this procedure show you how to create a login and user in the master database with permissions to
create and manage new user databases. The steps also show you how to create an additional server-level firewall
rule using Transact-SQL using sp_set_firewall_rule.

IMPORTANT
The first server-level firewall rule must always be created in Azure (in the Azure portal, using PowerShell, or the REST API).

IMPORTANT
Creating logins in the master database and creating a user account from a login is required for the server admin to
delegate create database permissions to another user. However, creating logins and then creating users from logins
decreases the portability of your environment.

1. Switch to your computer with a connection to SQL Database using the server admin account.
2. Open a query window connected to the master database and execute the following statement to create a
login called dbcreator in the master database.
CREATE LOGIN dbcreator
WITH PASSWORD = 'p@ssw0rd';

3. In the same query window,

CREATE USER dbcreator

FROM LOGIN dbcreator;

4. In the same query window, execute the following query to add the dbcreator user to the dbmanager
database role. This user can now create and manage databases created by the user.

ALTER ROLE dbmanager ADD MEMBER dbcreator;

5. In the same query window, execute the following query to create a server-level firewall by executing
sp_set_firewall_rule using an IP address appropriate for your environment:

EXEC sp_set_firewall_rule @name = N'dbcreatorFirewallRule',

@start_ip_address = 'x.x.x.x', @end_ip_address = 'x.x.x.x';

6. Switch computers (to one for which you have created a server-level firewall rule) and connect to the
master database using the dbcreator user account.
7. Open a query window to the master database and execute the following query to create a database called
foo.

CREATE DATABASE FOO (EDITION = 'basic');

a. Optionally, delete this database to save money using the following statement:

DROP DATABASE FOO;

Complete script
To create the logins and users, add them to roles, grant them permissions, create database-level firewall rules,
and create server-level firewall rules, execute the following statements in the appropriate databases on your
server.
master database
Execute these statements in the master database using the server admin account, adding the appropriate IP
addresses or range.

CREATE LOGIN dbcreator WITH PASSWORD = 'p@ssw0rd';

CREATE USER dbcreator FROM LOGIN dbcreator;
ALTER ROLE dbmanager ADD MEMBER dbcreator;
EXEC sp_set_firewall_rule @name = N'dbcreatorFirewallRule',
@start_ip_address = 'x.x.x.x', @end_ip_address = 'x.x.x.x';

sqldbtutorialdb database
Execute these statements in the sqldbtutorialdb database using the server admin account, adding the appropriate
IP addresses or range.
CREATE USER user1 WITH PASSWORD = 'p@ssw0rd';
GRANT SELECT ON OBJECT::[SalesLT].[ProductCategory] to user1;
EXEC sp_set_database_firewall_rule @name = N'sqldbtutorialdbFirewallRule',
@start_ip_address = 'x.x.x.x', @end_ip_address = 'x.x.x.x';

blankdb database
Execute these statements in the blankdb database using the server admin account, adding the appropriate IP
addresses or range.

CREATE USER blankdbadmin

WITH PASSWORD = 'p@ssw0rd';
ALTER ROLE db_owner ADD MEMBER blankdbadmin;
EXEC sp_set_database_firewall_rule @name = N'blankdbFirewallRule',
@start_ip_address = 'x.x.x.x', @end_ip_address = 'x.x.x.x';
CREATE USER blankdbuser1
WITH PASSWORD = 'p@ssw0rd';

Next steps
For an overview of access and control in SQL Database, see SQL Database access and control.
For an overview of logins, users, and database roles in SQL Database, see Logins, users, and database roles.
For more information about database principals, see Principals.
For more information about database roles, see Database roles.
For more information about firewall rules in SQL Database, see SQL Database firewall rules.
For a tutorial using Azure Active Directory authentication, see Azure AD authentication and authorization.
Azure AD authentication, access, and database-level
firewall rules
4/14/2017 • 10 min to read • Edit Online

In this how-to guide, you learn how to use SQL Server Management Studio to work with Azure Active Directory
authentication, logins, users, and database roles that grant access and permissions to Azure SQL Database
servers and databases. You learn to:
View user permissions in the master database and in user databases
Create logins and users based on Azure Active Directory authentication
Grant server-wide and database-specific permissions to users
Log in to a user database as a non-admin user
Create database-level firewall rules for database users
Create server-level firewall rules for server admins
Time estimate: This how-to guide takes approximately 45 minutes to complete (assuming you have already met
the prerequisites).

Prerequisites
An Azure account. You need an Azure account. You can open a free Azure account or Activate Visual
Studio subscriber benefits.
Azure create permissions. You must be able to connect to the Azure portal using an account that is a
member of either the subscription owner or contributor role. For more information on role-based access
control (RBAC), see Getting started with access management in the Azure portal.
SQL Server Management Studio. You can download and install the latest version of SQL Server
Management Studio (SSMS) at Download SQL Server Management Studio. Always use the latest version
of SSMS when connecting to Azure SQL Database as new capabilities are continually being released.
Base server and databases To install and configure a server and the two databases used in this how-to
guide, click the Deploy to Azure button. Clicking the button opens the Deploy from a template blade;
create a new resource group, and provide the Admin Login Password for the new server that will be
created:

NOTE
The completion of the related how-to guide for SQL Server authentication, SQL authentication, logins and user
accounts, database roles, permissions, server-level firewall rules, and database-level firewall rules, is optional -
however, there are concepts covered in that how-to guide that are not repeated here. The procedures in this how-
to guide related to server and database level firewalls are not required if you completed this related how-to guide
on the same computers (with the same IP addresses) and are marked as optional for that reason. Also, the
screenshots in this how-to guide assume that you have completed of this related how-to guide.

You have created and populated an Azure Active Directory. For more information, see Integrating your on-
premises identities with Azure Active Directory, Add your own domain name to Azure AD, Microsoft Azure
now supports federation with Windows Server Active Directory, Administering your Azure AD directory,
Manage Azure AD using Windows PowerShell, and Hybrid Identity Required Ports and Protocols.

NOTE
This how-to guide helps you to learn the content of these learn topics: SQL Database access and control, Logins, users, and
database roles, Principals, Database roles, SQL Database firewall rules, and Azure Active Directory authentication.

Sign in to the Azure portal using your Azure account

Using your existing subscription, follow these steps to connect to the Azure portal.
1. Open your browser of choice and connect to the Azure portal.
2. Sign in to the Azure portal.
3. On the Sign in page, provide the credentials for your subscription.

Provision an Azure Active Directory admin for your SQL logical server
In this section of the how-to guide, you view information about the security configuration for your logical server
in the Azure portal.
1. Open the SQL Server blade for your logical server and view the information in the Overview page. Notice
that an Azure Active Directory admin has not been configured.

2. Click Not configured in the Essentials pane to open the Active Directory admin blade.

3. Click Set admin to open the Add admin blade and then select an Active Directory user or group account
as the Active Directory admin for your server.
4. Click Select and then click Save.

NOTE
To review connection information for this server, go to Connect with SSMS. For this how-to guide series, the fully qualified
server name is 'sqldbtutorialserver.database.windows.net'.

Connect to SQL server using SQL Server Management Studio (SSMS)

1. If you have not already done so, download and install the latest version of SSMS at Download SQL Server
Management Studio. To stay up-to-date, the latest version of SSMS prompts you when there is a new
version available to download.
2. After installing, type Microsoft SQL Server Management Studio in the Windows search box and click
Enter to open SSMS.

3. In the Connect to Server dialog box, select one of the Active Directory authentication methods and then
provide the appropriate authentication information. For information on choosing a method, see Azure
Active Directory authentication and SSMS support for Azure AD MFA.
4. Enter the necessary information to connect to your SQL server using SQL Server Authentication and the
Server admin account.
5. Click Connect.

View the Server admin account and its permissions

In this section of the how-to guide, you view information about the server admin account and its permissions in
the master database and in user databases.
1. In Object Explorer, expand Databases, expand System databases, expand master, expand Security, and
then expand Users. Notice that a user account has been created in the master database for the Active
Directory admin. Notice also that a login was not created for Active Directory admin user account.
NOTE
For information about the other user accounts that appear, see Principals.

2. In Object Explorer, right-click master and then click New Query to open a query window connected to the
master database.
3. In the query window, execute the following query to return information about the user executing the
query. Notice that user@microsoft.com is returned for the user account executing this query (we see a
different result when we query a user database later in this procedure).

SELECT USER;

4. In the query window, execute the following query to return information about the permissions of the
Active Directory admin user. Notice that the Active Directory admin user has permissions to connect to the
master database, create logins and users, select information from the sys.sql_logins table, and add users to
the dbmanager and dbcreator database roles. These permissions are in addition to permissions granted to
the public role from which all users inherit permissions (such as permissions to select information from
certain tables). See Permissions for more information.

5. In Object Explorer, expand blankdb, expand Security, and then expand Users. Notice that there is no user
account called user@microsoft.com in this database.
6. In Object Explorer, right-click blankdb and then click New Query.
7. In the query window, execute the following query to return information about the user executing the
query. Notice that dbo is returned for the user account executing this query (by default, the Server admin
login is mapped to the dbo user account in each user database).

SELECT USER;

8. In the query window, execute the following query to return information about the permissions of the dbo
user. Notice that dbo is a member of the public role and also a member of the db_owner fixed database
role. See Database-Level Roles for more information.
SELECT prm.permission_name
, prm.class_desc
, prm.state_desc
, p2.name as 'Database role'
, p3.name as 'Additional database role'
FROM sys.database_principals AS p
JOIN sys.database_permissions AS prm
ON p.principal_id = prm.grantee_principal_id
LEFT JOIN sys.database_principals AS p2
ON prm.major_id = p2.principal_id
LEFT JOIN sys.database_role_members r
ON p.principal_id = r.member_principal_id
LEFT JOIN sys.database_principals AS p3
ON r.role_principal_id = p3.principal_id
WHERE p.name = 'dbo';

9. Optionally, repeat the previous three steps for the AdventureWorksLT user database.

Create a new user in the AdventureWorksLT database with SELECT

permissions
In this section of the how-to guide, you create a user account in the AdventureWorksLT database based on a
user's principal name of an Azure AD user or display name for an Azure AD group, test this user's permissions as
member of the public role, grant this user SELECT permissions, and then test this user's permissions again.

NOTE
Database-level users (contained users) increase the portability of your database, a capability that we explore in later how-to
guides.

1. In Object Explorer, right-click AdventureWorksLT and then click New Query to open a query window
connected to the AdventureWorksLT database.
2. Execute the following statement to create a user account in the AdventureWorksLT database for a user in
the Microsoft domain called aaduser1.

CREATE USER [aaduser1@microsoft.com]

FROM EXTERNAL PROVIDER;
3. In the query window, execute the following query to return information about the permissions of user1.
Notice that the only permissions that user1 has are the permissions inherited from the public role.

4. Execute the following queries to attempt to query a table in the AdventureWorksLT database as user1.

EXECUTE AS USER = 'aaduser1@microsoft.com';

SELECT * FROM [SalesLT].[ProductCategory];
REVERT;
5. Execute the following statement to grant SELECT permissions on the ProductCategory table in the SalesLT
schema to user1.

GRANT SELECT ON OBJECT::[SalesLT].[ProductCategory] to [aaduser1@microsoft.com];

6. Execute the following queries to attempt to query a table in the AdventureWorksLT database as user1.

EXECUTE AS USER = 'aaduser1@microsoft.com';

SELECT * FROM [SalesLT].[ProductCategory];
REVERT;

Create a database-level firewall rule for AdventureWorksLT database

users
NOTE
You do not need to complete this procedure if you completed the equivalent procedure in the related how-to guide for
SQL Server authentication, SQL authentication and authorization and are learning using the same computer with the same
IP address.

In this section of the how-to guide, you attempt to log in using the new user account from a computer with a
different IP address, create a database-level firewall rule as the Server admin, and then successfully log in using
this new database-level firewall rule.

NOTE
Database-level firewall rules increase the portability of your database, a capability that we explore in later how-to guides.

1. On another computer for which you have not already created a server-level firewall rule, open SQL Server
Management Studio.

IMPORTANT
Always use the latest version of SSMS at Download SQL Server Management Studio.

2. In the Connect to Server window, enter the server name and authentication information to connect using
SQL Server authentication with the aaduser1@microsoft.com account.

3. Click Options in the Connect to server dialog box to specify the database to which you want to connect
and then type AdventureWorksLT in the Connect to Database drop-down box on the Connection
Properties tab.
4. Click Connect. A dialog box appears informing you that the computer from which you are attempting to
connect to SQL Database does not have a firewall rule enabling access to the database. The dialog box that
you receive has two variations depending upon steps you have previously taken with firewalls, but you
usually get the first dialog box shown.
NOTE
The newest versions of SSMS include the functionality to allow subscription owners and contributors to sign in to
Microsoft Azure and create a server-level firewall rule.

5. Copy the client IP address from this dialog box for use in step 7.
6. Click Cancel but do not close the Connect to Server dialog box.
7. Switch back to a computer for which you have already created a server-level firewall rule and connect to your
server using the Server admin account.
8. In a new query window connected to the AdventureWorksLT database as Server admin, execute the
following statement to create a database-level firewall by executing sp_set_database_firewall_rule using
the IP address from step 4:

EXEC sp_set_database_firewall_rule @name = N'AdventureWorksLTFirewallRule',

@start_ip_address = 'x.x.x.x', @end_ip_address = 'x.x.x.x';

9. Switch computers again and click Connect in the Connect to Server dialog box to connect to
AdventureWorksLT as aaduser1.
10. In Object Explorer, expand Databases, expand AdventureWorksLT, and then expand Tables. Notice that
user1 only has permission to view a single table, the SalesLT.ProductCategory table.
11. In Object Explorer, right-click SalesLT.ProductCategory and click Select Top 1000 Rows.

Next steps
For an overview of access and control in SQL Database, see SQL Database access and control.
For an overview of logins, users, and database roles in SQL Database, see Logins, users, and database roles.
For more information about database principals, see Principals.
For more information about database roles, see Database roles.
For more information about firewall rules in SQL Database, see SQL Database firewall rules.
Configure and manage Azure Active Directory
authentication with SQL Database or SQL Data
Warehouse
4/14/2017 • 16 min to read • Edit Online

This article shows you how to create and populate Azure AD, and then use Azure AD with Azure SQL Database and
SQL Data Warehouse. For an overview, see Azure Active Directory Authentication.

Create and populate an Azure AD

Create an Azure AD and populate it with users and groups. Azure AD can be the initial domain Azure AD managed
domain. Azure AD can also be an on-premises Active Directory Domain Services that is federated with the Azure
AD.
For more information, see Integrating your on-premises identities with Azure Active Directory, Add your own
domain name to Azure AD, Microsoft Azure now supports federation with Windows Server Active Directory,
Administering your Azure AD directory, Manage Azure AD using Windows PowerShell, and Hybrid Identity
Required Ports and Protocols.

Optional: Associate or change the active directory that is currently

associated with your Azure Subscription
To associate your database with the Azure AD directory for your organization, make the directory a trusted
directory for the Azure subscription hosting the database. For more information, see How Azure subscriptions are
associated with Azure AD.
Additional information: Every Azure subscription has a trust relationship with an Azure AD instance. This means
that it trusts that directory to authenticate users, services, and devices. Multiple subscriptions can trust the same
directory, but a subscription trusts only one directory. You can see which directory is trusted by your subscription
under the Settings tab at https://manage.windowsazure.com/. This trust relationship that a subscription has with a
directory is unlike the relationship that a subscription has with all other resources in Azure (websites, databases,
and so on), which are more like child resources of a subscription. If a subscription expires, then access to those
other resources associated with the subscription also stops. But the directory remains in Azure, and you can
associate another subscription with that directory and continue to manage the directory users. For more
information about resources, see Understanding resource access in Azure.
The following procedures show you how to change the associated directory for a given subscription.
1. Connect to your Azure Classic Portal by using an Azure subscription administrator.
2. On the left banner, select SETTINGS.
3. Your subscriptions appear in the settings screen. If the desired subscription does not appear, click
Subscriptions at the top, drop down the FILTER BY DIRECTORY box and select the directory that contains
your subscriptions, and then click APPLY.
4. In the settings area, click your subscription, and then click EDIT DIRECTORY at the bottom of the page.

5. In the EDIT DIRECTORY box, select the Azure Active Directory that is associated with your SQL Server or
SQL Data Warehouse, and then click the arrow for next.
6. In the CONFIRM directory Mapping dialog box, confirm that "All co-administrators will be removed."

7. Click the check to reload the portal.

NOTE
When you change the directory, access to all co-administrators, Azure AD users and groups, and directory-backed
resource users are removed and they no longer have access to this subscription or its resources. Only you, as a
service administrator, can configure access for principals based on the new directory. This change might take a
substantial amount of time to propagate to all resources. Changing the directory, also changes the Azure AD
administrator for SQL Database and SQL Data Warehouse and disallow database access for any existing Azure AD
users. The Azure AD admin must be reset (as described below) and new Azure AD users must be created.

Create an Azure AD administrator for Azure SQL server

Each Azure SQL server (which hosts a SQL Database or SQL Data Warehouse) starts with a single server
administrator account that is the administrator of the entire Azure SQL server. A second SQL Server administrator
must be created, that is an Azure AD account. This principal is created as a contained database user in the master
database. As administrators, the server administrator accounts are members of the db_owner role in every user
database, and enter each user database as the dbo user. For more information about the server administrator
accounts, see Managing Databases and Logins in Azure SQL Database.
When using Azure Active Directory with geo-replication, the Azure Active Directory administrator must be
configured for both the primary and the secondary servers. If a server does not have an Azure Active Directory
administrator, then Azure Active Directory logins and users receive a "Cannot connect" to server error.

NOTE
Users that are not based on an Azure AD account (including the Azure SQL server administrator account), cannot create
Azure AD-based users, because they do not have permission to validate proposed database users with the Azure AD.

Provision an Azure Active Directory administrator for your Azure SQL

server
The following two procedures show you how to provision an Azure Active Directory administrator for your Azure
SQL server in the Azure portal and by using PowerShell.
Azure portal
1. In the Azure portal, in the upper-right corner, click your connection to drop down a list of possible Active
Directories. Choose the correct Active Directory as the default Azure AD. This step links the subscription
association with Active Directory with Azure SQL server making sure that the same subscription is used for
both Azure AD and SQL Server. (The Azure SQL server can be hosting either Azure SQL Database or Azure
SQL Data Warehouse.)

2. In the left banner select SQL servers, select your SQL server, and then in the SQL Server blade, at the top
click Settings.
3. In the Settings blade, click Active Directory admin.
4. In the Active Directory admin blade, click Active Directory admin, and then at the top, click Set admin.
5. In the Add admin blade, search for a user, select the user or group to be an administrator, and then click
Select. (The Active Directory admin blade shows all members and groups of your Active Directory. Users or
groups that are grayed out cannot be selected because they are not supported as Azure AD administrators. (See
the list of supported admins in Azure AD Features and Limitations above.) Role-based access control (RBAC)
applies only to the portal and is not propagated to SQL Server.
6. At the top of the Active Directory admin blade, click SAVE.

The process of changing the administrator may take several minutes. Then the new administrator appears in
the Active Directory admin box.
NOTE
When setting up the Azure AD admin, the new admin name (user or group) cannot already be present in the virtual
master database as a SQL Server authentication user. If present, the Azure AD admin setup will fail; rolling back its
creation and indicating that such an admin (name) already exists. Since such a SQL Server authentication user is not
part of the Azure AD, any effort to connect to the server using Azure AD authentication fails.

To later remove an Admin, at the top of the Active Directory admin blade, click Remove admin, and then click
Save.
PowerShell
To run PowerShell cmdlets, you need to have Azure PowerShell installed and running. For detailed information, see
How to install and configure Azure PowerShell.
To provision an Azure AD admin, execute the following Azure PowerShell commands:
Add-AzureRmAccount
Select-AzureRmSubscription
Cmdlets used to provision and manage Azure AD admin:

CMDLET NAME DESCRIPTION

Set-AzureRmSqlServerActiveDirectoryAdministrator Provisions an Azure Active Directory administrator for Azure

SQL server or Azure SQL Data Warehouse. (Must be from the
current subscription.)

Remove-AzureRmSqlServerActiveDirectoryAdministrator Removes an Azure Active Directory administrator for Azure

SQL server or Azure SQL Data Warehouse.

Get-AzureRmSqlServerActiveDirectoryAdministrator Returns information about an Azure Active Directory

administrator currently configured for the Azure SQL server or
Azure SQL Data Warehouse.

Use PowerShell command get-help to see more details for each of these commands, for example
get-help Set-AzureRmSqlServerActiveDirectoryAdministrator .

The following script provisions an Azure AD administrator group named DBA_Group (object id
40b79501-b343-44ed-9ce7-da4c8cc7353f ) for the demo_server server in a resource group named Group-23:

Set-AzureRmSqlServerActiveDirectoryAdministrator -ResourceGroupName "Group-23"

-ServerName "demo_server" -DisplayName "DBA_Group"

The DisplayName input parameter accepts either the Azure AD display name or the User Principal Name. For
example, DisplayName="John Smith" and DisplayName="johns@contoso.com" . For Azure AD groups only the Azure AD
display name is supported.

NOTE
The Azure PowerShell command Set-AzureRmSqlServerActiveDirectoryAdministrator does not prevent you from
provisioning Azure AD admins for unsupported users. An unsupported user can be provisioned, but can not connect to a
database.

The following example uses the optional ObjectID:

Set-AzureRmSqlServerActiveDirectoryAdministrator -ResourceGroupName "Group-23"
-ServerName "demo_server" -DisplayName "DBA_Group" -ObjectId "40b79501-b343-44ed-9ce7-da4c8cc7353f"

NOTE
The Azure AD ObjectID is required when the DisplayName is not unique. To retrieve the ObjectID and DisplayName
values, use the Active Directory section of Azure Classic Portal, and view the properties of a user or group.

The following example returns information about the current Azure AD admin for Azure SQL server:

Get-AzureRmSqlServerActiveDirectoryAdministrator -ResourceGroupName "Group-23" -ServerName "demo_server" | Format-List

The following example removes an Azure AD administrator:

Remove-AzureRmSqlServerActiveDirectoryAdministrator -ResourceGroupName "Group-23" -ServerName "demo_server"

You can also provision an Azure Active Directory Administrator by using the REST APIs. For more information, see
Service Management REST API Reference and Operations for Azure SQL Databases Operations for Azure SQL
Databases

Configure your client computers

On all client machines, from which your applications or users connect to Azure SQL Database or Azure SQL Data
Warehouse using Azure AD identities, you must install the following software:
.NET Framework 4.6 or later from https://msdn.microsoft.com/library/5a4x27ek.aspx.
Azure Active Directory Authentication Library for SQL Server (ADALSQL.DLL) is available in multiple languages
(both x86 and amd64) from the download center at Microsoft Active Directory Authentication Library for
Microsoft SQL Server.
You can meet these requirements by:
Installing either SQL Server 2016 Management Studio or SQL Server Data Tools for Visual Studio 2015 meets
the .NET Framework 4.6 requirement.
SSMS installs the x86 version of ADALSQL.DLL.
SSDT installs the amd64 version of ADALSQL.DLL.
The latest Visual Studio from Visual Studio Downloads meets the .NET Framework 4.6 requirement, but does
not install the required amd64 version of ADALSQL.DLL.

Create contained database users in your database mapped to Azure AD

identities
Azure Active Directory authentication requires database users to be created as contained database users. A
contained database user based on an Azure AD identity, is a database user that does not have a login in the master
database, and which maps to an identity in the Azure AD directory that is associated with the database. The Azure
AD identity can be either an individual user account or a group. For more information about contained database
users, see Contained Database Users- Making Your Database Portable.
NOTE
Database users (with the exception of administrators) cannot be created using portal. RBAC roles are not propagated to SQL
Server, SQL Database, or SQL Data Warehouse. Azure RBAC roles are used for managing Azure Resources, and do not apply
to database permissions. For example, the SQL Server Contributor role does not grant access to connect to the SQL
Database or SQL Data Warehouse. The access permission must be granted directly in the database using Transact-SQL
statements.

To create an Azure AD-based contained database user (other than the server administrator that owns the
database), connect to the database with an Azure AD identity, as a user with at least the ALTER ANY USER
permission. Then use the following Transact-SQL syntax:

CREATE USER <Azure_AD_principal_name> FROM EXTERNAL PROVIDER;

Azure_AD_principal_name can be the user principal name of an Azure AD user or the display name for an Azure
AD group.
Examples: To create a contained database user representing an Azure AD federated or managed domain user:

CREATE USER [bob@contoso.com] FROM EXTERNAL PROVIDER;

CREATE USER [alice@fabrikam.onmicrosoft.com] FROM EXTERNAL PROVIDER;

To create a contained database user representing an Azure AD or federated domain group, provide the display
name of a security group:

CREATE USER [ICU Nurses] FROM EXTERNAL PROVIDER;

To create a contained database user representing an application that connects using an Azure AD token:

CREATE USER [appName] FROM EXTERNAL PROVIDER;

TIP
You cannot directly create a user from an Azure Active Directory other than the Azure Active Directory that is associated
with your Azure subscription. However, members of other Active Directories that are imported users in the associated Active
Directory (known as external users) can be added to an Active Directory group in the tenant Active Directory. By creating a
contained database user for that AD group, the users from the external Active Directory can gain access to SQL Database.

For more information about creating contained database users based on Azure Active Directory identities, see
CREATE USER (Transact-SQL).

NOTE
Removing the Azure Active Directory administrator for Azure SQL server prevents any Azure AD authentication user from
connecting to the server. If necessary, unusable Azure AD users can be dropped manually by a SQL Database administrator.
NOTE
If you receive a Connection Timeout Expired, you may need to set the TransparentNetworkIPResolution parameter of
the connection string to false. For more information, see Connection timeout issue with .NET Framework 4.6.1 -
TransparentNetworkIPResolution.

When you create a database user, that user receives the CONNECT permission and can connect to that database
as a member of the PUBLIC role. Initially the only permissions available to the user are any permissions granted to
the PUBLIC role, or any permissions granted to any Windows groups that they are a member of. Once you
provision an Azure AD-based contained database user, you can grant the user additional permissions, the same
way as you grant permission to any other type of user. Typically grant permissions to database roles, and add
users to roles. For more information, see Database Engine Permission Basics. For more information about special
SQL Database roles, see Managing Databases and Logins in Azure SQL Database. A federated domain user that is
imported into a manage domain, must use the managed domain identity.

NOTE
Azure AD users are marked in the database metadata with type E (EXTERNAL_USER) and for groups with type X
(EXTERNAL_GROUPS). For more information, see sys.database_principals.

Connect to the user database or data warehouse by using SQL Server

Management Studio or SQL Server Data Tools
To confirm the Azure AD administrator is properly set up, connect to the master database using the Azure AD
administrator account. To provision an Azure AD-based contained database user (other than the server
administrator that owns the database), connect to the database with an Azure AD identity that has access to the
database.

IMPORTANT
Support for Azure Active Directory authentication is available with SQL Server 2016 Management Studio and SQL Server
Data Tools in Visual Studio 2015. The August 2016 release of SSMS also includes support for Active Directory Universal
Authentication, which allows administrators to require Multi-Factor Authentication using a phone call, text message, smart
cards with pin, or mobile app notification.

Using an Azure AD identity to connect using SQL Server Management

Studio or SQL Server Database Tools
The following procedures show you how to connect to a SQL database with an Azure AD identity using SQL Server
Management Studio or SQL Server Database Tools.
Active Directory integrated authentication
Use this method if you are logged in to Windows using your Azure Active Directory credentials from a federated
domain.
1. Start Management Studio or Data Tools and in the Connect to Server (or Connect to Database Engine)
dialog box, in the Authentication box, select Active Directory Integrated Authentication. No password
is needed or can be entered because your existing credentials will be presented for the connection.
2. Click the Options button, and on the Connection Properties page, in the Connect to database box, type
the name of the user database you want to connect to.

Active Directory password authentication

Use this method when connecting with an Azure AD principal name using the Azure AD managed domain. You can
also use it for federated account without access to the domain, for example when working remotely.
Use this method if you are logged in to Windows using credentials from a domain that is not federated with Azure,
or when using Azure AD authentication using Azure AD based on the initial or the client domain.
1. Start Management Studio or Data Tools and in the Connect to Server (or Connect to Database Engine)
dialog box, in the Authentication box, select Active Directory Password Authentication.
2. In the User name box, type your Azure Active Directory user name in the format username@domain.com.
This must be an account from the Azure Active Directory or an account from a domain federate with the Azure
Active Directory.
3. In the Password box, type your user password for the Azure Active Directory account or federated domain
account.

4. Click the Options button, and on the Connection Properties page, in the Connect to database box, type the
name of the user database you want to connect to. (See the graphic in the previous option.)

Using an Azure AD identity to connect from a client application

The following procedures show you how to connect to a SQL database with an Azure AD identity from a client
application.
Active Directory integrated authentication
To use integrated Windows authentication, your domain’s Active Directory must be federated with Azure Active
Directory. Your client application (or a service) connecting to the database must be running on a domain-joined
machine under a user’s domain credentials.
To connect to a database using integrated authentication and an Azure AD identity, the Authentication keyword in
the database connection string must be set to Active Directory Integrated. The following C# code sample uses ADO
.NET.

string ConnectionString =
@"Data Source=n9lxnyuzhv.database.windows.net; Authentication=Active Directory Integrated; Initial Catalog=testdb;";
SqlConnection conn = new SqlConnection(ConnectionString);
conn.Open();

The connection string keyword Integrated Security=True is not supported for connecting to Azure SQL Database.
When making an ODBC connection, you will need to remove spaces and set Authentication to
'ActiveDirectoryIntegrated'.
Active Directory password authentication
To connect to a database using integrated authentication and an Azure AD identity, the Authentication keyword
must be set to Active Directory Password. The connection string must contain User ID/UID and Password/PWD
keywords and values. The following C# code sample uses ADO .NET.
string ConnectionString =
@"Data Source=n9lxnyuzhv.database.windows.net; Authentication=Active Directory Password; Initial Catalog=testdb;
UID=bob@contoso.onmicrosoft.com; PWD=MyPassWord!";
SqlConnection conn = new SqlConnection(ConnectionString);
conn.Open();

Learn more about Azure AD authentication methods using the demo code samples available at Azure AD
Authentication GitHub Demo.

Azure AD token
This authentication method allows middle-tier services to connect to Azure SQL Database or Azure SQL Data
Warehouse by obtaining a token from Azure Active Directory (AAD). It enables sophisticated scenarios including
certificate-based authentication. You must complete four basic steps to use Azure AD token authentication:
1. Register your application with Azure Active Directory and get the client id for your code.
2. Create a database user representing the application. (Completed earlier in step 6.)
3. Create a certificate on the client computer runs the application.
4. Add the certificate as a key for your application.
Sample connection string:

string ConnectionString =@"Data Source=n9lxnyuzhv.database.windows.net; Initial Catalog=testdb;"

SqlConnection conn = new SqlConnection(ConnectionString);
connection.AccessToken = "Your JWT token"
conn.Open();

For more information, see SQL Server Security Blog.

sqlcmd
The following statements, connect using version 13.1 of sqlcmd, which is available from the Download Center.

sqlcmd -S Target_DB_or_DW.testsrv.database.windows.net -G
sqlcmd -S Target_DB_or_DW.testsrv.database.windows.net -U bob@contoso.com -P MyAADPassword -G -l 30

Next steps
For an overview of access and control in SQL Database, see SQL Database access and control.
For an overview of logins, users, and database roles in SQL Database, see Logins, users, and database roles.
For more information about database principals, see Principals.
For more information about database roles, see Database roles.
For more information about firewall rules in SQL Database, see SQL Database firewall rules.
For a tutorial using SQL Server authentication, see SQL authentication and authorization.
For a tutorial using Azure Active Directory authentication, see Azure AD authentication and authorization.
Always Encrypted: Protect sensitive data in SQL
Database and store your encryption keys in the
Windows certificate store
4/14/2017 • 12 min to read • Edit Online

This article shows you how to secure sensitive data in a SQL database with database encryption by using the
Always Encrypted Wizard in SQL Server Management Studio (SSMS). It also shows you how to store your
encryption keys in the Windows certificate store.
Always Encrypted is a new data encryption technology in Azure SQL Database and SQL Server that helps protect
sensitive data at rest on the server, during movement between client and server, and while the data is in use,
ensuring that sensitive data never appears as plaintext inside the database system. After you encrypt data, only
client applications or app servers that have access to the keys can access plaintext data. For detailed information,
see Always Encrypted (Database Engine).
After configuring the database to use Always Encrypted, you will create a client application in C# with Visual Studio
to work with the encrypted data.
Follow the steps in this article to learn how to set up Always Encrypted for an Azure SQL database. In this article,
you will learn how to perform the following tasks:
Use the Always Encrypted wizard in SSMS to create Always Encrypted Keys.
Create a Column Master Key (CMK).
Create a Column Encryption Key (CEK).
Create a database table and encrypt columns.
Create an application that inserts, selects, and displays data from the encrypted columns.

Create a blank SQL database

1. Sign in to the Azure portal.
2. Click New > Data + Storage > SQL Database.
3. Create a Blank database named Clinic on a new or existing server. For detailed instructions about creating
a database in the Azure portal, see Your first Azure SQL database.
You will need the connection string later in the tutorial. After the database is created, go to the new Clinic database
and copy the connection string. You can get the connection string at any time, but it's easy to copy it when you're
in the Azure portal.
1. Click SQL databases > Clinic > Show database connection strings.
2. Copy the connection string for ADO.NET.

Connect to the database with SSMS

Open SSMS and connect to the server with the Clinic database.
1. Open SSMS. (Click Connect > Database Engine to open the Connect to Server window if it is not open).
2. Enter your server name and credentials. The server name can be found on the SQL database blade and in
the connection string you copied earlier. Type the complete server name including database.windows.net.

If the New Firewall Rule window opens, sign in to Azure and let SSMS create a new firewall rule for you.

Create a table
In this section, you will create a table to hold patient data. This will be a normal table initially--you will configure
encryption in the next section.
1. Expand Databases.
2. Right-click the Clinic database and click New Query.
3. Paste the following Transact-SQL (T-SQL) into the new query window and Execute it.

CREATE TABLE [dbo].[Patients](

[PatientId] [int] IDENTITY(1,1),
[SSN] [char](11) NOT NULL,
[FirstName] [nvarchar](50) NULL,
[LastName] [nvarchar](50) NULL,
[MiddleName] [nvarchar](50) NULL,
[StreetAddress] [nvarchar](50) NULL,
[City] [nvarchar](50) NULL,
[ZipCode] [char](5) NULL,
[State] [char](2) NULL,
[BirthDate] [date] NOT NULL
PRIMARY KEY CLUSTERED ([PatientId] ASC) ON [PRIMARY] );
GO

Encrypt columns (configure Always Encrypted)

SSMS provides a wizard to easily configure Always Encrypted by setting up the CMK, CEK, and encrypted columns
for you.
1. Expand Databases > Clinic > Tables.
2. Right-click the Patients table and select Encrypt Columns to open the Always Encrypted wizard:
The Always Encrypted wizard includes the following sections: Column Selection, Master Key Configuration
(CMK), Validation, and Summary.
Column Selection
Click Next on the Introduction page to open the Column Selection page. On this page, you will select which
columns you want to encrypt, the type of encryption, and what column encryption key (CEK) to use.
Encrypt SSN and BirthDate information for each patient. The SSN column will use deterministic encryption, which
supports equality lookups, joins, and group by. The BirthDate column will use randomized encryption, which does
not support operations.
Set the Encryption Type for the SSN column to Deterministic and the BirthDate column to Randomized. Click
Next.
Master Key Configuration
The Master Key Configuration page is where you set up your CMK and select the key store provider where the
CMK will be stored. Currently, you can store a CMK in the Windows certificate store, Azure Key Vault, or a hardware
security module (HSM). This tutorial shows how to store your keys in the Windows certificate store.
Verify that Windows certificate store is selected and click Next.
Validation
You can encrypt the columns now or save a PowerShell script to run later. For this tutorial, select Proceed to
finish now and click Next.
Summary
Verify that the settings are all correct and click Finish to complete the setup for Always Encrypted.
Verify the wizard's actions
After the wizard is finished, your database is set up for Always Encrypted. The wizard performed the following
actions:
Created a CMK.
Created a CEK.
Configured the selected columns for encryption. Your Patients table currently has no data, but any existing data
in the selected columns is now encrypted.
You can verify the creation of the keys in SSMS by going to Clinic > Security > Always Encrypted Keys. You can
now see the new keys that the wizard generated for you.

Create a client application that works with the encrypted data

Now that Always Encrypted is set up, you can build an application that performs inserts and selects on the
encrypted columns. To successfully run the sample application, you must run it on the same computer where you
ran the Always Encrypted wizard. To run the application on another computer, you must deploy your Always
Encrypted certificates to the computer running the client app.

IMPORTANT
Your application must use SqlParameter objects when passing plaintext data to the server with Always Encrypted columns.
Passing literal values without using SqlParameter objects will result in an exception.

1. Open Visual Studio and create a new C# console application. Make sure your project is set to .NET Framework
4.6 or later.
2. Name the project AlwaysEncryptedConsoleApp and click OK.

Modify your connection string to enable Always Encrypted

This section explains how to enable Always Encrypted in your database connection string. You will modify the
console app you just created in the next section, "Always Encrypted sample console application."
To enable Always Encrypted, you need to add the Column Encryption Setting keyword to your connection string
and set it to Enabled.
You can set this directly in the connection string, or you can set it by using a SqlConnectionStringBuilder. The
sample application in the next section shows how to use SqlConnectionStringBuilder.

NOTE
This is the only change required in a client application specific to Always Encrypted. If you have an existing application that
stores its connection string externally (that is, in a config file), you might be able to enable Always Encrypted without
changing any code.

Enable Always Encrypted in the connection string

Add the following keyword to your connection string:

Column Encryption Setting=Enabled

Enable Always Encrypted with a SqlConnectionStringBuilder

The following code shows how to enable Always Encrypted by setting the
SqlConnectionStringBuilder.ColumnEncryptionSetting to Enabled.

// Instantiate a SqlConnectionStringBuilder.
SqlConnectionStringBuilder connStringBuilder =
new SqlConnectionStringBuilder("replace with your connection string");

// Enable Always Encrypted.

connStringBuilder.ColumnEncryptionSetting =
SqlConnectionColumnEncryptionSetting.Enabled;

Always Encrypted sample console application

This sample demonstrates how to:
Modify your connection string to enable Always Encrypted.
Insert data into the encrypted columns.
Select a record by filtering for a specific value in an encrypted column.
Replace the contents of Program.cs with the following code. Replace the connection string for the global
connectionString variable in the line directly above the Main method with your valid connection string from the
Azure portal. This is the only change you need to make to this code.
Run the app to see Always Encrypted in action.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Data;
using System.Data.SqlClient;

namespace AlwaysEncryptedConsoleApp
{
class Program
class Program
{
// Update this line with your Clinic database connection string from the Azure portal.
static string connectionString = @"Replace with your connection string";

static void Main(string[] args)

{
Console.WriteLine("Original connection string copied from the Azure portal:");
Console.WriteLine(connectionString);

// Create a SqlConnectionStringBuilder.
SqlConnectionStringBuilder connStringBuilder =
new SqlConnectionStringBuilder(connectionString);

// Enable Always Encrypted for the connection.

// This is the only change specific to Always Encrypted
connStringBuilder.ColumnEncryptionSetting =
SqlConnectionColumnEncryptionSetting.Enabled;

Console.WriteLine(Environment.NewLine + "Updated connection string with Always Encrypted enabled:");

Console.WriteLine(connStringBuilder.ConnectionString);

// Update the connection string with a password supplied at runtime.

Console.WriteLine(Environment.NewLine + "Enter server password:");
connStringBuilder.Password = Console.ReadLine();

// Assign the updated connection string to our global variable.

connectionString = connStringBuilder.ConnectionString;

// Delete all records to restart this demo app.

ResetPatientsTable();

// Add sample data to the Patients table.

Console.Write(Environment.NewLine + "Adding sample patient data to the database...");

InsertPatient(new Patient() {
SSN = "999-99-0001", FirstName = "Orlando", LastName = "Gee", BirthDate = DateTime.Parse("01/04/1964") });
InsertPatient(new Patient() {
SSN = "999-99-0002", FirstName = "Keith", LastName = "Harris", BirthDate = DateTime.Parse("06/20/1977") });
InsertPatient(new Patient() {
SSN = "999-99-0003", FirstName = "Donna", LastName = "Carreras", BirthDate = DateTime.Parse("02/09/1973") });
InsertPatient(new Patient() {
SSN = "999-99-0004", FirstName = "Janet", LastName = "Gates", BirthDate = DateTime.Parse("08/31/1985") });
InsertPatient(new Patient() {
SSN = "999-99-0005", FirstName = "Lucy", LastName = "Harrington", BirthDate = DateTime.Parse("05/06/1993") });

// Fetch and display all patients.

Console.WriteLine(Environment.NewLine + "All the records currently in the Patients table:");

foreach (Patient patient in SelectAllPatients())

{
Console.WriteLine(patient.FirstName + " " + patient.LastName + "\tSSN: " + patient.SSN + "\tBirthdate: " + patient.BirthDate);
}

// Get patients by SSN.

Console.WriteLine(Environment.NewLine + "Now let's locate records by searching the encrypted SSN column.");

string ssn;

// This very simple validation only checks that the user entered 11 characters.
// In production be sure to check all user input and use the best validation for your specific application.
do
{
Console.WriteLine("Please enter a valid SSN (ex. 123-45-6789):");
ssn = Console.ReadLine();
} while (ssn.Length != 11);
// The example allows duplicate SSN entries so we will return all records
// that match the provided value and store the results in selectedPatients.
Patient selectedPatient = SelectPatientBySSN(ssn);

// Check if any records were returned and display our query results.
if (selectedPatient != null)
{
Console.WriteLine("Patient found with SSN = " + ssn);
Console.WriteLine(selectedPatient.FirstName + " " + selectedPatient.LastName + "\tSSN: "
+ selectedPatient.SSN + "\tBirthdate: " + selectedPatient.BirthDate);
}
else
{
Console.WriteLine("No patients found with SSN = " + ssn);
}

Console.WriteLine("Press Enter to exit...");

Console.ReadLine();
}

static int InsertPatient(Patient newPatient)

{
int returnValue = 0;

string sqlCmdText = @"INSERT INTO [dbo].[Patients] ([SSN], [FirstName], [LastName], [BirthDate])

VALUES (@SSN, @FirstName, @LastName, @BirthDate);";

SqlCommand sqlCmd = new SqlCommand(sqlCmdText);

SqlParameter paramSSN = new SqlParameter(@"@SSN", newPatient.SSN);

paramSSN.DbType = DbType.AnsiStringFixedLength;
paramSSN.Direction = ParameterDirection.Input;
paramSSN.Size = 11;

SqlParameter paramFirstName = new SqlParameter(@"@FirstName", newPatient.FirstName);

paramFirstName.DbType = DbType.String;
paramFirstName.Direction = ParameterDirection.Input;

SqlParameter paramLastName = new SqlParameter(@"@LastName", newPatient.LastName);

paramLastName.DbType = DbType.String;
paramLastName.Direction = ParameterDirection.Input;

SqlParameter paramBirthDate = new SqlParameter(@"@BirthDate", newPatient.BirthDate);

paramBirthDate.SqlDbType = SqlDbType.Date;
paramBirthDate.Direction = ParameterDirection.Input;

sqlCmd.Parameters.Add(paramSSN);
sqlCmd.Parameters.Add(paramFirstName);
sqlCmd.Parameters.Add(paramLastName);
sqlCmd.Parameters.Add(paramBirthDate);

using (sqlCmd.Connection = new SqlConnection(connectionString))

{
try
{
sqlCmd.Connection.Open();
sqlCmd.ExecuteNonQuery();
}
catch (Exception ex)
{
returnValue = 1;
Console.WriteLine("The following error was encountered: ");
Console.WriteLine(ex.Message);
Console.WriteLine(Environment.NewLine + "Press Enter key to exit");
Console.ReadLine();
Environment.Exit(0);
}
}
return returnValue;
}

static List<Patient> SelectAllPatients()

{
List<Patient> patients = new List<Patient>();

SqlCommand sqlCmd = new SqlCommand(

"SELECT [SSN], [FirstName], [LastName], [BirthDate] FROM [dbo].[Patients]",
new SqlConnection(connectionString));

using (sqlCmd.Connection = new SqlConnection(connectionString))

{
try
{
sqlCmd.Connection.Open();
SqlDataReader reader = sqlCmd.ExecuteReader();

if (reader.HasRows)
{
while (reader.Read())
{
patients.Add(new Patient()
{
SSN = reader[0].ToString(),
FirstName = reader[1].ToString(),
LastName = reader["LastName"].ToString(),
BirthDate = (DateTime)reader["BirthDate"]
});
}
}
}
catch (Exception ex)
{
throw;
}
}

return patients;
}

static Patient SelectPatientBySSN(string ssn)

{
Patient patient = new Patient();

SqlCommand sqlCmd = new SqlCommand(

"SELECT [SSN], [FirstName], [LastName], [BirthDate] FROM [dbo].[Patients] WHERE [SSN]=@SSN",
new SqlConnection(connectionString));

SqlParameter paramSSN = new SqlParameter(@"@SSN", ssn);

paramSSN.DbType = DbType.AnsiStringFixedLength;
paramSSN.Direction = ParameterDirection.Input;
paramSSN.Size = 11;

sqlCmd.Parameters.Add(paramSSN);

using (sqlCmd.Connection = new SqlConnection(connectionString))

{
try
{
{
sqlCmd.Connection.Open();
SqlDataReader reader = sqlCmd.ExecuteReader();

if (reader.HasRows)
{
while (reader.Read())
{
patient = new Patient()
{
SSN = reader[0].ToString(),
FirstName = reader[1].ToString(),
LastName = reader["LastName"].ToString(),
BirthDate = (DateTime)reader["BirthDate"]
};
}
}
else
{
patient = null;
}
}
catch (Exception ex)
{
throw;
}
}
return patient;
}

// This method simply deletes all records in the Patients table to reset our demo.
static int ResetPatientsTable()
{
int returnValue = 0;

SqlCommand sqlCmd = new SqlCommand("DELETE FROM Patients");

using (sqlCmd.Connection = new SqlConnection(connectionString))
{
try
{
sqlCmd.Connection.Open();
sqlCmd.ExecuteNonQuery();

}
catch (Exception ex)
{
returnValue = 1;
}
}
return returnValue;
}
}

class Patient
{
public string SSN { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public DateTime BirthDate { get; set; }
}
}

Verify that the data is encrypted

You can quickly check that the actual data on the server is encrypted by querying the Patients data with SSMS.
(Use your current connection where the column encryption setting is not yet enabled.)
Run the following query on the Clinic database.

SELECT FirstName, LastName, SSN, BirthDate FROM Patients;

You can see that the encrypted columns do not contain any plaintext data.

To use SSMS to access the plaintext data, you can add the Column Encryption Setting=enabled parameter to
the connection.
1. In SSMS, right-click your server in Object Explorer, and then click Disconnect.
2. Click Connect > Database Engine to open the Connect to Server window, and then click Options.
3. Click Additional Connection Parameters and type Column Encryption Setting=enabled.

4. Run the following query on the Clinic database.

SELECT FirstName, LastName, SSN, BirthDate FROM Patients;

You can now see the plaintext data in the encrypted columns.
NOTE
If you connect with SSMS (or any client) from a different computer, it will not have access to the encryption keys and will not
be able to decrypt the data.

Next steps
After you create a database that uses Always Encrypted, you may want to do the following:
Run this sample from a different computer. It won't have access to the encryption keys, so it will not have access
to the plaintext data and will not run successfully.
Rotate and clean up your keys.
Migrate data that is already encrypted with Always Encrypted.
Deploy Always Encrypted certificates to other client machines (see the "Making Certificates Available to
Applications and Users" section).

Related information
Always Encrypted (client development)
Transparent Data Encryption
SQL Server Encryption
Always Encrypted Wizard
Always Encrypted Blog
Always Encrypted: Protect sensitive data in SQL
Database and store your encryption keys in Azure
Key Vault
4/14/2017 • 14 min to read • Edit Online

This article shows you how to secure sensitive data in a SQL database with data encryption using the Always
Encrypted Wizard in SQL Server Management Studio (SSMS). It also includes instructions that will show you how
to store each encryption key in Azure Key Vault.
Always Encrypted is a new data encryption technology in Azure SQL Database and SQL Server that helps protect
sensitive data at rest on the server, during movement between client and server, and while the data is in use.
Always Encrypted ensures that sensitive data never appears as plaintext inside the database system. After you
configure data encryption, only client applications or app servers that have access to the keys can access plaintext
data. For detailed information, see Always Encrypted (Database Engine).
After you configure the database to use Always Encrypted, you will create a client application in C# with Visual
Studio to work with the encrypted data.
Follow the steps in this article and learn how to set up Always Encrypted for an Azure SQL database. In this article
you will learn how to perform the following tasks:
Use the Always Encrypted wizard in SSMS to create Always Encrypted keys.
Create a column master key (CMK).
Create a column encryption key (CEK).
Create a database table and encrypt columns.
Create an application that inserts, selects, and displays data from the encrypted columns.

Prerequisites
For this tutorial, you'll need:
An Azure account and subscription. If you don't have one, sign up for a free trial.
SQL Server Management Studio version 13.0.700.242 or later.
.NET Framework 4.6 or later (on the client computer).
Visual Studio.
Azure PowerShell, version 1.0 or later. Type (Get-Module azure -ListAvailable).Version to see what version
of PowerShell you are running.

Enable your client application to access the SQL Database service

You must enable your client application to access the SQL Database service by setting up the required
authentication and acquiring the ClientId and Secret that you will need to authenticate your application in the
following code.
1. Open the Azure classic portal.
2. Select Active Directory and click the Active Directory instance that your application will use.
3. Click Applications, and then click ADD.
4. Type a name for your application (for example: myClientApp), select WEB APPLICATION, and click the arrow to
continue.
5. For the SIGN-ON URL and APP ID URI you can type a valid URL (for example, http://myClientApp) and
continue.
6. Click CONFIGURE.
7. Copy your CLIENT ID. (You will need this value in your code later.)
8. In the keys section, select 1 year from the Select duration drop-down list. (You will copy the key after you
save in step 13.)
9. Scroll down and click Add application.
10. Leave SHOW set to Microsoft Apps and select Microsoft Azure Service Management API. Click the
checkmark to continue.
11. Select Access Azure Service Management... from the Delegated Permissions drop-down list.
12. Click SAVE.
13. After the save finishes, copy the key value in the keys section. (You will need this value in your code later.)

Create a key vault to store your keys

Now that your client app is configured and you have your client ID, it's time to create a key vault and configure its
access policy so you and your application can access the vault's secrets (the Always Encrypted keys). The create,
get, list, sign, verify, wrapKey, and unwrapKey permissions are required for creating a new column master key and
for setting up encryption with SQL Server Management Studio.
You can quickly create a key vault by running the following script. For a detailed explanation of these cmdlets and
more information about creating and configuring a key vault, see Get started with Azure Key Vault.

$subscriptionName = '<your Azure subscription name>'

$userPrincipalName = '<username@domain.com>'
$clientId = '<client ID that you copied in step 7 above>'
$resourceGroupName = '<resource group name>'
$location = '<datacenter location>'
$vaultName = 'AeKeyVault'

Login-AzureRmAccount
$subscriptionId = (Get-AzureRmSubscription -SubscriptionName $subscriptionName).SubscriptionId
Set-AzureRmContext -SubscriptionId $subscriptionId

New-AzureRmResourceGroup -Name $resourceGroupName -Location $location

New-AzureRmKeyVault -VaultName $vaultName -ResourceGroupName $resourceGroupName -Location $location

Set-AzureRmKeyVaultAccessPolicy -VaultName $vaultName -ResourceGroupName $resourceGroupName -PermissionsToKeys

create,get,wrapKey,unwrapKey,sign,verify,list -UserPrincipalName $userPrincipalName
Set-AzureRmKeyVaultAccessPolicy -VaultName $vaultName -ResourceGroupName $resourceGroupName -ServicePrincipalName $clientId -
PermissionsToKeys get,wrapKey,unwrapKey,sign,verify,list

Create a blank SQL database

1. Sign in to the Azure portal.
2. Go to New > Data + Storage > SQL Database.
3. Create a Blank database named Clinic on a new or existing server. For detailed directions about how to
create a database in the Azure portal, see Your first Azure SQL database.
You will need the connection string later in the tutorial, so after you create the database, browse to the new Clinic
database and copy the connection string. You can get the connection string at any time, but it's easy to copy it in
the Azure portal.
1. Go to SQL databases > Clinic > Show database connection strings.
2. Copy the connection string for ADO.NET.

Connect to the database with SSMS

Open SSMS and connect to the server with the Clinic database.
1. Open SSMS. (Go to Connect > Database Engine to open the Connect to Server window if it isn't open.)
2. Enter your server name and credentials. The server name can be found on the SQL database blade and in
the connection string you copied earlier. Type the complete server name, including database.windows.net.

If the New Firewall Rule window opens, sign in to Azure and let SSMS create a new firewall rule for you.

Create a table
In this section, you will create a table to hold patient data. It's not initially encrypted--you will configure encryption
in the next section.
1. Expand Databases.
2. Right-click the Clinic database and click New Query.
3. Paste the following Transact-SQL (T-SQL) into the new query window and Execute it.

CREATE TABLE [dbo].[Patients](

Encrypt columns (configure Always Encrypted)

SSMS provides a wizard that helps you easily configure Always Encrypted by setting up the column master key,
column encryption key, and encrypted columns for you.
1. Expand Databases > Clinic > Tables.
2. Right-click the Patients table and select Encrypt Columns to open the Always Encrypted wizard:
The Always Encrypted wizard includes the following sections: Column Selection, Master Key Configuration,
Validation, and Summary.
Column Selection
Click Next on the Introduction page to open the Column Selection page. On this page, you will select which
columns you want to encrypt, the type of encryption, and what column encryption key (CEK) to use.
Encrypt SSN and BirthDate information for each patient. The SSN column will use deterministic encryption, which
supports equality lookups, joins, and group by. The BirthDate column will use randomized encryption, which does
not support operations.
Set the Encryption Type for the SSN column to Deterministic and the BirthDate column to Randomized. Click
Next.
Master Key Configuration
The Master Key Configuration page is where you set up your CMK and select the key store provider where the
CMK will be stored. Currently, you can store a CMK in the Windows certificate store, Azure Key Vault, or a hardware
security module (HSM).
This tutorial shows how to store your keys in Azure Key Vault.
1. Select Azure Key Vault.
2. Select the desired key vault from the drop-down list.
3. Click Next.
Validation
You can encrypt the columns now or save a PowerShell script to run later. For this tutorial, select Proceed to
finish now and click Next.
Summary
Verify that the settings are all correct and click Finish to complete the setup for Always Encrypted.
Verify the wizard's actions
After the wizard is finished, your database is set up for Always Encrypted. The wizard performed the following
actions:
Created a column master key and stored it in Azure Key Vault.
Created a column encryption key and stored it in Azure Key Vault.
Configured the selected columns for encryption. The Patients table currently has no data, but any existing data
in the selected columns is now encrypted.
You can verify the creation of the keys in SSMS by expanding Clinic > Security > Always Encrypted Keys.

Create a client application that works with the encrypted data

Now that Always Encrypted is set up, you can build an application that performs inserts and selects on the
encrypted columns.

1. Open Visual Studio and create a new C# Console Application (Visual Studio 2015 and earlier) or Console
App (.NET Framework) (Visual Studio 2017 and later). Make sure your project is set to .NET Framework 4.6
or later.
2. Name the project AlwaysEncryptedConsoleAKVApp and click OK.
3. Install the following NuGet packages by going to Tools > NuGet Package Manager > Package Manager
Console.
Run these two lines of code in the Package Manager Console.

Install-Package Microsoft.SqlServer.Management.AlwaysEncrypted.AzureKeyVaultProvider
Install-Package Microsoft.IdentityModel.Clients.ActiveDirectory

Modify your connection string to enable Always Encrypted

This section explains how to enable Always Encrypted in your database connection string.
To enable Always Encrypted, you need to add the Column Encryption Setting keyword to your connection string
and set it to Enabled.
You can set this directly in the connection string, or you can set it by using SqlConnectionStringBuilder. The sample
application in the next section shows how to use SqlConnectionStringBuilder.
Enable Always Encrypted in the connection string
Add the following keyword to your connection string.

Column Encryption Setting=Enabled

Enable Always Encrypted with SqlConnectionStringBuilder

The following code shows how to enable Always Encrypted by setting
SqlConnectionStringBuilder.ColumnEncryptionSetting to Enabled.

// Instantiate a SqlConnectionStringBuilder.
SqlConnectionStringBuilder connStringBuilder =
new SqlConnectionStringBuilder("replace with your connection string");

// Enable Always Encrypted.

connStringBuilder.ColumnEncryptionSetting =
SqlConnectionColumnEncryptionSetting.Enabled;

Register the Azure Key Vault provider

The following code shows how to register the Azure Key Vault provider with the ADO.NET driver.

private static ClientCredential _clientCredential;

static void InitializeAzureKeyVaultProvider()

{
_clientCredential = new ClientCredential(clientId, clientSecret);

SqlColumnEncryptionAzureKeyVaultProvider azureKeyVaultProvider =
new SqlColumnEncryptionAzureKeyVaultProvider(GetToken);

Dictionary<string, SqlColumnEncryptionKeyStoreProvider> providers =

new Dictionary<string, SqlColumnEncryptionKeyStoreProvider>();

providers.Add(SqlColumnEncryptionAzureKeyVaultProvider.ProviderName, azureKeyVaultProvider);
SqlConnection.RegisterColumnEncryptionKeyStoreProviders(providers);
}
Always Encrypted sample console application
This sample demonstrates how to:
Modify your connection string to enable Always Encrypted.
Register Azure Key Vault as the application's key store provider.
Insert data into the encrypted columns.
Select a record by filtering for a specific value in an encrypted column.
Replace the contents of Program.cs with the following code. Replace the connection string for the global
connectionString variable in the line that directly precedes the Main method with your valid connection string from
the Azure portal. This is the only change you need to make to this code.
Run the app to see Always Encrypted in action.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Data;
using System.Data.SqlClient;
using Microsoft.IdentityModel.Clients.ActiveDirectory;
using Microsoft.SqlServer.Management.AlwaysEncrypted.AzureKeyVaultProvider;

namespace AlwaysEncryptedConsoleAKVApp
{
class Program
{
// Update this line with your Clinic database connection string from the Azure portal.
static string connectionString = @"<connection string from the portal>";
static string clientId = @"<client id from step 7 above>";
static string clientSecret = "<key from step 13 above>";

static void Main(string[] args)

{
InitializeAzureKeyVaultProvider();

Console.WriteLine("Signed in as: " + _clientCredential.ClientId);

Console.WriteLine("Original connection string copied from the Azure portal:");

Console.WriteLine(connectionString);

// Create a SqlConnectionStringBuilder.
SqlConnectionStringBuilder connStringBuilder =
new SqlConnectionStringBuilder(connectionString);

// Enable Always Encrypted for the connection.

// This is the only change specific to Always Encrypted
connStringBuilder.ColumnEncryptionSetting =
SqlConnectionColumnEncryptionSetting.Enabled;

Console.WriteLine(Environment.NewLine + "Updated connection string with Always Encrypted enabled:");

Console.WriteLine(connStringBuilder.ConnectionString);

// Update the connection string with a password supplied at runtime.

Console.WriteLine(Environment.NewLine + "Enter server password:");
connStringBuilder.Password = Console.ReadLine();

// Assign the updated connection string to our global variable.

connectionString = connStringBuilder.ConnectionString;
// Delete all records to restart this demo app.
ResetPatientsTable();

// Add sample data to the Patients table.

Console.Write(Environment.NewLine + "Adding sample patient data to the database...");

InsertPatient(new Patient()
{
SSN = "999-99-0001",
FirstName = "Orlando",
LastName = "Gee",
BirthDate = DateTime.Parse("01/04/1964")
});
InsertPatient(new Patient()
{
SSN = "999-99-0002",
FirstName = "Keith",
LastName = "Harris",
BirthDate = DateTime.Parse("06/20/1977")
});
InsertPatient(new Patient()
{
SSN = "999-99-0003",
FirstName = "Donna",
LastName = "Carreras",
BirthDate = DateTime.Parse("02/09/1973")
});
InsertPatient(new Patient()
{
SSN = "999-99-0004",
FirstName = "Janet",
LastName = "Gates",
BirthDate = DateTime.Parse("08/31/1985")
});
InsertPatient(new Patient()
{
SSN = "999-99-0005",
FirstName = "Lucy",
LastName = "Harrington",
BirthDate = DateTime.Parse("05/06/1993")
});

// Fetch and display all patients.

Console.WriteLine(Environment.NewLine + "All the records currently in the Patients table:");

foreach (Patient patient in SelectAllPatients())

{
Console.WriteLine(patient.FirstName + " " + patient.LastName + "\tSSN: " + patient.SSN + "\tBirthdate: " + patient.BirthDate);
}

// Get patients by SSN.

Console.WriteLine(Environment.NewLine + "Now lets locate records by searching the encrypted SSN column.");

string ssn;

// The example allows duplicate SSN entries so we will return all records
// that match the provided value and store the results in selectedPatients.
Patient selectedPatient = SelectPatientBySSN(ssn);

// Check if any records were returned and display our query results.
// Check if any records were returned and display our query results.
if (selectedPatient != null)
{
Console.WriteLine("Patient found with SSN = " + ssn);
Console.WriteLine(selectedPatient.FirstName + " " + selectedPatient.LastName + "\tSSN: "
+ selectedPatient.SSN + "\tBirthdate: " + selectedPatient.BirthDate);
}
else
{
Console.WriteLine("No patients found with SSN = " + ssn);
}

Console.WriteLine("Press Enter to exit...");

Console.ReadLine();
}

private static ClientCredential _clientCredential;

static void InitializeAzureKeyVaultProvider()

{

_clientCredential = new ClientCredential(clientId, clientSecret);

SqlColumnEncryptionAzureKeyVaultProvider azureKeyVaultProvider =
new SqlColumnEncryptionAzureKeyVaultProvider(GetToken);

Dictionary<string, SqlColumnEncryptionKeyStoreProvider> providers =

new Dictionary<string, SqlColumnEncryptionKeyStoreProvider>();

providers.Add(SqlColumnEncryptionAzureKeyVaultProvider.ProviderName, azureKeyVaultProvider);
SqlConnection.RegisterColumnEncryptionKeyStoreProviders(providers);
}

public async static Task<string> GetToken(string authority, string resource, string scope)
{
var authContext = new AuthenticationContext(authority);
AuthenticationResult result = await authContext.AcquireTokenAsync(resource, _clientCredential);

if (result == null)
throw new InvalidOperationException("Failed to obtain the access token");
return result.AccessToken;
}

static int InsertPatient(Patient newPatient)

{
int returnValue = 0;

string sqlCmdText = @"INSERT INTO [dbo].[Patients] ([SSN], [FirstName], [LastName], [BirthDate])

VALUES (@SSN, @FirstName, @LastName, @BirthDate);";

SqlCommand sqlCmd = new SqlCommand(sqlCmdText);

SqlParameter paramSSN = new SqlParameter(@"@SSN", newPatient.SSN);

paramSSN.DbType = DbType.AnsiStringFixedLength;
paramSSN.Direction = ParameterDirection.Input;
paramSSN.Size = 11;

SqlParameter paramFirstName = new SqlParameter(@"@FirstName", newPatient.FirstName);

paramFirstName.DbType = DbType.String;
paramFirstName.Direction = ParameterDirection.Input;

SqlParameter paramLastName = new SqlParameter(@"@LastName", newPatient.LastName);

paramLastName.DbType = DbType.String;
paramLastName.Direction = ParameterDirection.Input;

SqlParameter paramBirthDate = new SqlParameter(@"@BirthDate", newPatient.BirthDate);

paramBirthDate.SqlDbType = SqlDbType.Date;
paramBirthDate.Direction = ParameterDirection.Input;
paramBirthDate.Direction = ParameterDirection.Input;

sqlCmd.Parameters.Add(paramSSN);
sqlCmd.Parameters.Add(paramFirstName);
sqlCmd.Parameters.Add(paramLastName);
sqlCmd.Parameters.Add(paramBirthDate);

using (sqlCmd.Connection = new SqlConnection(connectionString))

static List<Patient> SelectAllPatients()

{
List<Patient> patients = new List<Patient>();

SqlCommand sqlCmd = new SqlCommand(

"SELECT [SSN], [FirstName], [LastName], [BirthDate] FROM [dbo].[Patients]",
new SqlConnection(connectionString));

using (sqlCmd.Connection = new SqlConnection(connectionString))

{
try
{
sqlCmd.Connection.Open();
SqlDataReader reader = sqlCmd.ExecuteReader();

return patients;
}
static Patient SelectPatientBySSN(string ssn)
{
Patient patient = new Patient();

SqlCommand sqlCmd = new SqlCommand(

"SELECT [SSN], [FirstName], [LastName], [BirthDate] FROM [dbo].[Patients] WHERE [SSN]=@SSN",
new SqlConnection(connectionString));

SqlParameter paramSSN = new SqlParameter(@"@SSN", ssn);

paramSSN.DbType = DbType.AnsiStringFixedLength;
paramSSN.Direction = ParameterDirection.Input;
paramSSN.Size = 11;

sqlCmd.Parameters.Add(paramSSN);

using (sqlCmd.Connection = new SqlConnection(connectionString))

{
try
{
sqlCmd.Connection.Open();
SqlDataReader reader = sqlCmd.ExecuteReader();

// This method simply deletes all records in the Patients table to reset our demo.
static int ResetPatientsTable()
{
int returnValue = 0;

SqlCommand sqlCmd = new SqlCommand("DELETE FROM Patients");

using (sqlCmd.Connection = new SqlConnection(connectionString))
{
try
{
sqlCmd.Connection.Open();
sqlCmd.ExecuteNonQuery();

}
catch (Exception ex)
{
returnValue = 1;
}
}
}
return returnValue;
}
}

class Patient
{
public string SSN { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public DateTime BirthDate { get; set; }
}
}

Verify that the data is encrypted

You can quickly check that the actual data on the server is encrypted by querying the Patients data with SSMS
(using your current connection where Column Encryption Setting is not yet enabled).
Run the following query on the Clinic database.

SELECT FirstName, LastName, SSN, BirthDate FROM Patients;

You can see that the encrypted columns do not contain any plaintext data.

To use SSMS to access the plaintext data, you can add the Column Encryption Setting=enabled parameter to the
connection.
1. In SSMS, right-click your server in Object Explorer and choose Disconnect.
2. Click Connect > Database Engine to open the Connect to Server window and click Options.
3. Click Additional Connection Parameters and type Column Encryption Setting=enabled.
4. Run the following query on the Clinic database.

SELECT FirstName, LastName, SSN, BirthDate FROM Patients;

You can now see the plaintext data in the encrypted columns.

Next steps
After you create a database that uses Always Encrypted, you may want to do the following:
Rotate and clean up your keys.
Migrate data that is already encrypted with Always Encrypted.

Related information
Always Encrypted (client development)
Transparent data encryption
SQL Server encryption
Always Encrypted wizard
Always Encrypted blog
Get started with SQL Database dynamic data masking
with the Azure Portal
4/14/2017 • 1 min to read • Edit Online

This topic shows you how to implement dynamic data masking with the Azure portal. You can also implement
dynamic data masking using Azure SQL Database cmdlets or the REST API.

Set up dynamic data masking for your database using the Azure Portal
1. Launch the Azure Portal at https://portal.azure.com.
2. Navigate to the settings blade of the database that includes the sensitive data you want to mask.
3. Click the Dynamic Data Masking tile which launches the Dynamic Data Masking configuration blade.
Alternatively, you can scroll down to the Operations section and click Dynamic Data Masking.

4. In the Dynamic Data Masking configuration blade you may see some database columns that the
recommendations engine has flagged for masking. In order to accept the recommendations, just click Add
Mask for one or more columns and a mask will be created based on the default type for this column. You
can change the masking function by clicking on the masking rule and editing the masking field format to a
different format of your choice. Be sure to click Save to save your settings.
5. To add a mask for any column in your database, at the top of the Dynamic Data Masking configuration
blade click Add Mask to open the Add Masking Rule configuration blade
6. Select the Schema, Table and Column to define the designated field that will be masked.
7. Choose a Masking Field Format from the list of sensitive data masking categories.

8. Click Save in the data masking rule blade to update the set of masking rules in the dynamic data masking
policy.
9. Type the SQL users or AAD identities that should be excluded from masking, and have access to the
unmasked sensitive data. This should be a semicolon-separated list of users. Note that users with
administrator privileges always have access to the original unmasked data.

TIP
To make it so the application layer can display sensitive data for application privileged users, add the SQL user or AAD
identity the application uses to query the database. It is highly recommended that this list contain a minimal number
of privileged users to minimize exposure of the sensitive data.

10. Click Save in the data masking configuration blade to save the new or updated masking policy.

Next steps
For an overview of dynamic data masking, see dynamic data masking.
You can also implement dynamic data masking using Azure SQL Database cmdlets or the REST API.
How to restore a single table from an Azure SQL
Database backup
4/10/2017 • 2 min to read • Edit Online

You may encounter a situation in which you accidentally modified some data in a SQL database and now you want
to recover the single affected table. This article describes how to restore a single table in a database from one of the
SQL Database automatic backups.

Preparation steps: Rename the table and restore a copy of the database
1. Identify the table in your Azure SQL database that you want to replace with the restored copy. Use Microsoft
SQL Management Studio to rename the table. For example, rename the table as <table name>_old.

NOTE
To avoid being blocked, make sure that there's no activity running on the table that you are renaming. If you
encounter issues, make sure that perform this procedure during a maintenance window.

2. Restore a backup of your database to a point in time that you want to recover to using the Point-In_Time
Restore steps.

NOTE
The name of the restored database will be in the DBName+TimeStamp format; for example,
Adventureworks2012_2016-01-01T22-12Z. This step doesn't overwrite the existing database name on the server.
This is a safety measure, and it's intended to allow you to verify the restored database before they drop their current
database and rename the restored database for production use.

Copying the table from the restored database by using the SQL
Database Migration tool
1. Download and install the SQL Database Migration Wizard.
2. Open the SQL Database Migration Wizard, on the Select Process page, select Database under
Analyze/Migrate, and then click Next.
3. In the Connect to Server dialog box, apply the following settings:
Server name: Your SQL server
Authentication: SQL Server Authentication
Login: Your login
Password: Your password
Database: Master DB (List all databases)

NOTE
By default the wizard saves your login information. If you don't want it to, select Forget Login Information.
4. In the Select Source dialog box, select the restored database name from the Preparation steps section as
your source, and then click Next.
5. In the Choose Objects dialog box, select the Select specific database objects option, and then select the
table(or tables) that you want to migrate to the target server.
6. On the Script Wizard Summary page, click Yes when you’re prompted about whether you’re ready to
generate a SQL script. You also have the option to save the TSQL Script for later use.
7. On the Results Summary page, click Next.

8. On the Setup Target Server Connection page, click Connect to Server, and then enter the details as
follows:
Server Name: Target server instance
Authentication: SQL Server authentication. Enter your login credentials.
Database: Master DB (List all databases). This option lists all the databases on the target server.
9. Click Connect, select the target database that you want to move the table to, and then click Next. This should
finish running the previously generated script, and you should see the newly moved table copied to the target
database.

Verification step
Query and test the newly copied table to make sure that the data is intact. Upon confirmation, you can drop the
renamed table form Preparation steps section. (for example, <table name>_old).

Next steps
SQL Database automatic backups
Configure and restore from Azure SQL Database
long-term backup retention
4/12/2017 • 6 min to read • Edit Online

You can configure the Azure Recovery Services vault to store Azure SQL database backups and then recover a
database using backups retained in the vault using the Azure portal or PowerShell.

Azure portal
The following sections show you how to use the Azure portal to configure the Azure Recovery Services vault,
view backups in the vault, and restore from the vault.
Configure the vault, register the server, and select databases
You configure an Azure Recovery Services vault to retain automated backups for a period longer than the
retention period for your service tier.

TIP
To delete backups in long-term backup retention, see Configure and use long-term backup retention.

1. Open the SQL Server page for your server.

2. Click Long-term backup retention.
3. On the Long-term backup retention page for your server, review and accept the preview terms (unless
you have already done so - or this feature is no longer in preview).

4. To configure long-term backup retention, select that database in the grid and then click Configure on the
toolbar.
5. On the Configure page, click Configure required settings under Recovery service vault.

6. On the Recovery services vault page, select an existing vault, if any. Otherwise, if no recovery services
vault found for your subscription, click to exit the flow and create a recovery services vault.

7. On the Recovery Services vaults page, click Add.

8. On the Recovery Services vault page, provide a valid name for the Recovery Services vault.

9. Select your subscription and resource group, and then select the location for the vault. When done, click
Create.
IMPORTANT
The vault must be located in the same region as the Azure SQL logical server, and must use the same resource
group as the logical server.

10. After the new vault is created, execute the necessary steps to return to the Recovery services vault page.
11. On the Recovery services vault page, click the vault and then click Select.

12. On the Configure page, provide a valid name for the new retention policy, modify the default retention
policy as appropriate, and then click OK.
13. On the Long-term backup retention page for your database, click Save and then click OK to apply the
long-term backup retention policy to all selected databases.

14. Click Save to enable long-term backup retention using this new policy to the Azure Recovery Services
vault that you configured.

IMPORTANT
Once configured, backups show up in the vault within next seven days. Do not continue this tutorial until backups show
up in the vault.

View backups in long-term retention using Azure portal

View information about your database backups in long-term backup retention.
1. In the Azure portal, open your Azure Recovery Services vault for your database backups (go to All
resources and select it from the list of resources for your subscription) to view the amount of storage
used by your database backups in the vault.

2. Open the SQL database page for your database.

3. On the toolbar, click Restore.

4. On the Restore page, click Long-term.

5. Under Azure vault backups, click Select a backup to view the available database backups in long-term
backup retention.
Restore a database from a backup in long-term backup retention using the Azure portal
You restore the database to a new database from a backup in the Azure Recovery Services vault.
1. On the Azure vault backups page, click the backup to restore and then click Select.
2. In the Database name text box, provide the name for the restored database.

3. Click OK to restore your database from the backup in the vault to the new database.
4. On the toolbar, click the notification icon to view the status of the restore job.
5. When the restore job is completed, open the SQL databases page to view the newly restored database.

NOTE
From here, you can connect to the restored database using SQL Server Management Studio to perform needed tasks,
such as to extract a bit of data from the restored database to copy into the existing database or to delete the existing
database and rename the restored database to the existing database name.

PowerShell
The following sections show you how to use PowerShell to configure the Azure Recovery Services vault, view
backups in the vault, and restore from the vault.
Create a recovery services vault
Use the New-AzureRmRecoveryServicesVault to create a recovery services vault.

IMPORTANT
The vault must be located in the same region as the Azure SQL logical server, and must use the same resource group as
the logical server.

# Create a recovery services vault

#$resourceGroupName = "{resource-group-name}"
#$serverName = "{server-name}"
$serverLocation = (Get-AzureRmSqlServer -ServerName $serverName -ResourceGroupName $resourceGroupName).Location
$recoveryServiceVaultName = "{new-vault-name}"

$vault = New-AzureRmRecoveryServicesVault -Name $recoveryServiceVaultName -ResourceGroupName $ResourceGroupName -Location

$serverLocation
Set-AzureRmRecoveryServicesBackupProperties -BackupStorageRedundancy LocallyRedundant -Vault $vault

Set your server to use the recovery vault for its long-term retention backups
Use the Set-AzureRmSqlServerBackupLongTermRetentionVault cmdlet to associate a previously created
recovery services vault with a specific Azure SQL server.

# Set your server to use the vault to for long-term backup retention

Set-AzureRmSqlServerBackupLongTermRetentionVault -ResourceGroupName $resourceGroupName -ServerName $serverName -ResourceId

$vault.Id

Create a retention policy

A retention policy is where you set how long to keep a database backup. Use the Get-
AzureRmRecoveryServicesBackupRetentionPolicyObject cmdlet to get the default retention policy to use as the
template for creating policies. In this template, the retention period is set for 2 years. Next, run the New-
AzureRmRecoveryServicesBackupProtectionPolicy to finally create the policy.

NOTE
Some cmdlets require that you set the vault context before running (Set-AzureRmRecoveryServicesVaultContext) so you
see this cmdlet in a few related snippets. You set the context because the policy is part of the vault. You can create
multiple retention policies for each vault and then apply the desired policy to specific databases.

# Retrieve the default retention policy for the AzureSQLDatabase workload type
$retentionPolicy = Get-AzureRmRecoveryServicesBackupRetentionPolicyObject -WorkloadType AzureSQLDatabase

# Set the retention value to two years (you can set to any time between 1 week and 10 years)
$retentionPolicy.RetentionDurationType = "Years"
$retentionPolicy.RetentionCount = 2
$retentionPolicyName = "my2YearRetentionPolicy"

# Set the vault context to the vault you are creating the policy for
Set-AzureRmRecoveryServicesVaultContext -Vault $vault

# Create the new policy

$policy = New-AzureRmRecoveryServicesBackupProtectionPolicy -name $retentionPolicyName -WorkloadType AzureSQLDatabase -
retentionPolicy $retentionPolicy
$policy

Configure a database to use the previously defined retention policy

Use the Set-AzureRmSqlDatabaseBackupLongTermRetentionPolicy cmdlet to apply the new policy to a specific
database.

# Enable long-term retention for a specific SQL database

$policyState = "enabled"
Set-AzureRmSqlDatabaseBackupLongTermRetentionPolicy -ResourceGroupName $resourceGroupName -ServerName $serverName -
DatabaseName $databaseName -State $policyState -ResourceId $policy.Id

IMPORTANT
Once configured, backups show up in the vault within next seven days. Continue this tutorial after backups show up in
the vault.

View backup info, and backups in long-term retention

View information about your database backups in long-term backup retention.
Use the following cmdlets to view backup information:
Get-AzureRmRecoveryServicesBackupContainer
Get-AzureRmRecoveryServicesBackupItem
Get-AzureRmRecoveryServicesBackupRecoveryPoint

#$resourceGroupName = "{resource-group-name}"
#$serverName = "{server-name}"
$databaseNeedingRestore = $databaseName

# Set the vault context to the vault we want to restore from

#$vault = Get-AzureRmRecoveryServicesVault -ResourceGroupName $resourceGroupName
Set-AzureRmRecoveryServicesVaultContext -Vault $vault

# the following commands find the container associated with the server 'myserver' under resource group 'myresourcegroup'
$container = Get-AzureRmRecoveryServicesBackupContainer -ContainerType AzureSQL -FriendlyName $vault.Name

# Get the long-term retention metadata associated with a specific database

$item = Get-AzureRmRecoveryServicesBackupItem -Container $container -WorkloadType AzureSQLDatabase -Name
$databaseNeedingRestore

# Get all available backups for the previously indicated database

# Optionally, set the -StartDate and -EndDate parameters to return backups within a specific time period
$availableBackups = Get-AzureRmRecoveryServicesBackupRecoveryPoint -Item $item
$availableBackups

Restore a database from a backup in long-term backup retention

Restoring from long-term backup retention uses the Restore-AzureRmSqlDatabase cmdlet.

# Restore the most recent backup: $availableBackups[0]

#$resourceGroupName = "{resource-group-name}"
#$serverName = "{server-name}"
$restoredDatabaseName = "{new-database-name}"
$edition = "Basic"
$performanceLevel = "Basic"

$restoredDb = Restore-AzureRmSqlDatabase -FromLongTermRetentionBackup -ResourceId $availableBackups[0].Id -ResourceGroupName

$resourceGroupName `
-ServerName $serverName -TargetDatabaseName $restoredDatabaseName -Edition $edition -ServiceObjectiveName $performanceLevel
$restoredDb

Next steps
To learn about service-generated automatic backups, see automatic backups
To learn about long-term backup retention, see long-term backup retention
To learn about restoring from backups, see restore from backup
Configure active geo-replication for Azure SQL
Database in the Azure portal and initiate failover
4/14/2017 • 3 min to read • Edit Online

This article shows you how to configure active geo-replication for SQL Database in the Azure portal and to initiate
failover.
To initiate failover with the Azure portal, see Initiate a planned or unplanned failover for Azure SQL Database with
the Azure portal.

NOTE
Active geo-replication (readable secondaries) is now available for all databases in all service tiers. In April 2017, the non-
readable secondary type will be retired, and existing non-readable databases will automatically be upgraded to readable
secondaries.

To configure active geo-replication by using the Azure portal, you need the following resource:
An Azure SQL database: The primary database that you want to replicate to a different geographical region.

NOTE
Active geo-replication must be between databases in the same subscription.

Add a secondary database

The following steps create a new secondary database in a geo-replication partnership.
To add a secondary database, you must be the subscription owner or co-owner.
The secondary database has the same name as the primary database and has, by default, the same service level.
The secondary database can be a single database or a database in an elastic pool. For more information, see
Service tiers. After the secondary is created and seeded, data begins replicating from the primary database to the
new secondary database.

NOTE
If the partner database already exists (for example, as a result of terminating a previous geo-replication relationship) the
command fails.

1. In the Azure portal, browse to the database that you want to set up for geo-replication.
2. On the SQL database page, select Geo-Replication, and then select the region to create the secondary
database. You can select any region other than the region hosting the primary database, but we
recommend the paired region.
3. Select or configure the server and pricing tier for the secondary database.

4. Optionally, you can add a secondary database to an elastic pool. To create the secondary database in a pool,
click elastic pool and select a pool on the target server. A pool must already exist on the target server. This
workflow does not create a pool.
5. Click Create to add the secondary.
6. The secondary database is created and the seeding process begins.

7. When the seeding process is complete, the secondary database displays its status.

Initiate a failover
The secondary database can be switched to become the primary.
1. In the Azure portal, browse to the primary database in the Geo-Replication partnership.
2. On the SQL Database blade, select All settings > Geo-Replication.
3. In the SECONDARIES list, select the database you want to become the new primary and click Failover.
4. Click Yes to begin the failover.
The command immediately switches the secondary database into the primary role.
There is a short period during which both databases are unavailable (on the order of 0 to 25 seconds) while the
roles are switched. If the primary database has multiple secondary databases, the command automatically
reconfigures the other secondaries to connect to the new primary. The entire operation should take less than a
minute to complete under normal circumstances.

NOTE
If the primary is online and committing transactions when the command is issued some data loss may occur.

Remove secondary database

This operation permanently terminates the replication to the secondary database, and changes the role of the
secondary to a regular read-write database. If the connectivity to the secondary database is broken, the command
succeeds but the secondary does not become read-write until after connectivity is restored.
1. In the Azure portal, browse to the primary database in the geo-replication partnership.
2. On the SQL database page, select Geo-Replication.
3. In the SECONDARIES list, select the database you want to remove from the geo-replication partnership.
4. Click Stop Replication.
5. A confirmation window opens. Click Yes to remove the database from the geo-replication partnership. (Set it
to a read-write database not part of any replication.)

Next steps
To learn more about active geo-replication, see Active geo-replication.
For a business continuity overview and scenarios, see Business continuity overview.
Configure active geo-replication for Azure SQL
Database with Transact-SQL
2/16/2017 • 7 min to read • Edit Online

This article shows you how to configure active geo-replication for an Azure SQL Database with Transact-SQL.
To initiate failover using Transact-SQL, see Initiate a planned or unplanned failover for Azure SQL Database with
Transact-SQL.

NOTE
Active geo-replication (readable secondaries) is now available for all databases in all service tiers. In April 2017 the non-
readable secondary type will be retired and existing non-readable databases will automatically be upgraded to readable
secondaries.

To configure Active Geo-Replication using Transact-SQL, you need the following:

An Azure subscription.
A logical Azure SQL Database server and a SQL database - The primary database that you want to replicate.
One or more logical Azure SQL Database servers - The logical servers that will be the partner servers in which
you will create secondary databases.
A login that is DBManager on the primary, have db_ownership of the local database that you will geo-replicate,
and be DBManager on the partner server(s) to which you will configure Geo-Replication.
SQL Server Management Studio (SSMS)

IMPORTANT
It is recommended that you always use the latest version of Management Studio to remain synchronized with updates to
Microsoft Azure and SQL Database. Update SQL Server Management Studio.

Add secondary database

You can use the ALTER DATABASE statement to create a geo-replicated secondary database on a partner server.
You execute this statement on the master database of the server containing the database to be replicated. The geo-
replicated database (the "primary database") will have the same name as the database being replicated and will, by
default, have the same service level as the primary database. The secondary database can be readable or non-
readable, and can be a single database or in an elastic pool. For more information, see ALTER DATABASE (Transact-
SQL) and Service Tiers. After the secondary database is created and seeded, data will begin replicating
asynchronously from the primary database. The steps below describe how to configure Geo-Replication using
Management Studio. Steps to create non-readable and readable secondaries, either as a single database or in an
elastic pool, are provided.

NOTE
If a database exists on the specified partner server with the same name as the primary database the command will fail.

Add non-readable secondary (single database )

Use the following steps to create a non-readable secondary as a single database.
1. Using version 13.0.600.65 or later of SQL Server Management Studio.

IMPORTANT
Download the latest version of SQL Server Management Studio. It is recommended that you always use the latest
version of Management Studio to remain in sync with updates to the Azure portal.

2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following ALTER DATABASE statement to make a local database into a Geo-Replication primary
with a non-readable secondary database on MySecondaryServer1 where MySecondaryServer1 is your
friendly server name.

ALTER DATABASE <MyDB>

ADD SECONDARY ON SERVER <MySecondaryServer1> WITH (ALLOW_CONNECTIONS = NO);

4. Click Execute to run the query.

Add readable secondary (single database )
Use the following steps to create a readable secondary as a single database.
1. In Management Studio, connect to your Azure SQL Database logical server.
2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following ALTER DATABASE statement to make a local database into a Geo-Replication primary
with a readable secondary database on a secondary server.

ALTER DATABASE <MyDB>

ADD SECONDARY ON SERVER <MySecondaryServer2> WITH (ALLOW_CONNECTIONS = ALL);

4. Click Execute to run the query.

Add non-readable secondary (elastic pool)
Use the following steps to create a non-readable secondary in an elastic pool.
1. In Management Studio, connect to your Azure SQL Database logical server.
2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following ALTER DATABASE statement to make a local database into a Geo-Replication primary
with a non-readable secondary database on a secondary server in an elastic pool.

ALTER DATABASE <MyDB>

ADD SECONDARY ON SERVER <MySecondaryServer3> WITH (ALLOW_CONNECTIONS = NO
, SERVICE_OBJECTIVE = ELASTIC_POOL (name = MyElasticPool1));

4. Click Execute to run the query.

Add readable secondary (elastic pool)
Use the following steps to create a readable secondary in an elastic pool.
1. In Management Studio, connect to your Azure SQL Database logical server.
2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following ALTER DATABASE statement to make a local database into a Geo-Replication primary
with a readable secondary database on a secondary server in an elastic pool.

ALTER DATABASE <MyDB>

ADD SECONDARY ON SERVER <MySecondaryServer4> WITH (ALLOW_CONNECTIONS = ALL
, SERVICE_OBJECTIVE = ELASTIC_POOL (name = MyElasticPool2));

4. Click Execute to run the query.

Remove secondary database

You can use the ALTER DATABASE statement to permanently terminate the replication partnership between a
secondary database and its primary. This statement is executed on the master database on which the primary
database resides. After the relationship termination, the secondary database becomes a regular read-write
database. If the connectivity to secondary database is broken the command succeeds but the secondary will
become read-write after connectivity is restored. For more information, see ALTER DATABASE (Transact-SQL) and
Service Tiers.
Use the following steps to remove geo-replicated secondary from a Geo-Replication partnership.
1. In Management Studio, connect to your Azure SQL Database logical server.
2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following ALTER DATABASE statement to remove a geo-replicated secondary.

ALTER DATABASE <MyDB>

REMOVE SECONDARY ON SERVER <MySecondaryServer1>;

4. Click Execute to run the query.

Monitor active geo-replication configuration and health

Monitoring tasks include monitoring of the Geo-Replication configuration and monitoring data replication health.
You can use the sys.dm_geo_replication_links dynamic management view in the master database to return
information about all exiting replication links for each database on the Azure SQL Database logical server. This
view contains a row for each of the replication link between primary and secondary databases. You can use the
sys.dm_replication_link_status dynamic management view to return a row for each Azure SQL Database that is
currently engaged in a replication replication link. This includes both primary and secondary databases. If more
than one continuous replication link exists for a given primary database, this table contains a row for each of the
relationships. The view is created in all databases, including the logical master. However, querying this view in the
logical master returns an empty set. You can use the sys.dm_operation_status dynamic management view to
show the status for all database operations including the status of the replication links. For more information, see
sys.geo_replication_links (Azure SQL Database), sys.dm_geo_replication_link_status (Azure SQL Database), and
sys.dm_operation_status (Azure SQL Database).
Use the following steps to monitor an active geo-replication partnership.
1. In Management Studio, connect to your Azure SQL Database logical server.
2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following statement to show all databases with Geo-Replication links.
SELECT database_id, start_date, modify_date, partner_server, partner_database, replication_state_desc, role,
secondary_allow_connections_desc FROM [sys].geo_replication_links;

4. Click Execute to run the query.

5. Open the Databases folder, expand the System Databases folder, right-click on MyDB, and then click New
Query.
6. Use the following statement to show the replication lags and last replication time of my secondary
databases of MyDB.

SELECT link_guid, partner_server, last_replication, replication_lag_sec FROM sys.dm_geo_replication_link_status

7. Click Execute to run the query.

8. Use the following statement to show the most recent geo-replication operations associated with database
MyDB.

SELECT * FROM sys.dm_operation_status where major_resource_id = 'MyDB'

ORDER BY start_time DESC

9. Click Execute to run the query.

Upgrade a non-readable secondary to readable

In April 2017, the non-readable secondary type will be retired and existing non-readable databases will
automatically be upgraded to readable secondaries. If you are using non-readable secondaries today and want to
upgrade them to be readable, you can use the following simple steps for each secondary.

IMPORTANT
There is no self-service method of in-place upgrading of a non-readable secondary to readable. If you drop your only
secondary, then the primary database will remain unprotected until the new secondary is fully synchronized. If your
application’s SLA requires that the primary is always protected, you should consider creating a parallel secondary in a
different server before applying the upgrade steps above. Note each primary can have up to 4 secondary databases.

1. First, connect to the secondary server and drop the non-readable secondary database:

DROP DATABASE <MyNonReadableSecondaryDB>;

2. Now connect to the primary server and add a new readable secondary

ALTER DATABASE <MyDB>

ADD SECONDARY ON SERVER <MySecondaryServer> WITH (ALLOW_CONNECTIONS = ALL);

Next steps
To learn more about active geo-replication, see - Active geo-replication
For a business continuity overview and scenarios, see Business continuity overview
Initiate a planned or unplanned failover for Azure
SQL Database with Transact-SQL
2/16/2017 • 4 min to read • Edit Online

This article shows you how to initiate failover to a secondary SQL Database using Transact-SQL. To configure Geo-
Replication, see Configure Geo-Replication for Azure SQL Database.
To initiate failover, you need the following:
A login that is DBManager on the primary, have db_ownership of the local database that you will geo-replicate,
and be DBManager on the partner server(s) to which you will configure Geo-Replication.
SQL Server Management Studio (SSMS)

IMPORTANT
It is recommended that you always use the latest version of Management Studio to remain synchronized with updates to
Microsoft Azure and SQL Database. Update SQL Server Management Studio.

Initiate a planned failover promoting a secondary database to become

the new primary
You can use the ALTER DATABASE statement to promote a secondary database to become the new primary
database in a planned fashion, demoting the existing primary to become a secondary. This statement is executed
on the master database on the Azure SQL Database logical server in which the geo-replicated secondary database
that is being promoted resides. This functionality is designed for planned failover, such as during the DR drills, and
requires that the primary database be available.
The command performs the following workflow:
1. Temporarily switches replication to synchronous mode, causing all outstanding transactions to be flushed to
the secondary and blocking all new transactions;
2. Switches the roles of the two databases in the Geo-Replication partnership.
This sequence guarantees that the two databases are synchronized before the roles switch and therefore no data
loss will occur. There is a short period during which both databases are unavailable (on the order of 0 to 25
seconds) while the roles are switched. If the primary database has multiple secondary databases, the command will
automatically reconfigure the other secondaries to connect to the new primary. The entire operation should take
less than a minute to complete under normal circumstances. For more information, see ALTER DATABASE
(Transact-SQL) and Service Tiers.
Use the following steps to initiate a planned failover.
1. In Management Studio, connect to the Azure SQL Database logical server in which a geo-replicated secondary
database resides.
2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following ALTER DATABASE statement to switch the secondary database to the primary role.
ALTER DATABASE <MyDB> FAILOVER;

4. Click Execute to run the query.

NOTE
In rare cases, it is possible that the operation cannot complete and may appear stuck. In this case, the user can execute the
force failover command and accept data loss.

Initiate an unplanned failover from the primary database to the

secondary database
You can use the ALTER DATABASE statement to promote a secondary database to become the new primary
database in an unplanned fashion, forcing the demotion of the existing primary to become a secondary at a time
when the primary databse is no longer available. This statement is executed on the master database on the Azure
SQL Database logical server in which the geo-replicated secondary database that is being promoted resides.
This functionality is designed for disaster recovery when restoring availability of the database is critical and some
data loss is acceptable. When forced failover is invoked, the specified secondary database immediately becomes
the primary database and begins accepting write transactions. As soon as the original primary database is able to
reconnect with this new primary database, an incremental backup is taken on the original primary database and
the old primary database is made into a secondary database for the new primary database; subsequently, it is
merely a synchronizing replica of the new primary.
However, because Point In Time Restore is not supported on the secondary databases, if the user wishes to recover
data committed to the old primary database that had not been replicated to the new primary database before the
forced failover occurred, the user will need to engage support to recover this lost data.
If the primary database has multiple secondary databases, the command will automatically reconfigure the other
secondaries to connect to the new primary.
Use the following steps to initiate an unplanned failover.
1. In Management Studio, connect to the Azure SQL Database logical server in which a geo-replicated secondary
database resides.
2. Open the Databases folder, expand the System Databases folder, right-click on master, and then click New
Query.
3. Use the following ALTER DATABASE statement to switch the secondary database to the primary role.

ALTER DATABASE <MyDB> FORCE_FAILOVER_ALLOW_DATA_LOSS;

4. Click Execute to run the query.

NOTE
If the command is issued when the both primary and secondary are online the old primary will become the new secondary
immediately without data synchronization. If the primary is committing transactions when the command is issued some data
loss may occur.

Next steps
After failover, ensure the authentication requirements for your server and database are configured on the new
primary. For details, see SQL Database security after disaster recovery.
To learn recovering after a disaster using Active Geo-Replication, including pre and post recovery steps and
performing a disaster recovery drill, see Disaster Recovery
For a Sasha Nosov blog post about Active Geo-Replication, see Spotlight on new Geo-Replication capabilities
For information about designing cloud applications to use Active Geo-Replication, see Designing cloud
applications for business continuity using Geo-Replication
For information about using Active Geo-Replication with elastic pools, see Elastic Pool disaster recovery
strategies.
For an overview of business continurity, see Business Continuity Overview
Restore an Azure SQL Database or failover to a
secondary
4/14/2017 • 5 min to read • Edit Online

Azure SQL Database offers the following capabilities for recovering from an outage:
Active Geo-Replication
Geo-Restore
To learn about business continuity scenarios and the features supporting these scenarios, see Business continuity.
Prepare for the event of an outage
For success with recovery to another data region using either Active Geo-Replication or geo-redundant backups,
you need to prepare a server in another data center outage to become the new primary server should the need
arise as well as have well defined steps documented and tested to ensure a smooth recovery. These preparation
steps include:
Identify the logical server in another region to become the new primary server. With Active Geo-Replication,
this will be at least one and perhaps each of the secondary servers. For Geo-Restore, this will generally be a
server in the paired region for the region in which your database is located.
Identify, and optionally define, the server-level firewall rules needed on for users to access the new primary
database.
Determine how you are going to redirect users to the new primary server, such as by changing connection
strings or by changing DNS entries.
Identify, and optionally create, the logins that must be present in the master database on the new primary
server, and ensure these logins have appropriate permissions in the master database, if any. For more
information, see SQL Database security after disaster recovery
Identify alert rules that will need to be updated to map to the new primary database.
Document the auditing configuration on the current primary database
Perform a disaster recovery drill. To simulate an outage for Geo-Restore, you can delete or rename the source
database to cause application connectivity failure. To simulate an outage for Active Geo-Replication, you can
disable the web application or virtual machine connected to the database or failover the database to cause
application connectity failures.

When to initiate recovery

The recovery operation impacts the application. It requires changing the SQL connection string or redirection
using DNS and could result in permanent data loss. Therefore, it should be done only when the outage is likely to
last longer than your application's recovery time objective. When the application is deployed to production you
should perform regular monitoring of the application health and use the following data points to assert that the
recovery is warranted:
1. Permanent connectivity failure from the application tier to the database.
2. The Azure portal shows an alert about an incident in the region with broad impact.
3. The Azure SQL Database server is marked as degraded.
Depending on your application tolerance to downtime and possible business liability you can consider the
following recovery options.
Use the Get Recoverable Database (LastAvailableBackupDate) to get the latest Geo-replicated restore point.

Wait for service recovery

The Azure teams work diligently to restore service availability as quickly as possible but depending on the root
cause it can take hours or days. If your application can tolerate significant downtime you can simply wait for the
recovery to complete. In this case, no action on your part is required. You can see the current service status on
our Azure Service Health Dashboard. After the recovery of the region your application’s availability will be
restored.

Failover to geo-replicated secondary database

If your application’s downtime can result in business liability you should be using geo-replicated database(s) in
your application. It will enable the application to quickly restore availability in a different region in case of an
outage. Learn how to configure Geo-Replication.
To restore availability of the database(s) you need to initiate the failover to the geo-replicated secondary using
one of the supported methods.
Use one of the following guides to failover to a geo-replicated secondary database:
Failover to a geo-replicated secondary using Azure Portal
Failover to a geo-replicated secondary using PowerShell
Failover to a geo-replicated secondary using T-SQL

Recover using Geo-Restore

If your application’s downtime does not result in business liability you can use Geo-Restore as a method to
recover your application database(s). It creates a copy of the database from its latest geo-redundant backup.

Configure your database after recovery

If you are using either geo-replication failover or geo-restore to recover from an outage, you must make sure
that the connectivity to the new databases is properly configured so that the normal application function can be
resumed. This is a checklist of tasks to get your recovered database production ready.
Update Connection Strings
Because your recovered database will reside in a different server, you need to update your application’s
connection string to point to that server.
For more information about changing connection strings, see the appropriate development language for your
connection library.
Configure Firewall Rules
You need to make sure that the firewall rules configured on server and on the database match those that were
configured on the primary server and primary database. For more information, see How to: Configure Firewall
Settings (Azure SQL Database).
Configure Logins and Database Users
You need to make sure that all the logins used by your application exist on the server which is hosting your
recovered database. For more information, see Security Configuration for Geo-Replication.
NOTE
You should configure and test your server firewall rules and logins (and their permissions) during a disaster recovery drill.
These server-level objects and their configuration may not be available during the outage.

Setup Telemetry Alerts

You need to make sure your existing alert rule settings are updated to map to the recovered database and the
different server.
For more information about database alert rules, see Receive Alert Notifications and Track Service Health.
Enable Auditing
If auditing is required to access your database, you need to enable Auditing after the database recovery. A good
indicator that auditing is required is that client applications use secure connection strings in a pattern of
*.database.secure.windows.net. For more information, see Database auditing.

Next steps
To learn about Azure SQL Database automated backups, see SQL Database automated backups
To learn about business continuity design and recovery scenarios, see Continuity scenarios
To learn about using automated backups for recovery, see restore a database from the service-initiated
backups
Performing Disaster Recovery Drill
3/10/2017 • 2 min to read • Edit Online

It is recommended that validation of application readiness for recovery workflow is performed periodically.
Verifying the application behavior and implications of data loss and/or the disruption that failover involves is a
good engineering practice. It is also a requirement by most industry standards as part of business continuity
certification.
Performing a disaster recovery drill consists of:
Simulating data tier outage
Recovering
Validate application integrity post recovery
Depending on how you designed your application for business continuity, the workflow to execute the drill can
vary. Below we describe the best practices conducting a disaster recovery drill in the context of Azure SQL
Database.

Geo-Restore
To prevent the potential data loss when conducting a disaster recovery drill, we recommend performing the drill
using a test environment by creating a copy of the production environment and using it to verify the application’s
failover workflow.
Outage simulation
To simulate the outage you can delete or rename the source database. This will cause application connectivity
failure.
Recovery
Perform the Geo-Restore of the database into a different server as described here.
Change the application configuration to connect to the recovered database(s) and follow the Configure a
database after recovery guide to complete the recovery.
Validation
Complete the drill by verifying the application integrity post recovery (i.e. connection strings, logins, basic
functionality testing or other validations part of standard application signoffs procedures).

Geo-Replication
For a database that is protected using Geo-Replication the drill exercise will involve planned failover to the
secondary database. The planned failover ensures that the primary and the secondary databases remain in sync
when the roles are switched. Unlike the unplanned failover, this operation will not result in data loss, so the drill can
be performed in the production environment.
Outage simulation
To simulate the outage you can disable the web application or virtual machine connected to the database. This will
result in the connectivity failures for the web clients.
Recovery
Make sure the application configuration in the DR region points to the former secondary which will become
fully accessible new primary.
Perform planned failover to make the secondary database a new primary
Follow the Configure a database after recovery guide to complete the recovery.
Validation
Complete the drill by verifying the application integrity post recovery (i.e. connection strings, logins, basic
functionality testing or other validations part of standard application signoffs procedures).

Next steps
To learn about business continuity scenarios, see Continuity scenarios
To learn about Azure SQL Database automated backups, see SQL Database automated backups
To learn about using automated backups for recovery, see restore a database from the service-initiated backups
To learn about faster recovery options, see Active-Geo-Replication
Load data from CSV into Azure SQL Database (flat
files)
4/10/2017 • 2 min to read • Edit Online

You can use the bcp command-line utility to import data from a CSV file into Azure SQL Database.

Before you begin

Prerequisites
To step through this tutorial, you need:
An Azure SQL Database logical server and database
The bcp command-line utility installed
The sqlcmd command-line utility installed
You can download the bcp and sqlcmd utilities from the Microsoft Download Center.
Data in ASCII or UTF -16 format
If you are trying this tutorial with your own data, your data needs to use the ASCII or UTF-16 encoding since bcp
does not support UTF-8.

1. Create a destination table

Define a table in SQL Database as the destination table. The columns in the table must correspond to the data in
each row of your data file.
To create a table, open a command prompt and use sqlcmd.exe to run the following command:

sqlcmd.exe -S <server name> -d <database name> -U <username> -P <password> -I -Q "

CREATE TABLE DimDate2
(
DateId INT NOT NULL,
CalendarQuarter TINYINT NOT NULL,
FiscalQuarter TINYINT NOT NULL
)
;
"

2. Create a source data file

Open Notepad and copy the following lines of data into a new text file and then save this file to your local temp
directory, C:\Temp\DimDate2.txt. This data is in ASCII format.
20150301,1,3
20150501,2,4
20151001,4,2
20150201,1,3
20151201,4,2
20150801,3,1
20150601,2,4
20151101,4,2
20150401,2,4
20150701,3,1
20150901,3,1
20150101,1,3

(Optional) To export your own data from a SQL Server database, open a command prompt and run the following
command. Replace TableName, ServerName, DatabaseName, Username, and Password with your own information.

bcp <TableName> out C:\Temp\DimDate2_export.txt -S <ServerName> -d <DatabaseName> -U <Username> -P <Password> -q -c -t ','

3. Load the data

To load the data, open a command prompt and run the following command, replacing the values for Server Name,
Database name, Username, and Password with your own information.

bcp DimDate2 in C:\Temp\DimDate2.txt -S <ServerName> -d <DatabaseName> -U <Username> -P <password> -q -c -t ','

Use this command to verify the data was loaded properly

sqlcmd.exe -S <server name> -d <database name> -U <username> -P <password> -I -Q "SELECT * FROM DimDate2 ORDER BY 1;"

The results should look like this:

DATEID CALENDARQUARTER FISCALQUARTER

20150101 1 3

20150201 1 3

20150301 1 3

20150401 2 4

20150501 2 4

20150601 2 4

20150701 3 1

20150801 3 1

20151001 4 2
DATEID CALENDARQUARTER FISCALQUARTER

20151101 4 2

20151201 4 2

Next steps
To migrate a SQL Server database, see SQL Server database migration.
Tutorial: Copy data from Blob Storage to SQL
Database using Data Factory
4/12/2017 • 4 min to read • Edit Online

In this tutorial, you create a data factory with a pipeline to copy data from Blob storage to SQL database.
The Copy Activity performs the data movement in Azure Data Factory. It is powered by a globally available service
that can copy data between various data stores in a secure, reliable, and scalable way. See Data Movement
Activities article for details about the Copy Activity.

NOTE
For a detailed overview of the Data Factory service, see the Introduction to Azure Data Factory article.

Prerequisites for the tutorial

Before you begin this tutorial, you must have the following prerequisites:
Azure subscription. If you don't have a subscription, you can create a free trial account in just a couple of
minutes. See the Free Trial article for details.
Azure Storage Account. You use the blob storage as a source data store in this tutorial. if you don't have an
Azure storage account, see the Create a storage account article for steps to create one.
Azure SQL Database. You use an Azure SQL database as a destination data store in this tutorial. If you don't
have an Azure SQL database that you can use in the tutorial, See How to create and configure an Azure SQL
Database to create one.
SQL Server 2012/2014 or Visual Studio 2013. You use SQL Server Management Studio or Visual Studio to
create a sample database and to view the result data in the database.

Collect blob storage account name and key

You need the account name and account key of your Azure storage account to do this tutorial. Note down account
name and account key for your Azure storage account.
1. Log in to the Azure portal.
2. Click More services on the left menu and select Storage Accounts.
3. In the Storage Accounts blade, select the Azure storage account that you want to use in this tutorial.
4. Select Access keys link under SETTINGS.
5. Click copy (image) button next to Storage account name text box and save/paste it somewhere (for example:
in a text file).
6. Repeat the previous step to copy or note down the key1.

7. Close all the blades by clicking X.

Collect SQL server, database, user names

You need the names of Azure SQL server, database, and user to do this tutorial. Note down names of server,
database, and user for your Azure SQL database.
1. In the Azure portal, click More services on the left and select SQL databases.
2. In the SQL databases blade, select the database that you want to use in this tutorial. Note down the database
name.
3. In the SQL database blade, click Properties under SETTINGS.
4. Note down the values for SERVER NAME and SERVER ADMIN LOGIN.
5. Close all the blades by clicking X.
Allow Azure services to access SQL server
Ensure that Allow access to Azure services setting turned ON for your Azure SQL server so that the Data Factory
service can access your Azure SQL server. To verify and turn on this setting, do the following steps:
1. Click More services hub on the left and click SQL servers.
2. Select your server, and click Firewall under SETTINGS.
3. In the Firewall settings blade, click ON for Allow access to Azure services.
4. Close all the blades by clicking X.

Prepare Blob Storage and SQL Database

Now, prepare your Azure blob storage and Azure SQL database for the tutorial by performing the following steps:
1. Launch Notepad. Copy the following text and save it as emp.txt to C:\ADFGetStarted folder on your hard
drive.

John, Doe
Jane, Doe

2. Use tools such as Azure Storage Explorer to create the adftutorial container and to upload the emp.txt file
to the container.

3. Use the following SQL script to create the emp table in your Azure SQL Database.

CREATE TABLE dbo.emp

(
ID int IDENTITY(1,1) NOT NULL,
FirstName varchar(50),
LastName varchar(50),
)
GO

CREATE CLUSTERED INDEX IX_emp_ID ON dbo.emp (ID);

If you have SQL Server 2012/2014 installed on your computer: follow instructions from Managing
Azure SQL Database using SQL Server Management Studio to connect to your Azure SQL server and run the
SQL script. This article uses the classic Azure portal, not the new Azure portal, to configure firewall for an
Azure SQL server.
If your client is not allowed to access the Azure SQL server, you need to configure firewall for your Azure
SQL server to allow access from your machine (IP Address). See this article for steps to configure the firewall
for your Azure SQL server.
You have completed the prerequisites. You can create a data factory using one of the following ways. Click one of
the options in the drop-down list at the top or the following links to perform the tutorial.
Copy Wizard
Azure portal
Visual Studio
PowerShell
Azure Resource Manager template
REST API
.NET API

NOTE
The data pipeline in this tutorial copies data from a source data store to a destination data store. It does not transform input
data to produce output data. For a tutorial on how to transform data using Azure Data Factory, see Tutorial: Build your first
pipeline to transform data using Hadoop cluster.
You can chain two activities (run one activity after another) by setting the output dataset of one activity as the input dataset
of the other activity. See Scheduling and execution in Data Factory for detailed information.
Getting Started with Azure SQL Data Sync (Preview)
4/10/2017 • 5 min to read • Edit Online

In this tutorial, you learn the fundamentals of Azure SQL Data Sync using the Azure Classic Portal.
This tutorial assumes minimal prior experience with SQL Server and Azure SQL Database. In this tutorial, you create
a hybrid (SQL Server and SQL Database instances) sync group fully configured and synchronizing on the schedule
you set.

NOTE
The complete technical documentation set for Azure SQL Data Sync, formerly located on MSDN, is available as a .pdf.
Download it here.

Step 1: Connect to the Azure SQL Database

1. Sign in to the Classic Portal.
2. Click SQL DATABASES in the left pane.
3. Click SYNC at the bottom of the page. When you click SYNC, a list appears showing the things you can add -
New Sync Group and New Sync Agent.
4. To launch the New SQL Data Sync Agent wizard, click New Sync Agent.
5. If you haven't added an agent before, click download it here.

Step 2: Add a Client Agent

This step is required only if you are going to have an on-premises SQL Server database included in your sync
group. Skip to Step 4 if your sync group has only SQL Database instances.
Step 2a: Install the required software
Be sure that you have the following installed on the computer where you install the Client Agent.
.NET Framework 4.0
Install .NET Framework 4.0 from here.
Microsoft SQL Server 2008 R2 SP1 System CLR Types (x86)
Install the Microsoft SQL Server 2008 R2 SP1 System CLR Types (x86) from here
Microsoft SQL Server 2008 R2 SP1 Shared Management Objects (x86)
Install the Microsoft SQL Server 2008 R2 SP1 Shared Management Objects (x86) from here
Step 2b: Install a new Client Agent
Follow the instructions in Install a Client Agent (SQL Data Sync) to install the agent.
Step 2c: Finish the New SQL Data Sync Agent wizard
1. Return to the New SQL Data Sync Agent wizard.
2. Give the agent a meaningful name.
3. From the dropdown, select the REGION (data center) to host this agent.
4. From the dropdown, select the SUBSCRIPTION to host this agent.
5. Click the right-arrow.

Step 3: Register a SQL Server database with the Client Agent

After the Client Agent is installed, register every on-premises SQL Server database that you intend to include in a
sync group with the agent. To register a database with the agent, follow the instructions at Register a SQL Server
Database with a Client Agent.

Step 4: Create a sync group

Step 4a: Start the New Sync Group wizard
1. Return to the Classic Portal.
2. Click SQL DATABASES.
3. Click ADD SYNC at the bottom of the page then select New Sync Group from the drawer.

Step 4b: Enter the basic settings

1. Enter a meaningful name for the sync group.
2. From the dropdown, select the REGION (Data Center) to host this sync group.
3. Click the right-arrow.
Step 4c: Define the sync hub
1. From the dropdown, select the SQL Database instance to serve as the sync group hub.
2. Enter the credentials for this SQL Database instance - HUB USERNAME and HUB PASSWORD.
3. Wait for SQL Data Sync to confirm the USERNAME and PASSWORD. You will see a green check mark appear to
the right of the PASSWORD when the credentials are confirmed.
4. From the dropdown, select the CONFLICT RESOLUTION policy.
Hub Wins - any change written to the hub database write to the reference databases, overwriting changes
in the same reference database record. Functionally, this means that the first change written to the hub
propagates to the other databases.
Client Wins - changes written to the hub are overwritten by changes in reference databases. Functionally,
this means that the last change written to the hub is the one kept and propagated to the other databases.
5. Click the right-arrow.
Step 4d: Add a reference database
Repeat this step for each additional database you want to add to the sync group.
1. From the dropdown, select the database to add.
Databases in the dropdown include both SQL Server databases that have been registered with the agent and
SQL Database instances.
2. Enter credentials for this database - USERNAME and PASSWORD.
3. From the dropdown, select the SYNC DIRECTION for this database.
Bi-directional - changes in the reference database are written to the hub database, and changes to the hub
database are written to the reference database.
Sync from the Hub - The database receives updates from the Hub. It does not send changes to the Hub.
Sync to the Hub - The database sends updates to the Hub. Changes in the Hub are not written to this
database.
4. To finish creating the sync group, click the check mark in the lower right of the wizard. Wait for the SQL Data
Sync to confirm the credentials. A green check indicates that the credentials are confirmed.
5. Click the check mark a second time. This returns you to the SYNC page under SQL Databases. This sync
group is now listed with your other sync groups and agents.

Step 5: Define the data to sync

Azure SQL Data Sync allows you to select tables and columns to synchronize. If you also want to filter a column so
that only rows with specific values (such as, Age>=65) synchronize, use the SQL Data Sync portal at Azure and the
documentation at Select the Tables, Columns, and Rows to Synchronize to define the data to sync.
1. Return to the Classic Portal.
2. Click SQL DATABASES.
3. Click the SYNC tab.
4. Click the name of this sync group.
5. Click the SYNC RULES tab.
6. Select the database you want to provide the sync group schema.
7. Click the right-arrow.
8. Click REFRESH SCHEMA.
9. For each table in the database, select the columns to include in the synchronizations.
Columns with unsupported data types cannot be selected.
If no columns in a table are selected, the table is not included in the sync group.
To select/unselect all the tables, click SELECT at the bottom of the screen.
10. Click SAVE, then wait for the sync group to finish provisioning.
11. To return to the Data Sync landing page, click the back-arrow in the upper left of the screen (above the sync
group's name).

Step 6: Configure your sync group

You can always synchronize a sync group by clicking SYNC at the bottom of the Data Sync landing page. To
synchronize on a schedule, you configure the sync group.
1. Return to the Classic Portal.
2. Click SQL DATABASES.
3. Click the SYNC tab.
4. Click the name of this sync group.
5. Click the CONFIGURE tab.
6. AUTOMATIC SYNC
To configure the sync group to sync on a set frequency, click ON. You can still sync on demand by
clicking SYNC.
Click OFF to configure the sync group to sync only when you click SYNC.
7. SYNC FREQUENCY
If AUTOMATIC SYNC is ON, set the synchronization frequency. The frequency must be between 5 Minutes
and 1 Month.
8. Click SAVE.
Congratulations. You have created a sync group that includes both a SQL Database instance and a SQL Server
database.

Next steps
For additional information on SQL Database and SQL Data Sync see:
Download the complete SQL Data Sync technical documentation
SQL Database Overview
Database Lifecycle Management
Connect to SQL Database using C and C++
4/14/2017 • 5 min to read • Edit Online

This post is aimed at C and C++ developers trying to connect to Azure SQL DB. It is broken down into sections so
you can jump to the section that best captures your interest.

Prerequisites for the C/C++ tutorial

Make sure you have the following items:
An active Azure account. If you don't have one, you can sign up for a Free Azure Trial.
Visual Studio. You must install the C++ language components to build and run this sample.
Visual Studio Linux Development. If you are developing on Linux, you must also install the Visual Studio Linux
extension.

Azure SQL Database and SQL Server on virtual machines

Azure SQL is built on Microsoft SQL Server and is designed to provide a high-availability, performant, and scalable
service. There are many benefits to using SQL Azure over your proprietary database running on premises. With
SQL Azure you don’t have to install, set up, maintain, or manage your database but only the content and the
structure of your database. Typical things that we worry about with databases like fault tolerance and redundancy
are all built in.
Azure currently has two options for hosting SQL server workloads: Azure SQL database, database as a service and
SQL server on Virtual Machines (VM). We will not get into detail about the differences between these two except
that Azure SQL database is your best bet for new cloud-based applications to take advantage of the cost savings
and performance optimization that cloud services provide. If you are considering migrating or extending your on-
premises applications to the cloud, SQL server on Azure virtual machine might work out better for you. To keep
things simple for this article, let's create an Azure SQL database.

Data access technologies: ODBC and OLE DB

Connecting to Azure SQL DB is no different and currently there are two ways to connect to databases: ODBC (Open
Database connectivity) and OLE DB (Object Linking and Embedding database). In recent years, Microsoft has
aligned with ODBC for native relational data access. ODBC is relatively simple, and also much faster than OLE DB.
The only caveat here is that ODBC does use an old C-style API.

Step 1: Creating your Azure SQL Database

See the getting started page to learn how to create a sample database. Alternatively, you can follow this short two-
minute video to create an Azure SQL database using the Azure portal.

Step 2: Get connection string

After your Azure SQL database has been provisioned, you need to carry out the following steps to determine
connection information and add your client IP for firewall access.
In Azure portal, go to your Azure SQL database ODBC connection string by using the Show database connection
strings listed as a part of the overview section for your database:
Copy the contents of the ODBC (Includes Node.js) [SQL authentication] string. We use this string later to
connect from our C++ ODBC command-line interpreter. This string provides details such as the driver, server, and
other database connection parameters.

Step 3: Add your IP to the firewall

Go to the firewall section for your Database server and add your client IP to the firewall using these steps to make
sure we can establish a successful connection:

At this point, you have configured your Azure SQL DB and are ready to connect from your C++ code.

Step 4: Connecting from a Windows C/C++ application

You can easily connect to your Azure SQL DB using ODBC on Windows using this sample that builds with Visual
Studio. The sample implements an ODBC command-line interpreter that can be used to connect to our Azure SQL
DB. This sample takes either a Database source name file (DSN) file as a command-line argument or the verbose
connection string that we copied earlier from the Azure portal. Bring up the property page for this project and paste
the connection string as a command argument as shown here:
Make sure you provide the right authentication details for your database as a part of that database connection
string.
Launch the application to build it. You should see the following window validating a successful connection. You can
even run some basic SQL commands like create table to validate your database connectivity:

Alternatively, you could create a DSN file using the wizard that is launched when no command arguments are
provided. We recommend that you try this option as well. You can use this DSN file for automation and protecting
your authentication settings:

Congratulations! You have now successfully connected to Azure SQL using C++ and ODBC on Windows. You can
continue reading to do the same for Linux platform as well.

Step 5: Connecting from a Linux C/C++ application

In case you haven’t heard the news yet, Visual Studio now allows you to develop C++ Linux application as well. You
can read about this new scenario in the Visual C++ for Linux Development blog. To build for Linux, you need a
remote machine where your Linux distro is running. If you don’t have one available, you can set one up quickly
using Linux Azure Virtual machines.
For this tutorial, let us assume that you have an Ubuntu 16.04 Linux distribution set up. The steps here should also
apply to Ubuntu 15.10, Red Hat 6, and Red Hat 7.
The following steps install the libraries needed for SQL and ODBC for your distro:

sudo su
sh -c 'echo "deb [arch=amd64] https://apt-mo.trafficmanager.net/repos/mssql-ubuntu-test/ xenial main" > /etc/apt/sources.list.d/mssqlpreview.list'
sudo apt-key adv --keyserver apt-mo.trafficmanager.net --recv-keys 417A0893
apt-get update
apt-get install msodbcsql
apt-get install unixodbc-dev-utf16 #this step is optional but recommended*

Launch Visual Studio. Under Tools -> Options -> Cross Platform -> Connection Manager, add a connection to your
Linux box:

After connection over SSH is established, create an Empty project (Linux) template:

You can then add a new C source file and replace it with this content. Using the ODBC APIs SQLAllocHandle,
SQLSetConnectAttr, and SQLDriverConnect, you should be able to initialize and establish a connection to your
database. Like with the Windows ODBC sample, you need to replace the SQLDriverConnect call with the details
from your database connection string parameters copied from the Azure portal previously.
retcode = SQLDriverConnect(
hdbc, NULL, "Driver=ODBC Driver 13 for SQL"
"Server;Server=<yourserver>;Uid=<yourusername>;Pwd=<"
"yourpassword>;database=<yourdatabase>",
SQL_NTS, outstr, sizeof(outstr), &outstrlen, SQL_DRIVER_NOPROMPT);

The last thing to do before compiling is to add odbc as a library dependency:

To launch your application, bring up the Linux Console from the Debug menu:

If your connection was successful, you should now see the current database name printed in the Linux Console:

Congratulations! You have successfully completed the tutorial and can now connect to your Azure SQL DB from
C++ on Windows and Linux platforms.

Get the complete C/C++ tutorial solution

You can find the GetStarted solution that contains all the samples in this article at github:
ODBC C++ Windows sample, Download the Windows C++ ODBC Sample to connect to Azure SQL
ODBC C++ Linux sample, Download the Linux C++ ODBC Sample to connect to Azure SQL

Next steps
Review the SQL Database Development Overview
More information on the ODBC API Reference
Additional resources
Design Patterns for Multi-tenant SaaS Applications with Azure SQL Database
Explore all the capabilities of SQL Database
Connect Excel to an Azure SQL database and create a
report
4/14/2017 • 3 min to read • Edit Online

Connect Excel to a SQL database in the cloud and import data and create tables and charts based on values in the
database. In this tutorial you will set up the connection between Excel and a database table, save the file that stores
data and the connection information for Excel, and then create a pivot chart from the database values.
You'll need a SQL database in Azure before you get started. If you don't have one, see Create your first SQL
database to get a database with sample data up and running in a few minutes. In this article, you'll import sample
data into Excel from that article, but you can follow similar steps with your own data.
You'll also need a copy of Excel. This article uses Microsoft Excel 2016.

Connect Excel to a SQL database and create an odc file

1. To connect Excel to SQL database, open Excel and then create a new workbook or open an existing Excel
workbook.
2. In the menu bar at the top of the page click Data, click From Other Sources, and then click From SQL
Server.

The Data Connection Wizard opens.

3. In the Connect to Database Server dialog box, type the SQL Database Server name you want to connect to in
the form <servername>.database.windows.net. For example, adworkserver.database.windows.net.
4. Under Log on credentials, click Use the following User Name and Password, type the User Name and
Password you set up for the SQL Database server when you created it, and then click Next.
TIP
Depending on your network environment, you may not be able to connect or you may lose the connection if the SQL
Database server doesn't allow traffic from your client IP address. Go to the Azure portal, click SQL servers, click your
server, click firewall under settings and add your client IP address. See How to configure firewall settings for details.

5. In the Select Database and Table dialog, select the database you want to work with from the list, and then
click the tables or views you want to work with (we chose vGetAllCategories), and then click Next.

The Save Data Connection File and Finish dialog box opens, where you provide information about the
Office database connection (*.odc) file that Excel uses. You can leave the defaults or customize your
selections.
6. You can leave the defaults, but note the File Name in particular. A Description, a Friendly Name, and
Search Keywords help you and other users remember what you're connecting to and find the connection.
Click Always attempt to use this file to refresh data if you want connection information stored in the odc
file so it can update when you connect to it, and then click Finish.

The Import data dialog box appears.

Import the data into Excel and create a pivot chart

Now that you've established the connection and created the file with data and connection information, you're
reading to import the data.
1. In the Import Data dialog, click the option you want for presenting your data in the worksheet and then
click OK. We chose PivotChart. You can also choose to create a New worksheet or to Add this data to a
Data Model. For more information on Data Models, see Create a data model in Excel. Click Properties to
explore information about the odc file you created in the previous step and to choose options for refreshing
the data.
The worksheet now has an empty pivot table and chart.
2. Under PivotTable Fields, select all the check-boxes for the fields you want to view.
TIP
If you want to connect other Excel workbooks and worksheets to the database, click Data, click Connections, click Add,
choose the connection you created from the list, and then click Open.

Next steps
Learn how to Connect to SQL Database with SQL Server Management Studio for advanced querying and
analysis.
Learn about the benefits of elastic pools.
Learn how to create a web application that connects to SQL Database on the back-end.
Troubleshoot, diagnose, and prevent SQL connection
errors and transient errors for SQL Database
4/10/2017 • 15 min to read • Edit Online

This article describes how to prevent, troubleshoot, diagnose, and mitigate connection errors and transient errors
that your client application encounters when it interacts with Azure SQL Database. Learn how to configure retry
logic, build the connection string, and adjust other connection settings.

Transient errors (transient faults)

A transient error - also, transient fault - has an underlying cause that will soon resolve itself. An occasional cause of
transient errors is when the Azure system quickly shifts hardware resources to better load-balance various
workloads. Most of these reconfiguration events often complete in less than 60 seconds. During this
reconfiguration time span, you may have connectivity issues to Azure SQL Database. Applications connecting to
Azure SQL Database should be built to expect these transient errors, handle them by implementing retry logic in
their code instead of surfacing them to users as application errors.
If your client program is using ADO.NET, your program is told about the transient error by the throw of an
SqlException. The Number property can be compared against the list of transient errors near the top of the topic:
SQL error codes for SQL Database client applications.
Connection versus command
You'll retry the SQL connection or establish it again, depending on the following:
A transient error occurs during a connection try: The connection should be retried after delaying for several
seconds.
A transient error occurs during an SQL query command: The command should not be immediately retried.
Instead, after a delay, the connection should be freshly established. Then the command can be retried.
Retry logic for transient errors
Client programs that occasionally encounter a transient error are more robust when they contain retry logic.
When your program communicates with Azure SQL Database through a 3rd party middleware, inquire with the
vendor whether the middleware contains retry logic for transient errors.
Principles for retry
An attempt to open a connection should be retried if the error is transient.
An SQL SELECT statement that fails with a transient error should not be retried directly.
Instead, establish a fresh connection, and then retry the SELECT.
When an SQL UPDATE statement fails with a transient error, a fresh connection should be established
before the UPDATE is retried.
The retry logic must ensure that either the entire database transaction completed, or that the entire
transaction is rolled back.
Other considerations for retry
A batch program that is automatically started after work hours, and which will complete before morning, can
afford to very patient with long time intervals between its retry attempts.
A user interface program should account for the human tendency to give up after too long a wait.
However, the solution must not be to retry every few seconds, because that policy can flood the system
with requests.
Interval increase between retries
We recommend that you delay for 5 seconds before your first retry. Retrying after a delay shorter than 5 seconds
risks overwhelming the cloud service. For each subsequent retry the delay should grow exponentially, up to a
maximum of 60 seconds.
A discussion of the blocking period for clients that use ADO.NET is available in SQL Server Connection Pooling
(ADO.NET).
You might also want to set a maximum number of retries before the program self-terminates.
Code samples with retry logic
Code samples with retry logic, in a variety of programming languages, are available at:
Connection libraries for SQL Database and SQL Server
Test your retry logic
To test your retry logic, you must simulate or cause an error than can be corrected while your program is still
running.
Te st b y d i sc o n n e c t i n g fr o m t h e n e t w o r k

One way you can test your retry logic is to disconnect your client computer from the network while the program is
running. The error will be:
SqlException.Number = 11001
Message: "No such host is known"
As part of the first retry attempt, your program can correct the misspelling, and then attempt to connect.
To make this practical, you unplug your computer from the network before you start your program. Then your
program recognizes a run time parameter that causes the program to:
1. Temporarily add 11001 to its list of errors to consider as transient.
2. Attempt its first connection as usual.
3. After the error is caught, remove 11001 from the list.
4. Display a message telling the user to plug the computer into the network.
Pause further execution by using either the Console.ReadLine method or a dialog with an OK button.
The user presses the Enter key after the computer plugged into the network.
5. Attempt again to connect, expecting success.
Te st b y m i ssp e l l i n g t h e d a t a b a se n a m e w h e n c o n n e c t i n g

Your program can purposely misspell the user name before the first connection attempt. The error will be:
SqlException.Number = 18456
Message: "Login failed for user 'WRONG_MyUserName'."
As part of the first retry attempt, your program can correct the misspelling, and then attempt to connect.
To make this practical, your program could recognize a run time parameter that causes the program to:
1. Temporarily add 18456 to its list of errors to consider as transient.
2. Purposely add 'WRONG_' to the user name.
3. After the error is caught, remove 18456 from the list.
4. Remove 'WRONG_' from the user name.
5. Attempt again to connect, expecting success.
.NET SqlConnection parameters for connection retry
If your client program connects to to Azure SQL Database by using the .NET Framework class
System.Data.SqlClient.SqlConnection, you should use .NET 4.6.1 or later (or .NET Core) so you can leverage its
connection retry feature. Details of the feature are here.
When you build the connection string for your SqlConnection object, you should coordinate the values among
the following parameters:
ConnectRetryCount (Default is 1. Range is 0 through 255.)
ConnectRetryInterval (Default is 1 second. Range is 1 through 60.)
Connection Timeout (Default is 15 seconds. Range is 0 through 2147483647)
Specifically, your chosen values should make the following equality true:
Connection Timeout = ConnectRetryCount * ConnectionRetryInterval
For example, if the count = 3, and interval = 10 seconds, a timeout of only 29 seconds would not quite give the
system enough time for its 3rd and final retry at connecting: 29 < 3 * 10.
Connection versus command
The ConnectRetryCount and ConnectRetryInterval parameters let your SqlConnection object retry the
connect operation without telling or bothering your program, such as returning control to your program. The
retries can occur in the following situations:
mySqlConnection.Open method call
mySqlConnection.Execute method call
There is a subtlety. If a transient error occurs while your query is being executed, your SqlConnection object does
not retry the connect operation, and it certainly does not retry your query. However, SqlConnection very quickly
checks the connection before sending your query for execution. If the quick check detects a connection problem,
SqlConnection retries the connect operation. If the retry succeeds, you query is sent for execution.
Should ConnectRetryCount be combined with application retry logic?
Suppose your application has robust custom retry logic. It might retry the connect operation 4 times. If you add
ConnectRetryInterval and ConnectRetryCount =3 to your connection string, you will increase the retry count to
4 * 3 = 12 retries. You might not intend such a high number of retries.

Connections to Azure SQL Database

Connection: Connection string
The connection string necessary for connecting to Azure SQL Database is slightly different from the string for
connecting to Microsoft SQL Server. You can copy the connection string for your database from the Azure Portal.
Obtain the connection string from the Azure portal
Use the Azure portal to obtain the connection string necessary for your client program to interact with Azure SQL
Database:
1. Click BROWSE > SQL databases.
2. Enter the name of your database into the filter text box near the upper-left of the SQL databases blade.
3. Click the row for your database.
4. After the blade appears for your database, for visual convenience you can click the standard minimize
controls to collapse the blades you used for browsing and database filtering.
5. On the blade for your database, click Show database connection strings.
6. If you intend to use the ADO.NET connection library, copy the string labeled ADO.

7. In one format or another, paste the connection string information into your client program code.
For more information, see:
Connection Strings and Configuration Files.
Connection: IP address
You must configure the SQL Database server to accept communication from the IP address of the computer that
hosts your client program. You do this by editing the firewall settings through the Azure Portal.
If you forget to configure the IP address, your program will fail with a handy error message that states the
necessary IP address.
1. Log in to the Azure portal at http://portal.azure.com/.
2. In the left banner, click BROWSE ALL. The Browse blade is displayed.
3. Scroll and click SQL servers. The SQL servers blade is displayed.
4. For convenience, click the minimize control on the earlier Browse blade.
5. In the filter text box, start typing the name of your server. Your row is displayed.
6. Click the row for your server. A blade for your server is displayed.
7. On your server blade, click Settings. The Settings blade is displayed.
8. Click Firewall. The Firewall Settings blade is displayed.

9. Click Add Client IP. Type in a name for your new rule into the first text box.
10. Type in the low and high IP address values for the range you want to enable.
It can be handy to have the low value end with .0 and the high with .255.
11. Click Save.
For more information, see: How to: Configure firewall settings on SQL Database
Connection: Ports
Typically you only need to ensure that port 1433 is open for outbound communication, on the computer that hosts
you client program.
For example, when your client program is hosted on a Windows computer, the Windows Firewall on the host
enables you to open port 1433:
1. Open the Control Panel
2. > All Control Panel Items
3. > Windows Firewall
4. > Advanced Settings
5. > Outbound Rules
6. > Actions
7. > New Rule
If your client program is hosted on an Azure virtual machine (VM), you should read:
Ports beyond 1433 for ADO.NET 4.5 and SQL Database.
For background information about cofiguration of ports and IP address, see: Azure SQL Database firewall
Connection: ADO.NET 4.6.1
If your program uses ADO.NET classes like System.Data.SqlClient.SqlConnection to connect to Azure SQL
Database, we recommend that you use .NET Framework version 4.6.1 or higher.
ADO.NET 4.6.1:
For Azure SQL Database, there is improved reliability when you open a connection by using the
SqlConnection.Open method. The Open method now incorporates best effort retry mechanisms in response
to transient faults, for certain errors within the Connection Timeout period.
Supports connection pooling. This includes an efficient verification that the connection object it gives your
program is functioning.
When you use a connection object from a connection pool, we recommend that your program temporarily close
the connection when not immediately using it. Re-opening a connection is not expensive the way creating a new
connection is.
If you are using ADO.NET 4.0 or earlier, we recommend that you upgrade to the latest ADO.NET.
As of November 2015, you can download ADO.NET 4.6.1.

Diagnostics
Diagnostics: Test whether utilities can connect
If your program is failing to connect to Azure SQL Database, one diagnostic option is to try to connect with a utility
program. Ideally the utility would connect by using the same library that your program uses.
On any Windows computer, you can try these utilities:
SQL Server Management Studio (ssms.exe), which connects by using ADO.NET.
sqlcmd.exe, which connects by using ODBC.
Once connected, test whether a short SQL SELECT query works.
Diagnostics: Check the open ports
Suppose you suspect that connection attempts are failing due to port issues. On your computer you can run a
utility that reports on the port configurations.
On Linux the following utilities might be helpful:
netstat -nap
nmap -sS -O 127.0.0.1
(Change the example value to be your IP address.)
On Windows the PortQry.exe utility might be helpful. Here is an example execution that queried the port situation
on an Azure SQL Database server, and which was run on a laptop computer:

[C:\Users\johndoe\]
>> portqry.exe -n johndoesvr9.database.windows.net -p tcp -e 1433

Querying target system called:

johndoesvr9.database.windows.net

Attempting to resolve name to IP address...

Name resolved to 23.100.117.95

querying...
TCP port 1433 (ms-sql-s service): LISTENING

[C:\Users\johndoe\]
>>

Diagnostics: Log your errors

An intermittent problem is sometimes best diagnosed by detection of a general pattern over days or weeks.
Your client can assist in a diagnosis by logging all errors it encounters. You might be able to correlate the log
entries with error data that Azure SQL Database logs itself internally.
Enterprise Library 6 (EntLib60) offers .NET managed classes to assist with logging:
5 - As Easy As Falling Off a Log: Using the Logging Application Block
Diagnostics: Examine system logs for errors
Here are some Transact-SQL SELECT statements that query logs of error and other information.
QUERY OF LOG DESCRIPTION

SELECT e.* The sys.event_log view offers information about individual

FROM sys.event_log AS e events, including some that can cause transient errors or
WHERE e.database_name = 'myDbName' connectivity failures.
AND e.event_category = 'connectivity'
AND 2 >= DateDiff
Ideally you can correlate the start_time or end_time values
with information about when your client program experienced
(hour, e.end_time, GetUtcDate())
problems.
ORDER BY e.event_category,
e.event_type, e.end_time; TIP: You must connect to the master database to run this.

SELECT c.* The sys.database_connection_stats view offers aggregated

FROM sys.database_connection_stats AS c counts of event types, for additional diagnostics.
WHERE c.database_name = 'myDbName'
AND 24 >= DateDiff TIP: You must connect to the master database to run this.
(hour, c.end_time, GetUtcDate())
ORDER BY c.end_time;

Diagnostics: Search for problem events in the SQL Database log

You can search for entries about problem events in the log of Azure SQL Database. Try the following Transact-SQL
SELECT statement in the master database:

SELECT
object_name
,CAST(f.event_data as XML).value
('(/event/@timestamp)[1]', 'datetime2') AS [timestamp]
,CAST(f.event_data as XML).value
('(/event/data[@name="error"]/value)[1]', 'int') AS [error]
,CAST(f.event_data as XML).value
('(/event/data[@name="state"]/value)[1]', 'int') AS [state]
,CAST(f.event_data as XML).value
('(/event/data[@name="is_success"]/value)[1]', 'bit') AS [is_success]
,CAST(f.event_data as XML).value
('(/event/data[@name="database_name"]/value)[1]', 'sysname') AS [database_name]
FROM
sys.fn_xe_telemetry_blob_target_read_file('el', null, null, null) AS f
WHERE
object_name != 'login_event' -- Login events are numerous.
and
'2015-06-21' < CAST(f.event_data as XML).value
('(/event/@timestamp)[1]', 'datetime2')
ORDER BY
[timestamp] DESC
;

A few returned rows from sys.fn_xe_telemetry_blob_target_read_file

Next is what a returned row might look like. The null values shown are often not null in other rows.

object_name timestamp error state is_success database_name

database_xml_deadlock_report 2015-10-16 20:28:01.0090000 NULL NULL NULL AdventureWorks

Enterprise Library 6
Enterprise Library 6 (EntLib60) is a framework of .NET classes that helps you implement robust clients of cloud
services, one of which is the Azure SQL Database service. You can locate topics dedicated to each area in which
EntLib60 can assist by first visiting:
Enterprise Library 6 - April 2013
Retry logic for handling transient errors is one area in which EntLib60 can assist:
4 - Perseverance, Secret of All Triumphs: Using the Transient Fault Handling Application Block

NOTE
The source code for EntLib60 is available for public download. Microsoft has no plans to make further feature updates or
maintenance updates to EntLib.

EntLib60 classes for transient errors and retry

The following EntLib60 classes are particularly useful for retry logic. All these are in, or are further under, the
namespace Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling:
In the namespace **Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling:
RetryPolicy class
ExecuteAction method
ExponentialBackoff class
SqlDatabaseTransientErrorDetectionStrategy class
ReliableSqlConnection class
ExecuteCommand method
In the namespace Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.TestSupport:
AlwaysTransientErrorDetectionStrategy class
NeverTransientErrorDetectionStrategy class
Here are links to information about EntLib60:
Free Book Download: Developer's Guide to Microsoft Enterprise Library, 2nd Edition
Best practices: Retry general guidance has an excellent in-depth discussion of retry logic.
NuGet download of Enterprise Library - Transient Fault Handling application block 6.0
EntLib60: The logging block
The Logging block is a highly flexible and configurable solution that allows you to:
Create and store log messages in a wide variety of locations.
Categorize and filter messages.
Collect contextual information that is useful for debugging and tracing, as well as for auditing and
general logging requirements.
The Logging block abstracts the logging functionality from the log destination so that the application code is
consistent, irrespective of the location and type of the target logging store.
For details see: 5 - As Easy As Falling Off a Log: Using the Logging Application Block
EntLib60 IsTransient method source code
Next, from the SqlDatabaseTransientErrorDetectionStrategy class, is the C# source code for the IsTransient
method. The source code clarifies which errors were considered to be transient and worthy of retry, as of April
2013.
Numerous //comment lines have been removed from this copy to emphasize readability.
public bool IsTransient(Exception ex)
{
if (ex != null)
{
SqlException sqlException;
if ((sqlException = ex as SqlException) != null)
{
// Enumerate through all errors found in the exception.
foreach (SqlError err in sqlException.Errors)
{
switch (err.Number)
{
// SQL Error Code: 40501
// The service is currently busy. Retry the request after 10 seconds.
// Code: (reason code to be decoded).
case ThrottlingCondition.ThrottlingErrorNumber:
// Decode the reason code from the error message to
// determine the grounds for throttling.
var condition = ThrottlingCondition.FromError(err);

// Attach the decoded values as additional attributes to

// the original SQL exception.
sqlException.Data[condition.ThrottlingMode.GetType().Name] =
condition.ThrottlingMode.ToString();
sqlException.Data[condition.GetType().Name] = condition;

return true;

case 10928:
case 10929:
case 10053:
case 10054:
case 10060:
case 40197:
case 40540:
case 40613:
case 40143:
case 233:
case 64:
// DBNETLIB Error Code: 20
// The instance of SQL Server you attempted to connect to
// does not support encryption.
case (int)ProcessNetLibErrorCode.EncryptionNotSupported:
return true;
}
}
}
else if (ex is TimeoutException)
{
return true;
}
else
{
EntityException entityException;
if ((entityException = ex as EntityException) != null)
{
return this.IsTransient(entityException.InnerException);
}
}
}

return false;
}

Next steps
For troubleshooting other common Azure SQL Database connection issues, visit Troubleshoot connection
issues to Azure SQL Database.
SQL Server Connection Pooling (ADO.NET)
Retrying is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of
adding retry behavior to just about anything.
Troubleshoot connection issues to Azure SQL
Database
4/14/2017 • 4 min to read • Edit Online

When the connection to Azure SQL Database fails, you receive error messages. This article is a centralized topic that
helps you troubleshoot Azure SQL Database connectivity issues. It introduces the common causes of connection
issues, recommends a troubleshooting tool that helps you identity the problem, and provides troubleshooting
steps to solve transient errors and persistent or non-transient errors. Finally, it lists all the relevant articles for
Azure SQL Database connectivity issues.
If you encounter the connection issues, try the troubleshoot steps that are described in this article.
If your Azure issue is not addressed in this article, visit the Azure forums on MSDN and the Stack Overflow. You can
post your issue on these forums or to @AzureSupport on Twitter. Also, you can file an Azure support request by
selecting Get support on the Azure support site.

Cause
Connection problems may be caused by any of the following:
Failure to apply best practices and design guidelines during the application design process. See SQL Database
Development Overview to get started.
Azure SQL Database reconfiguration
Firewall settings
Connection time-out
Incorrect login information
Maximum limit reached on some Azure SQL Database resources
Generally, connection issues to Azure SQL Database can be classified as follows:
Transient errors (short-lived or intermittent)
Persistent or non-transient errors (errors that regularly recur)

Try the troubleshooter for Azure SQL Database connectivity issues

If you encounter a specific connection error, try this tool, which will help you quickly identity and resolve your
problem.

Troubleshoot transient errors

When an application connects to an Azure SQL database, you receive the following error message:

Error code 40613: "Database <x> on server <y> is not currently available. Please retry the connection later. If the problem persists, contact
customer support, and provide them the session tracing ID of <z>"

NOTE
This error message is typically transient (short-lived).
This error occurs when the Azure database is being moved (or reconfigured) and your application loses its
connection to the SQL database. SQL database reconfiguration events occurs because of a planned event (for
example, a software upgrade) or an unplanned event (for example, a process crash, or load balancing). Most
reconfiguration events are generally short-lived and should be completed in less than 60 seconds at most.
However, these events can occasionally take longer to finish, such as when a large transaction causes a long-
running recovery.
Steps to resolve transient connectivity issues
1. Check the Microsoft Azure Service Dashboard for any known outages that occurred during the time during
which the errors were reported by the application.
2. Applications that connect to a cloud service such as Azure SQL Database should expect periodic reconfiguration
events and implement retry logic to handle these errors instead of surfacing these as application errors to users.
Review the Transient errors section and the best practices and design guidelines at SQL Database Development
Overview for more information and general retry strategies. Then, see code samples at Connection Libraries for
SQL Database and SQL Server for specifics.
3. As a database approaches its resource limits, it can seem to be a transient connectivity issue. See
Troubleshooting Performance Issues.
4. If connectivity problems continue, or if the duration for which your application encounters the error exceeds 60
seconds or if you see multiple occurrences of the error in a given day, file an Azure support request by selecting
Get Support on the Azure Support site.

Troubleshoot persistent errors (non-transient errors)

If the application persistently fails to connect to Azure SQL Database, it usually indicates an issue with one of the
following:
Firewall configuration. The Azure SQL database or client-side firewall is blocking connections to Azure SQL
Database.
Network reconfiguration on the client side: for example, a new IP address or a proxy server.
User error: for example, mistyped connection parameters, such as the server name in the connection string.
Steps to resolve persistent connectivity issues
1. Set up firewall rules to allow the client IP address. For temporary testing purposes, set up a firewall rule using
0.0.0.0 as the starting IP address range and using 255.255.255.255 as the ending IP address range. This will
open the server to all IP addresses. If this resolves your connectivity issue, remove this rule and create a firewall
rule for a appropriately limited IP address or address range.
2. On all firewalls between the client and the Internet, make sure that port 1433 is open for outbound connections.
Review Configure the Windows Firewall to Allow SQL Server Access and Hybrid Identity Required Ports and
Protocols for additional pointers related to additional ports that you need to open for Azure Active Directory
authentication.
3. Verify your connection string and other connection settings. See the Connection String section in the
connectivity issues topic.
4. Check service health in the dashboard. If you think there’s a regional outage, see Recover from an outage for
steps to recover to a new region.

Next steps
Troubleshoot Azure SQL Database performance issues
Search the documentation on Microsoft Azure
View the latest updates to the Azure SQL Database service

Additional resources
SQL Database Development Overview
General transient fault-handling guidance
Connection libraries for SQL Database and SQL Server
Use C# to create a SQL database with the SQL
Database Library for .NET
4/14/2017 • 5 min to read • Edit Online

Learn how to use C# to create an Azure SQL database with the Microsoft Azure SQL Management Library for .NET.
This article describes how to create a single database with SQL and C#. To create elastic pools, see Create an elastic
pool.
The Azure SQL Database Management Library for .NET provides an Azure Resource Manager-based API that wraps
the Resource Manager-based SQL Database REST API.

NOTE
Many new features of SQL Database are only supported when you are using the Azure Resource Manager deployment
model, so you should always use the latest Azure SQL Database Management Library for .NET (docs | NuGet
Package). The older classic deployment model based libraries are supported for backward compatibility only, so we
recommend you use the newer Resource Manager based libraries.

To complete the steps in this article, you need the following:

An Azure subscription. If you need an Azure subscription simply click FREE ACCOUNT at the top of this page,
and then come back to finish this article.
Visual Studio. For a free copy of Visual Studio, see the Visual Studio Downloads page.

NOTE
This article creates a new, blank SQL database. Modify the CreateOrUpdateDatabase(...) method in the following sample to
copy databases, scale databases, create a database in a pool, etc. For more information, see DatabaseCreateMode and
DatabaseProperties classes.

Create a console app and install the required libraries

1. Start Visual Studio.
2. Click File > New > Project.
3. Create a C# Console Application and name it: SqlDbConsoleApp
To create a SQL database with C#, load the required management libraries (using the package manager console):
1. Click Tools > NuGet Package Manager > Package Manager Console.
2. Type Install-Package Microsoft.Azure.Management.Sql -Pre to install the latest Microsoft Azure SQL Management Library.
3. Type Install-Package Microsoft.Azure.Management.ResourceManager -Pre to install the Microsoft Azure Resource Manager
Library.
4. Type Install-Package Microsoft.Azure.Common.Authentication -Pre to install the Microsoft Azure Common Authentication
Library.
NOTE
The examples in this article use a synchronous form of each API request and block until completion of the REST call on the
underlying service. There are async methods available.

Create a SQL Database server, firewall rule, and SQL database - C#

example
The following sample creates a resource group, server, firewall rule, and a SQL database. See, Create a service
principal to access resources to get the _subscriptionId, _tenantId, _applicationId, and _applicationSecret variables.
Replace the contents of Program.cs with the following, and update the {variables} with your app values (do not
include the {} ).

namespace SqlDbConsoleApp
{
class Program
{
// For details about these four (4) values, see
// https://azure.microsoft.com/documentation/articles/resource-group-authenticate-service-principal/
static string _subscriptionId = "{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}";
static string _tenantId = "{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}";
static string _applicationId = "{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}";
static string _applicationSecret = "{your-password}";

// Authentication token
static AuthenticationResult _token;

// Azure resource variables

static string _resourceGroupName = "{resource-group-name}";
static string _resourceGrouplocation = "{Azure-region}";

static string _serverlocation = _resourceGrouplocation;

static string _serverName = "{server-name}";
static string _serverAdmin = "{server-admin-login}";
static string _serverAdminPassword = "{server-admin-password}";

static string _firewallRuleName = "{firewall-rule-name}";

static string _startIpAddress = "{0.0.0.0}";
static string _endIpAddress = "{255.255.255.255}";

static string _databaseName = "{dbfromcsarticle}";

static string _databaseEdition = DatabaseEditions.Basic;
static string _databasePerfLevel = ""; // "S0", "S1", and so on here for other tiers

static void Main(string[] args)

{
// Authenticate:
_token = GetToken(_tenantId, _applicationId, _applicationSecret);
Console.WriteLine("Token acquired. Expires on:" + _token.ExpiresOn);

// Instantiate management clients:

Console.WriteLine("Database...");
DatabaseCreateOrUpdateResponse dbr = CreateOrUpdateDatabase(_sqlMgmtClient, _resourceGroupName, _serverName, _databaseName,
_databaseEdition, _databasePerfLevel);
Console.WriteLine("Database: " + dbr.Database.Id);

Console.WriteLine("Press any key to continue...");

Console.ReadKey();
}

static ResourceGroup CreateOrUpdateResourceGroup(ResourceManagementClient resourceMgmtClient, string subscriptionId, string

static ServerGetResponse CreateOrUpdateServer(SqlManagementClient sqlMgmtClient, string resourceGroupName, string serverLocation,

string serverName, string serverAdmin, string serverAdminPassword)
{
ServerCreateOrUpdateParameters serverParameters = new ServerCreateOrUpdateParameters()
{
Location = serverLocation,
Properties = new ServerCreateOrUpdateProperties()
{
AdministratorLogin = serverAdmin,
AdministratorLoginPassword = serverAdminPassword,
Version = "12.0"
}
};
ServerGetResponse serverResult = sqlMgmtClient.Servers.CreateOrUpdate(resourceGroupName, serverName, serverParameters);
return serverResult;
}

static FirewallRuleGetResponse CreateOrUpdateFirewallRule(SqlManagementClient sqlMgmtClient, string resourceGroupName, string

serverName, string firewallRuleName, string startIpAddress, string endIpAddress)
{
FirewallRuleCreateOrUpdateParameters firewallParameters = new FirewallRuleCreateOrUpdateParameters()
{
Properties = new FirewallRuleCreateOrUpdateProperties()
{
StartIpAddress = startIpAddress,
EndIpAddress = endIpAddress
}
};
FirewallRuleGetResponse firewallResult = sqlMgmtClient.FirewallRules.CreateOrUpdate(resourceGroupName, serverName, firewallRuleName,
firewallParameters);
return firewallResult;
}

static DatabaseCreateOrUpdateResponse CreateOrUpdateDatabase(SqlManagementClient sqlMgmtClient, string resourceGroupName, string

serverName, string databaseName, string databaseEdition, string databasePerfLevel)
{
// Retrieve the server that will host this database
Server currentServer = sqlMgmtClient.Servers.Get(resourceGroupName, serverName).Server;

// Create a database: configure create or update parameters and properties explicitly

DatabaseCreateOrUpdateParameters newDatabaseParameters = new DatabaseCreateOrUpdateParameters()
{
Location = currentServer.Location,
Properties = new DatabaseCreateOrUpdateProperties()
{
CreateMode = DatabaseCreateMode.Default,
Edition = databaseEdition,
RequestedServiceObjectiveName = databasePerfLevel
}
};
DatabaseCreateOrUpdateResponse dbResponse = sqlMgmtClient.Databases.CreateOrUpdate(resourceGroupName, serverName,
databaseName, newDatabaseParameters);
return dbResponse;
}

private static AuthenticationResult GetToken(string tenantId, string applicationId, string applicationSecret)

Create a service principal to access resources

The following PowerShell script creates the Active Directory (AD) application and the service principal that we need
to authenticate our C# app. The script outputs values we need for the preceding C# sample. For detailed
information, see Use Azure PowerShell to create a service principal to access resources.
# Sign in to Azure.
Add-AzureRmAccount

# Provide these values for your new AAD app.

# $appName is the display name for your app, must be unique in your directory.
# $uri does not need to be a real uri.
# $secret is a password you create.

$appName = "{app-name}"
$uri = "http://{app-name}"
$secret = "{app-password}"

# Create a AAD app

$azureAdApplication = New-AzureRmADApplication -DisplayName $appName -HomePage $Uri -IdentifierUris $Uri -Password $secret

# Create a Service Principal for the app

$svcprincipal = New-AzureRmADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId

# To avoid a PrincipalNotFound error, I pause here for 15 seconds.

Start-Sleep -s 15

# Output the values we need for our C# application to successfully authenticate

Write-Output "Copy these values into the C# sample app"

Write-Output "_subscriptionId:" (Get-AzureRmContext).Subscription.SubscriptionId

Write-Output "_tenantId:" (Get-AzureRmContext).Tenant.TenantId
Write-Output "_applicationId:" $azureAdApplication.ApplicationId.Guid
Write-Output "_applicationSecret:" $secret

Next steps
Now that you've tried SQL Database and set up a database with C#, you're ready for the following articles:
Connect to SQL Database with SQL Server Management Studio and perform a sample T-SQL query

Additional Resources
SQL Database
Database Class
Use In-Memory OLTP to improve your application
performance in SQL Database
4/10/2017 • 4 min to read • Edit Online

In-Memory OLTP can be used to improve the performance of transaction processing, data ingestion, and transient
data scenarios, in Premium Azure SQL Databases without increasing the pricing tier.

NOTE
Learn how Quorum doubles key database’s workload while lowering DTU by 70% with SQL Database

Follow these steps to adopt In-Memory OLTP in your existing database.

Step 1: Ensure you are using a Premium database

In-Memory OLTP is supported only in Premium databases. In-Memory is supported if the returned result is 1 (not
0):

SELECT DatabasePropertyEx(Db_Name(), 'IsXTPSupported');

XTP stands for Extreme Transaction Processing

Step 2: Identify objects to migrate to In-Memory OLTP

SSMS includes a Transaction Performance Analysis Overview report that you can run against a database with
an active workload. The report identifies tables and stored procedures that are candidates for migration to In-
Memory OLTP.
In SSMS, to generate the report:
In the Object Explorer, right-click your database node.
Click Reports > Standard Reports > Transaction Performance Analysis Overview.
For more information, see Determining if a Table or Stored Procedure Should Be Ported to In-Memory OLTP.

Step 3: Create a comparable test database

Suppose the report indicates your database has a table that would benefit from being converted to a memory-
optimized table. We recommend that you first test to confirm the indication by testing.
You need a test copy of your production database. The test database should be at the same service tier level as
your production database.
To ease testing, tweak your test database as follows:
1. Connect to the test database by using SSMS.
2. To avoid needing the WITH (SNAPSHOT) option in queries, set the database option as shown in the
following T-SQL statement:
ALTER DATABASE CURRENT
SET
MEMORY_OPTIMIZED_ELEVATE_TO_SNAPSHOT = ON;

Step 4: Migrate tables

You must create and populate a memory-optimized copy of the table you want to test. You can create it by using
either:
The handy Memory Optimization Wizard in SSMS.
Manual T-SQL.
Memory Optimization Wizard in SSMS
To use this migration option:
1. Connect to the test database with SSMS.
2. In the Object Explorer, right-click on the table, and then click Memory Optimization Advisor.
The Table Memory Optimizer Advisor wizard is displayed.
3. In the wizard, click Migration validation (or the Next button) to see if the table has any unsupported
features that are unsupported in memory-optimized tables. For more information, see:
The memory optimization checklist in Memory Optimization Advisor.
Transact-SQL Constructs Not Supported by In-Memory OLTP.
Migrating to In-Memory OLTP.
4. If the table has no unsupported features, the advisor can perform the actual schema and data migration for you.
Manual T-SQL
To use this migration option:
1. Connect to your test database by using SSMS (or a similar utility).
2. Obtain the complete T-SQL script for your table and its indexes.
In SSMS, right-click your table node.
Click Script Table As > CREATE To > New Query Window.
3. In the script window, add WITH (MEMORY_OPTIMIZED = ON) to the CREATE TABLE statement.
4. If there is a CLUSTERED index, change it to NONCLUSTERED.
5. Rename the existing table by using SP_RENAME.
6. Create the new memory-optimized copy of the table by running your edited CREATE TABLE script.
7. Copy the data to your memory-optimized table by using INSERT...SELECT * INTO:

INSERT INTO <new_memory_optimized_table>

SELECT * FROM <old_disk_based_table>;

Step 5 (optional): Migrate stored procedures

The In-Memory feature can also modify a stored procedure for improved performance.
Considerations with natively compiled stored procedures
A natively compiled stored procedure must have the following options on its T-SQL WITH clause:
NATIVE_COMPILATION
SCHEMABINDING: meaning tables that the stored procedure cannot have their column definitions changed in
any way that would affect the stored procedure, unless you drop the stored procedure.
A native module must use one big ATOMIC blocks for transaction management. There is no role for an explicit
BEGIN TRANSACTION, or for ROLLBACK TRANSACTION. If your code detects a violation of a business rule, it can
terminate the atomic block with a THROW statement.
Typical CREATE PROCEDURE for natively compiled
Typically the T-SQL to create a natively compiled stored procedure is similar to the following template:

CREATE PROCEDURE schemaname.procedurename

@param1 type1, …
WITH NATIVE_COMPILATION, SCHEMABINDING
AS
BEGIN ATOMIC WITH
(TRANSACTION ISOLATION LEVEL = SNAPSHOT,
LANGUAGE = N'your_language__see_sys.languages'
)
…
END;

For the TRANSACTION_ISOLATION_LEVEL, SNAPSHOT is the most common value for the natively compiled
stored procedure. However, a subset of the other values are also supported:
REPEATABLE READ
SERIALIZABLE
The LANGUAGE value must be present in the sys.languages view.
How to migrate a stored procedure
The migration steps are:
1. Obtain the CREATE PROCEDURE script to the regular interpreted stored procedure.
2. Rewrite its header to match the previous template.
3. Ascertain whether the stored procedure T-SQL code uses any features that are not supported for natively
compiled stored procedures. Implement workarounds if necessary.
For details see Migration Issues for Natively Compiled Stored Procedures.
4. Rename the old stored procedure by using SP_RENAME. Or simply DROP it.
5. Run your edited CREATE PROCEDURE T-SQL script.

Step 6: Run your workload in test

Run a workload in your test database that is similar to the workload that runs in your production database. This
should reveal the performance gain achieved by your use of the In-Memory feature for tables and stored
procedures.
Major attributes of the workload are:
Number of concurrent connections.
Read/write ratio.
To tailor and run the test workload, consider using the handy ostress.exe tool, which illustrated in here.
To minimize network latency, run your test in the same Azure geographic region where the database exists.

Step 7: Post-implementation monitoring

Consider monitoring the performance effects of your In-Memory implementations in production:
Monitor In-Memory Storage.
Monitoring Azure SQL Database using dynamic management views

Related links
In-Memory OLTP (In-Memory Optimization)
Introduction to Natively Compiled Stored Procedures
Memory Optimization Advisor
Get started with elastic database tools
4/21/2017 • 3 min to read • Edit Online

This document introduces you to the developer experience by helping you to run the sample app. The
sample creates a simple sharded application and explores key capabilities of elastic database tools. The
sample demonstrates functions of the elastic database client library.
To install the library, go to Microsoft.Azure.SqlDatabase.ElasticScale.Client. The library is installed with the
sample app that's described in the following section.

Prerequisites
Visual Studio 2012 or later with C#. Download a free version at Visual Studio Downloads.
NuGet 2.7 or later. To get the latest version, see Installing NuGet.

Download and run the sample app

The Elastic DB Tools for Azure SQL - Getting Started sample application illustrates the most important
aspects of the development experience for sharded applications that use elastic database tools. It focuses on
key use cases for shard map management, data-dependent routing, and multi-shard querying. To download
and run the sample, follow these steps:
1. Download the Elastic DB Tools for Azure SQL - Getting Started sample from MSDN. Unzip the sample
to a location that you choose.
2. To create a project, open the ElasticScaleStarterKit.sln solution from the C# directory.
3. In the solution for the sample project, open the app.config file. Then follow the instructions in the
file to add your Azure SQL Database server name and your sign-in information (user name and
password).
4. Build and run the application. When prompted, enable Visual Studio to restore the NuGet packages
of the solution. This downloads the latest version of the elastic database client library from NuGet.
5. Experiment with the different options to learn more about the client library capabilities. Note the
steps the application takes in the console output and feel free to explore the code behind the scenes.
Congratulations--you have successfully built and run your first sharded application by using elastic
database tools on SQL Database. Use Visual Studio or SQL Server Management Studio to connect to your
SQL database and take a quick look at the shards that the sample created. You will notice new sample shard
databases and a shard map manager database that the sample has created.

IMPORTANT
We recommend that you always use the latest version of Management Studio so that you stay synchronized with
updates to Azure and SQL Database. Update SQL Server Management Studio.

Key pieces of the code sample

Managing shards and shard maps: The code illustrates how to work with shards, ranges, and
mappings in the file ShardManagementUtils.cs. For more information, see Scale out databases
with the shard map manager.
Data-dependent routing: Routing of transactions to the right shard is shown in
DataDependentRoutingSample.cs. For more information, see Data-dependent routing.
Querying over multiple shards: Querying across shards is illustrated in the file
MultiShardQuerySample.cs. For more information, see Multi-shard querying.
Adding empty shards: The iterative adding of new empty shards is performed by the code in the
file CreateShardSample.cs. For more information, see Scale out databases with the shard map
manager.
Other elastic scale operations
Splitting an existing shard: The capability to split shards is provided by the split-merge tool. For
more information, see Moving data between scaled-out cloud databases.
Merging existing shards: Shard merges are also performed by using the split-merge tool. For
more information, see Moving data between scaled-out cloud databases.

Cost
The elastic database tools are free. When you use elastic database tools, you don't receive any additional
charges on top of the cost of your Azure usage.
For example, the sample application creates new databases. The cost for this depends on the SQL Database
edition you choose and the Azure usage of your application.
For pricing information, see SQL Database pricing details.

Next steps
For more information about elastic database tools, see the following pages:
Elastic database tools documentation map
Code samples:
Elastic DB Tools for Azure SQL - Getting Started
Elastic DB Tools for Azure SQL - Entity Framework Integration
Shard Elasticity on Script Center
Blog: Elastic Scale announcement
Microsoft Virtual Academy: Implementing Scale-Out Using Sharding with the Elastic Database Client
Library Video
Channel 9: Elastic Scale overview video
Discussion forum: Azure SQL Database forum
To measure performance: Performance counters for shard map manager
Deploy a split-merge service
1/17/2017 • 12 min to read • Edit Online

The split-merge tool lets you move data between sharded databases. See Moving data between scaled-out cloud
databases

Download the Split-Merge packages

1. Download the latest NuGet version from NuGet.
2. Open a command prompt and navigate to the directory where you downloaded nuget.exe. The download
includes PowerShell commmands.
3. Download the latest Split-Merge package into the current directory with the below command:
nuget install Microsoft.Azure.SqlDatabase.ElasticScale.Service.SplitMerge

The files are placed in a directory named

Microsoft.Azure.SqlDatabase.ElasticScale.Service.SplitMerge.x.x.xxx.x where x.x.xxx.x reflects the version
number. Find the split-merge Service files in the content\splitmerge\service sub-directory, and the Split-Merge
PowerShell scripts (and required client .dlls) in the content\splitmerge\powershell sub-directory.

Prerequisites
1. Create an Azure SQL DB database that will be used as the split-merge status database. Go to the Azure portal.
Create a new SQL Database. Give the database a name and create a new administrator and password. Be sure
to record the name and password for later use.
2. Ensure that your Azure SQL DB server allows Azure Services to connect to it. In the portal, in the Firewall
Settings, ensure that the Allow access to Azure Services setting is set to On. Click the "save" icon.

3. Create an Azure Storage account that will be used for diagnostics output. Go to the Azure Portal. In the left bar,
click New, click Data + Storage, then Storage.
4. Create an Azure Cloud Service that will contain your Split-Merge service. Go to the Azure Portal. In the left bar,
click New, then Compute, Cloud Service, and Create.

Configure your Split-Merge service

Split-Merge service configuration
1. In the folder where you downloaded the Split-Merge assemblies, create a copy of the
ServiceConfiguration.Template.cscfg file that shipped alongside SplitMergeService.cspkg and rename it
ServiceConfiguration.cscfg.
2. Open ServiceConfiguration.cscfg in a text editor such as Visual Studio that validates inputs such as the
format of certificate thumbprints.
3. Create a new database or choose an existing database to serve as the status database for Split-Merge
operations and retrieve the connection string of that database.

IMPORTANT
At this time, the status database must use the Latin collation (SQL_Latin1_General_CP1_CI_AS). For more
information, see Windows Collation Name (Transact-SQL).

With Azure SQL DB, the connection string typically is of the form:

Server=myservername.database.windows.net; Database=mydatabasename;User ID=myuserID; Password=mypassword;

Encrypt=True; Connection Timeout=30

4. Enter this connection string in the cscfg file in both the SplitMergeWeb and SplitMergeWorker role
sections in the ElasticScaleMetadata setting.
5. For the SplitMergeWorker role, enter a valid connection string to Azure storage for the
WorkerRoleSynchronizationStorageAccountConnectionString setting.
Configure security
For detailed instructions to configure the security of the service, refer to the Split-Merge security configuration.
For the purposes of a simple test deployment for this tutorial, a minimal set of configuration steps will be
performed to get the service up and running. These steps enable only the one machine/account executing them to
communicate with the service.
Create a self-signed certificate
Create a new directory and from this directory execute the following command using a Developer Command
Prompt for Visual Studio window:

makecert ^
-n "CN=*.cloudapp.net" ^
-r -cy end -sky exchange -eku "1.3.6.1.5.5.7.3.1,1.3.6.1.5.5.7.3.2" ^
-a sha1 -len 2048 ^
-sr currentuser -ss root ^
-sv MyCert.pvk MyCert.cer

You are asked for a password to protect the private key. Enter a strong password and confirm it. You are then
prompted for the password to be used once more after that. Click Yes at the end to import it to the Trusted
Certification Authorities Root store.
Create a PFX file
Execute the following command from the same window where makecert was executed; use the same password
that you used to create the certificate:

pvk2pfx -pvk MyCert.pvk -spc MyCert.cer -pfx MyCert.pfx -pi <password>

Import the client certificate into the personal store

1. In Windows Explorer, double-click MyCert.pfx.
2. In the Certificate Import Wizard select Current User and click Next.
3. Confirm the file path and click Next.
4. Type the password, leave Include all extended properties checked and click Next.
5. Leave Automatically select the certificate store[…] checked and click Next.
6. Click Finish and OK.
Upload the PFX file to the cloud service
1. Go to the Azure Portal.
2. Select Cloud Services.
3. Select the cloud service you created above for the Split/Merge service.
4. Click Certificates on the top menu.
5. Click Upload in the bottom bar.
6. Select the PFX file and enter the same password as above.
7. Once completed, copy the certificate thumbprint from the new entry in the list.
Update the service configuration file
Paste the certificate thumbprint copied above into the thumbprint/value attribute of these settings. For the worker
role:

<Setting name="DataEncryptionPrimaryCertificateThumbprint" value="" />

For the web role:

<Setting name="AdditionalTrustedRootCertificationAuthorities" value="" />

Please note that for production deployments separate certificates should be used for the CA, for encryption, the
Server certificate and client certificates. For detailed instructions on this, see Security Configuration.

Deploy your service

1. Go to the Azure portal.
2. Click the Cloud Services tab on the left, and select the cloud service that you created earlier.
3. Click Dashboard.
4. Choose the staging environment, then click Upload a new staging deployment.

5. In the dialog box, enter a deployment label. For both 'Package' and 'Configuration', click 'From Local' and
choose the SplitMergeService.cspkg file and your .cscfg file that you configured earlier.
6. Ensure that the checkbox labeled Deploy even if one or more roles contain a single instance is checked.
7. Hit the tick button in the bottom right to begin the deployment. Expect it to take a few minutes to complete.
Troubleshoot the deployment
If your web role fails to come online, it is likely a problem with the security configuration. Check that the SSL is
configured as described above.
If your worker role fails to come online, but your web role succeeds, it is most likely a problem connecting to the
status database that you created earlier.
Make sure that the connection string in your .cscfg is accurate.
Check that the server and database exist, and that the user id and password are correct.
For Azure SQL DB, the connection string should be of the form:

Server=myservername.database.windows.net; Database=mydatabasename;User ID=myuserID; Password=mypassword; Encrypt=True;

Connection Timeout=30

Ensure that the server name does not begin with https://.
Ensure that your Azure SQL DB server allows Azure Services to connect to it. To do this, open
https://manage.windowsazure.com, click “SQL Databases” on the left, click “Servers” at the top, and select your
server. Click Configure at the top and ensure that the Azure Services setting is set to “Yes”. (See the
Prerequisites section at the top of this article).

Test the service deployment

Connect with a web browser
Determine the web endpoint of your Split-Merge service. You can find this in the Azure Classic Portal by going to
the Dashboard of your cloud service and looking under Site URL on the right side. Replace http:// with https://
since the default security settings disable the HTTP endpoint. Load the page for this URL into your browser.
Test with PowerShell scripts
The deployment and your environment can be tested by running the included sample PowerShell scripts.
The script files included are:
1. SetupSampleSplitMergeEnvironment.ps1 - sets up a test data tier for Split/Merge (see table below for
detailed description)
2. ExecuteSampleSplitMerge.ps1 - executes test operations on the test data tier (see table below for detailed
description)
3. GetMappings.ps1 - top-level sample script that prints out the current state of the shard mappings.
4. ShardManagement.psm1 - helper script that wraps the ShardManagement API
5. SqlDatabaseHelpers.psm1 - helper script for creating and managing SQL databases

POWERSHELL FILE STEPS

SETUPSAMPLESPLITMERGEENVIRONMENT.PS1 1. Creates a shard map manager database

2. Creates 2 shard databases.

3. Creates a shard map for those database (deletes any

existing shard maps on those databases).

4. Creates a small sample table in both the shards, and

populates the table in one of the shards.

5. Declares the SchemaInfo for the sharded table.

POWERSHELL FILE STEPS

EXECUTESAMPLESPLITMERGE.PS1 1. Sends a split request to the Split-Merge Service web

frontend, which splits half the data from the first shard to
the second shard.

2. Polls the web frontend for the split request status and
waits until the request completes.

3. Sends a merge request to the Split-Merge Service web

frontend, which moves the data from the second shard
back to the first shard.

4. Polls the web frontend for the merge request status

and waits until the request completes.

Use PowerShell to verify your deployment

1. Open a new PowerShell window and navigate to the directory where you downloaded the Split-Merge
package, and then navigate into the “powershell” directory.
2. Create an Azure SQL database server (or choose an existing server) where the shard map manager and
shards will be created.

NOTE
The SetupSampleSplitMergeEnvironment.ps1 script creates all these databases on the same server by default to
keep the script simple. This is not a restriction of the Split-Merge Service itself.
A SQL authentication login with read/write access to the DBs will be needed for the Split-Merge service to
move data and update the shard map. Since the Split-Merge Service runs in the cloud, it does not currently
support Integrated Authentication.
Make sure the Azure SQL server is configured to allow access from the IP address of the machine running
these scripts. You can find this setting under the Azure SQL server / configuration / allowed IP addresses.
3. Execute the SetupSampleSplitMergeEnvironment.ps1 script to create the sample environment.
Running this script will wipe out any existing shard map management data structures on the shard map
manager database and the shards. It may be useful to rerun the script if you wish to re-initialize the shard
map or shards.
Sample command line:

.\SetupSampleSplitMergeEnvironment.ps1

-UserName 'mysqluser'
-Password 'MySqlPassw0rd'
-ShardMapManagerServerName 'abcdefghij.database.windows.net'

4. Execute the Getmappings.ps1 script to view the mappings that currently exist in the sample environment.

.\GetMappings.ps1

-UserName 'mysqluser'
-Password 'MySqlPassw0rd'
-ShardMapManagerServerName 'abcdefghij.database.windows.net'

5. Execute the ExecuteSampleSplitMerge.ps1 script to execute a split operation (moving half the data on the
first shard to the second shard) and then a merge operation (moving the data back onto the first shard). If
you configured SSL and left the http endpoint disabled, ensure that you use the https:// endpoint instead.
Sample command line:

.\ExecuteSampleSplitMerge.ps1

-UserName 'mysqluser'
-Password 'MySqlPassw0rd'
-ShardMapManagerServerName 'abcdefghij.database.windows.net'
-SplitMergeServiceEndpoint 'https://mysplitmergeservice.cloudapp.net'
-CertificateThumbprint '0123456789abcdef0123456789abcdef01234567'

If you receive the below error, it is most likely a problem with your Web endpoint’s certificate. Try
connecting to the Web endpoint with your favorite Web browser and check if there is a certificate error.

Invoke-WebRequest : The underlying connection was closed: Could not establish trust relationship for the SSL/TLSsecure channel.

If it succeeded, the output should look like the below:

> .\ExecuteSampleSplitMerge.ps1 -UserName 'mysqluser' -Password 'MySqlPassw0rd' -ShardMapManagerServerName
'abcdefghij.database.windows.net' -SplitMergeServiceEndpoint 'http://mysplitmergeservice.cloudapp.net' -CertificateThumbprint
0123456789abcdef0123456789abcdef01234567
> Sending split request
> Began split operation with id dc68dfa0-e22b-4823-886a-9bdc903c80f3
> Polling split-merge request status. Press Ctrl-C to end
> Progress: 0% | Status: Queued | Details: [Informational] Queued request
> Progress: 5% | Status: Starting | Details: [Informational] Starting split-merge state machine for request.
> Progress: 5% | Status: Starting | Details: [Informational] Performing data consistency checks on target shards.
> Progress: 20% | Status: CopyingReferenceTables | Details: [Informational] Moving reference tables from source to target shard.
> Progress: 20% | Status: CopyingReferenceTables | Details: [Informational] Waiting for reference tables copy completion.
> Progress: 20% | Status: CopyingReferenceTables | Details: [Informational] Moving reference tables from source to target shard.
> Progress: 44% | Status: CopyingShardedTables | Details: [Informational] Moving key range [100:110) of Sharded tables
> Progress: 44% | Status: CopyingShardedTables | Details: [Informational] Successfully copied key range [100:110) for table [dbo].
[MyShardedTable]
> ...
> ...
> Progress: 90% | Status: Completing | Details: [Informational] Successfully deleted shardlets in table [dbo].[MyShardedTable].
> Progress: 90% | Status: Completing | Details: [Informational] Deleting any temp tables that were created while processing the
request.
> Progress: 100% | Status: Succeeded | Details: [Informational] Successfully processed request.
> Sending merge request
> Began merge operation with id 6ffc308f-d006-466b-b24e-857242ec5f66
> Polling request status. Press Ctrl-C to end
> Progress: 0% | Status: Queued | Details: [Informational] Queued request
> Progress: 5% | Status: Starting | Details: [Informational] Starting split-merge state machine for request.
> Progress: 5% | Status: Starting | Details: [Informational] Performing data consistency checks on target shards.
> Progress: 20% | Status: CopyingReferenceTables | Details: [Informational] Moving reference tables from source to target shard.
> Progress: 44% | Status: CopyingShardedTables | Details: [Informational] Moving key range [100:110) of Sharded tables
> Progress: 44% | Status: CopyingShardedTables | Details: [Informational] Successfully copied key range [100:110) for table [dbo].
[MyShardedTable]
> ...
> ...
> Progress: 90% | Status: Completing | Details: [Informational] Successfully deleted shardlets in table [dbo].[MyShardedTable].
> Progress: 90% | Status: Completing | Details: [Informational] Deleting any temp tables that were created while processing the
request.
> Progress: 100% | Status: Succeeded | Details: [Informational] Successfully processed request.
>

6. Experiment with other data types! All of these scripts take an optional -ShardKeyType parameter that allows
you to specify the key type. The default is Int32, but you can also specify Int64, Guid, or Binary.

Create requests
The service can be used either by using the web UI or by importing and using the SplitMerge.psm1 PowerShell
module which will submit your requests through the web role.
The service can move data in both sharded tables and reference tables. A sharded table has a sharding key column
and has different row data on each shard. A reference table is not sharded so it contains the same row data on
every shard. Reference tables are useful for data that does not change often and is used to JOIN with sharded
tables in queries.
In order to perform a split-merge operation, you must declare the sharded tables and reference tables that you
want to have moved. This is accomplished with the SchemaInfo API. This API is in the
Microsoft.Azure.SqlDatabase.ElasticScale.ShardManagement.Schema namespace.
1. For each sharded table, create a ShardedTableInfo object describing the table’s parent schema name
(optional, defaults to “dbo”), the table name, and the column name in that table that contains the sharding key.
2. For each reference table, create a ReferenceTableInfo object describing the table’s parent schema name
(optional, defaults to “dbo”) and the table name.
3. Add the above TableInfo objects to a new SchemaInfo object.
4. Get a reference to a ShardMapManager object, and call GetSchemaInfoCollection.
5. Add the SchemaInfo to the SchemaInfoCollection, providing the shard map name.
An example of this can be seen in the SetupSampleSplitMergeEnvironment.ps1 script.
The Split-Merge service does not create the target database (or schema for any tables in the database) for you.
They must be pre-created before sending a request to the service.

Troubleshooting
You may see the below message when running the sample powershell scripts:

Invoke-WebRequest : The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

This error means that your SSL certificate is not configured correctly. Please follow the instructions in section
'Connecting with a web browser'.
If you cannot submit requests you may see this:

[Exception] System.Data.SqlClient.SqlException (0x80131904): Could not find stored procedure 'dbo.InsertRequest'.

In this case, check your configuration file, in particular the setting for
WorkerRoleSynchronizationStorageAccountConnectionString. This error typically indicates that the worker
role could not successfully initialize the metadata database on first use.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Split-merge security configuration
1/17/2017 • 12 min to read • Edit Online

To use the Split/Merge service, you must correctly configure security. The service is part of the Elastic Scale feature
of Microsoft Azure SQL Database. For more information, see Elastic Scale Split and Merge Service Tutorial.

Configuring certificates
Certificates are configured in two ways.
1. To Configure the SSL Certificate
2. To Configure Client Certificates

To obtain certificates
Certificates can be obtained from public Certificate Authorities (CAs) or from the Windows Certificate Service.
These are the preferred methods to obtain certificates.
If those options are not available, you can generate self-signed certificates.

Tools to generate certificates

makecert.exe
pvk2pfx.exe
To run the tools
From a Developer Command Prompt for Visual Studios, see Visual Studio Command Prompt
If installed, go to:

%ProgramFiles(x86)%\Windows Kits\x.y\bin\x86

Get the WDK from Windows 8.1: Download kits and tools

To configure the SSL certificate

A SSL certificate is required to encrypt the communication and authenticate the server. Choose the most
applicable of the three scenarios below, and execute all its steps:
Create a new self-signed certificate
1. Create a Self-Signed Certificate
2. Create PFX file for Self-Signed SSL Certificate
3. Upload SSL Certificate to Cloud Service
4. Update SSL Certificate in Service Configuration File
5. Import SSL Certification Authority
To use an existing certificate from the certificate store
1. Export SSL Certificate From Certificate Store
2. Upload SSL Certificate to Cloud Service
3. Update SSL Certificate in Service Configuration File
To use an existing certificate in a PFX file
1. Upload SSL Certificate to Cloud Service
2. Update SSL Certificate in Service Configuration File

To configure client certificates

Client certificates are required in order to authenticate requests to the service. Choose the most applicable of the
three scenarios below, and execute all its steps:
Turn off client certificates
1. Turn Off Client Certificate-Based Authentication
Issue new self-signed client certificates
1. Create a Self-Signed Certification Authority
2. Upload CA Certificate to Cloud Service
3. Update CA Certificate in Service Configuration File
4. Issue Client Certificates
5. Create PFX files for Client Certificates
6. Import Client Certificate
7. Copy Client Certificate Thumbprints
8. Configure Allowed Clients in the Service Configuration File
Use existing client certificates
1. Find CA Public Key
2. Upload CA Certificate to Cloud Service
3. Update CA Certificate in Service Configuration File
4. Copy Client Certificate Thumbprints
5. Configure Allowed Clients in the Service Configuration File
6. Configure Client Certificate Revocation Check

Allowed IP addresses
Access to the service endpoints can be restricted to specific ranges of IP addresses.

To configure encryption for the store

A certificate is required to encrypt the credentials that are stored in the metadata store. Choose the most
applicable of the three scenarios below, and execute all its steps:
Use a new self-signed certificate
1. Create a Self-Signed Certificate
2. Create PFX file for Self-Signed Encryption Certificate
3. Upload Encryption Certificate to Cloud Service
4. Update Encryption Certificate in Service Configuration File
Use an existing certificate from the certificate store
1. Export Encryption Certificate From Certificate Store
2. Upload Encryption Certificate to Cloud Service
3. Update Encryption Certificate in Service Configuration File
Use an existing certificate in a PFX file
1. Upload Encryption Certificate to Cloud Service
2. Update Encryption Certificate in Service Configuration File

The default configuration

The default configuration denies all access to the HTTP endpoint. This is the recommended setting, since the
requests to these endpoints may carry sensitive information like database credentials. The default configuration
allows all access to the HTTPS endpoint. This setting may be restricted further.
Changing the Configuration
The group of access control rules that apply to and endpoint are configured in the section in the service
configuration file.

The rules in an access control group are configured in a section of the service configuration file.
The format is explained in Network Access Control Lists documentation. For example, to allow only IPs in the
range 100.100.0.0 to 100.100.255.255 to access the HTTPS endpoint, the rules would look like this:

Denial of service prevention

There are two different mechanisms supported to detect and prevent Denial of Service attacks:
Restrict number of concurrent requests per remote host (off by default)
Restrict rate of access per remote host (on by default)
These are based on the features further documented in Dynamic IP Security in IIS. When changing this
configuration beware of the following factors:
The behavior of proxies and Network Address Translation devices over the remote host information
Each request to any resource in the web role is considered (e.g. loading scripts, images, etc)

Restricting number of concurrent accesses

The settings that configure this behavior are:

<Setting name="DynamicIpRestrictionDenyByConcurrentRequests" value="false" />

Change DynamicIpRestrictionDenyByConcurrentRequests to true to enable this protection.

Restricting rate of access

The settings that configure this behavior are:
<Setting name="DynamicIpRestrictionDenyByRequestRate" value="true" />
<Setting name="DynamicIpRestrictionMaxRequests" value="100" />
<Setting name="DynamicIpRestrictionRequestIntervalInMilliseconds" value="2000" />

Configuring the response to a denied request

The following setting configures the response to a denied request:

<Setting name="DynamicIpRestrictionDenyAction" value="AbortRequest" />

Refer to the documentation for Dynamic IP Security in IIS for other supported values.

Operations for configuring service certificates

This topic is for reference only. Please follow the configuration steps outlined in:
Configure the SSL certificate
Configure client certificates

Create a self-signed certificate

Execute:

makecert ^
-n "CN=myservice.cloudapp.net" ^
-e MM/DD/YYYY ^
-r -cy end -sky exchange -eku "1.3.6.1.5.5.7.3.1" ^
-a sha1 -len 2048 ^
-sv MySSL.pvk MySSL.cer

To customize:
-n with the service URL. Wildcards ("CN=*.cloudapp.net") and alternative names
("CN=myservice1.cloudapp.net, CN=myservice2.cloudapp.net") are supported.
-e with the certificate expiration date Create a strong password and specify it when prompted.

Create PFX file for self-signed SSL certificate

Execute:

pvk2pfx -pvk MySSL.pvk -spc MySSL.cer

Enter password and then export certificate with these options:

Yes, export the private key
Export all extended properties

Export SSL certificate from certificate store

Find certificate
Click Actions -> All tasks -> Export…
Export certificate into a .PFX file with these options:
Yes, export the private key
Include all certificates in the certification path if possible *Export all extended properties

Upload SSL certificate to cloud service

Upload certificate with the existing or generated .PFX file with the SSL key pair:
Enter the password protecting the private key information

Update SSL certificate in service configuration file

Update the thumbprint value of the following setting in the service configuration file with the thumbprint of the
certificate uploaded to the cloud service:

<Certificate name="SSL" thumbprint="" thumbprintAlgorithm="sha1" />

Import SSL certification authority

Follow these steps in all account/machine that will communicate with the service:
Double-click the .CER file in Windows Explorer
In the Certificate dialog, click Install Certificate…
Import certificate into the Trusted Root Certification Authorities store

Turn off client certificate-based authentication

Only client certificate-based authentication is supported and disabling it will allow for public access to the service
endpoints, unless other mechanisms are in place (e.g. Microsoft Azure Virtual Network).
Change these settings to false in the service configuration file to turn the feature off:

<Setting name="SetupWebAppForClientCertificates" value="false" />

Then, copy the same thumbprint as the SSL certificate in the CA certificate setting:

<Certificate name="CA" thumbprint="" thumbprintAlgorithm="sha1" />

Create a self-signed certification authority

Execute the following steps to create a self-signed certificate to act as a Certification Authority:

makecert ^
-n "CN=MyCA" ^
-e MM/DD/YYYY ^
-r -cy authority -h 1 ^
-a sha1 -len 2048 ^
-sr localmachine -ss my ^
MyCA.cer

To customize it
-e with the certification expiration date
Find CA public key
All client certificates must have been issued by a Certification Authority trusted by the service. Find the public key
to the Certification Authority that issued the client certificates that are going to be used for authentication in order
to upload it to the cloud service.
If the file with the public key is not available, export it from the certificate store:
Find certificate
Search for a client certificate issued by the same Certification Authority
Double-click the certificate.
Select the Certification Path tab in the Certificate dialog.
Double-click the CA entry in the path.
Take notes of the certificate properties.
Close the Certificate dialog.
Find certificate
Search for the CA noted above.
Click Actions -> All tasks -> Export…
Export certificate into a .CER with these options:
No, do not export the private key
Include all certificates in the certification path if possible.
Export all extended properties.

Upload CA certificate to cloud service

Upload certificate with the existing or generated .CER file with the CA public key.

Update CA certificate in service configuration file

Update the thumbprint value of the following setting in the service configuration file with the thumbprint of the
certificate uploaded to the cloud service:

<Certificate name="CA" thumbprint="" thumbprintAlgorithm="sha1" />

Update the value of the following setting with the same thumbprint:

<Setting name="AdditionalTrustedRootCertificationAuthorities" value="" />

Issue client certificates

Each individual authorized to access the service should have a client certificate issued for his/hers exclusive use
and should choose his/hers own strong password to protect its private key.
The following steps must be executed in the same machine where the self-signed CA certificate was generated
and stored:
makecert ^
-n "CN=My ID" ^
-e MM/DD/YYYY ^
-cy end -sky exchange -eku "1.3.6.1.5.5.7.3.2" ^
-a sha1 -len 2048 ^
-in "MyCA" -ir localmachine -is my ^
-sv MyID.pvk MyID.cer

Customizing:
-n with an ID for to the client that will be authenticated with this certificate
-e with the certificate expiration date
MyID.pvk and MyID.cer with unique filenames for this client certificate
This command will prompt for a password to be created and then used once. Use a strong password.

Create PFX files for client certificates

For each generated client certificate, execute:

pvk2pfx -pvk MyID.pvk -spc MyID.cer

Customizing:

MyID.pvk and MyID.cer with the filename for the client certificate

Enter password and then export certificate with these options:

Yes, export the private key
Export all extended properties
The individual to whom this certificate is being issued should choose the export password

Import client certificate

Each individual for whom a client certificate has been issued should import the key pair in the machines he/she
will use to communicate with the service:
Double-click the .PFX file in Windows Explorer
Import certificate into the Personal store with at least this option:
Include all extended properties checked

Copy client certificate thumbprints

Each individual for whom a client certificate has been issued must follow these steps in order to obtain the
thumbprint of his/hers certificate which will be added to the service configuration file:
Run certmgr.exe
Select the Personal tab
Double-click the client certificate to be used for authentication
In the Certificate dialog that opens, select the Details tab
Make sure Show is displaying All
Select the field named Thumbprint in the list
Copy the value of the thumbprint ** Delete non-visible Unicode characters in front of the first digit ** Delete all
spaces

Configure Allowed clients in the service configuration file

Update the value of the following setting in the service configuration file with a comma-separated list of the
thumbprints of the client certificates allowed access to the service:

<Setting name="AllowedClientCertificateThumbprints" value="" />

Configure client certificate revocation check

The default setting does not check with the Certification Authority for client certificate revocation status. To turn
on the checks, if the Certification Authority which issued the client certificates supports such checks, change the
following setting with one of the values defined in the X509RevocationMode Enumeration:

<Setting name="ClientCertificateRevocationCheck" value="NoCheck" />

Create PFX file for self-signed encryption certificates

For an encryption certificate, execute:

pvk2pfx -pvk MyID.pvk -spc MyID.cer

Customizing:

MyID.pvk and MyID.cer with the filename for the encryption certificate

Enter password and then export certificate with these options:

Yes, export the private key
Export all extended properties
You will need the password when uploading the certificate to the cloud service.

Export encryption certificate from certificate store

Upload encryption certificate to cloud service

Upload certificate with the existing or generated .PFX file with the encryption key pair:
Enter the password protecting the private key information

Update encryption certificate in service configuration file

Update the thumbprint value of the following settings in the service configuration file with the thumbprint of the
certificate uploaded to the cloud service:

<Certificate name="DataEncryptionPrimary" thumbprint="" thumbprintAlgorithm="sha1" />

Common certificate operations

Configure the SSL certificate
Configure client certificates

Find certificate
Follow these steps:
1. Run mmc.exe.
2. File -> Add/Remove Snap-in…
3. Select Certificates.
4. Click Add.
5. Choose the certificate store location.
6. Click Finish.
7. Click OK.
8. Expand Certificates.
9. Expand the certificate store node.
10. Expand the Certificate child node.
11. Select a certificate in the list.

Export certificate
In the Certificate Export Wizard:
1. Click Next.
2. Select Yes, then Export the private key.
3. Click Next.
4. Select the desired output file format.
5. Check the desired options.
6. Check Password.
7. Enter a strong password and confirm it.
8. Click Next.
9. Type or browse a filename where to store the certificate (use a .PFX extension).
10. Click Next.
11. Click Finish.
12. Click OK.

Import certificate
In the Certificate Import Wizard:
1. Select the store location.
Select Current User if only processes running under current user will access the service
Select Local Machine if other processes in this computer will access the service
2. Click Next.
3. If importing from a file, confirm the file path.
4. If importing a .PFX file:
a. Enter the password protecting the private key
b. Select import options
5. Select "Place" certificates in the following store
6. Click Browse.
7. Select the desired store.
8. Click Finish.
If the Trusted Root Certification Authority store was chosen, click Yes.
9. Click OK on all dialog windows.

Upload certificate
In the Azure Portal
1. Select Cloud Services.
2. Select the cloud service.
3. On the top menu, click Certificates.
4. On the bottom bar, click Upload.
5. Select the certificate file.
6. If it is a .PFX file, enter the password for the private key.
7. Once completed, copy the certificate thumbprint from the new entry in the list.

Other security considerations

The SSL settings described in this document encrypt communication between the service and its clients when the
HTTPS endpoint is used. This is important since credentials for database access and potentially other sensitive
information are contained in the communication. Note, however, that the service persists internal status, including
credentials, in its internal tables in the Microsoft Azure SQL database that you have provided for metadata storage
in your Microsoft Azure subscription. That database was defined as part of the following setting in your service
configuration file (.CSCFG file):

<Setting name="ElasticScaleMetadata" value="Server=…" />

Credentials stored in this database are encrypted. However, as a best practice, ensure that both web and worker
roles of your service deployments are kept up to date and secure as they both have access to the metadata
database and the certificate used for encryption and decryption of stored credentials.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Adding a shard using Elastic Database tools
1/17/2017 • 2 min to read • Edit Online

To add a shard for a new range or key

Applications often need to simply add new shards to handle data that is expected from new keys or key ranges,
for a shard map that already exists. For example, an application sharded by Tenant ID may need to provision a
new shard for a new tenant, or data sharded monthly may need a new shard provisioned before the start of each
new month.
If the new range of key values is not already part of an existing mapping, it is very simple to add the new shard
and associate the new key or range to that shard.
Example: adding a shard and its range to an existing shard map
This sample uses the TryGetShard the CreateShard, CreateRangeMapping methods, and creates an instance of the
ShardLocation class. In the sample below, a database named sample_shard_2 and all necessary schema objects
inside of it have been created to hold range [300, 400).

// sm is a RangeShardMap object.
// Add a new shard to hold the range being added.
Shard shard2 = null;

if (!sm.TryGetShard(new ShardLocation(shardServer, "sample_shard_2"),out shard2))

{
shard2 = sm.CreateShard(new ShardLocation(shardServer, "sample_shard_2"));
}

// Create the mapping and associate it with the new shard

sm.CreateRangeMapping(new RangeMappingCreationInfo<long>
(new Range<long>(300, 400), shard2, MappingStatus.Online));

As an alternative, you can use Powershell to create a new Shard Map Manager. An example is available here.

To add a shard for an empty part of an existing range

In some circumstances, you may have already mapped a range to a shard and partially filled it with data, but you
now want upcoming data to be directed to a different shard. For example, you shard by day range and have
already allocated 50 days to a shard, but on day 24, you want future data to land in a different shard. The elastic
database split-merge tool can perform this operation, but if data movement is not necessary (for example, data for
the range of days [25, 50), i.e., day 25 inclusive to 50 exclusive, does not yet exist) you can perform this entirely
using the Shard Map Management APIs directly.
Example: splitting a range and assigning the empty portion to a newly-added shard
A database named “sample_shard_2” and all necessary schema objects inside of it have been created.
// sm is a RangeShardMap object.
// Add a new shard to hold the range we will move
Shard shard2 = null;

if (!sm.TryGetShard(new ShardLocation(shardServer, "sample_shard_2"),out shard2))

{

shard2 = sm.CreateShard(new ShardLocation(shardServer, "sample_shard_2"));

}

// Split the Range holding Key 25

sm.SplitMapping(sm.GetMappingForKey(25), 25);

// Map new range holding [25-50) to different shard:

// first take existing mapping offline
sm.MarkMappingOffline(sm.GetMappingForKey(25));
// now map while offline to a different shard and take online
RangeMappingUpdate upd = new RangeMappingUpdate();
upd.Shard = shard2;
sm.MarkMappingOnline(sm.UpdateMapping(sm.GetMappingForKey(25), upd));

Important: Use this technique only if you are certain that the range for the updated mapping is empty. The
methods above do not check data for the range being moved, so it is best to include checks in your code. If rows
exist in the range being moved, the actual data distribution will not match the updated shard map. Use the split-
merge tool to perform the operation instead in these cases.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Using the RecoveryManager class to fix shard map
problems
4/14/2017 • 8 min to read • Edit Online

The RecoveryManager class provides ADO.Net applications the ability to easily detect and correct any
inconsistencies between the global shard map (GSM) and the local shard map (LSM) in a sharded database
environment.
The GSM and LSM track the mapping of each database in a sharded environment. Occasionally, a break occurs
between the GSM and the LSM. In that case, use the RecoveryManager class to detect and repair the break.
The RecoveryManager class is part of the Elastic Database client library.

For term definitions, see Elastic Database tools glossary. To understand how the ShardMapManager is used to
manage data in a sharded solution, see Shard map management.

Why use the recovery manager?

In a sharded database environment, there is one tenant per database, and many databases per server. There can
also be many servers in the environment. Each database is mapped in the shard map, so calls can be routed to the
correct server and database. Databases are tracked according to a sharding key, and each shard is assigned a
range of key values. For example, a sharding key may represent the customer names from "D" to "F." The
mapping of all shards (aka databases) and their mapping ranges are contained in the global shard map (GSM).
Each database also contains a map of the ranges contained on the shard that is known as the local shard map
(LSM). When an app connects to a shard, the mapping is cached with the app for quick retrieval. The LSM is used
to validate cached data.
The GSM and LSM may become out of sync for the following reasons:
1. The deletion of a shard whose range is believed to no longer be in use, or renaming of a shard. Deleting a shard
results in an orphaned shard mapping. Similarly, a renamed database can cause an orphaned shard mapping.
Depending on the intent of the change, the shard may need to be removed or the shard location needs to be
updated. To recover a deleted database, see Restore a deleted database.
2. A geo-failover event occurs. To continue, one must update the server name, and database name of shard map
manager in the application and then update the shard mapping details for all shards in a shard map. If there is a
geo-failover, such recovery logic should be automated within the failover workflow. Automating recovery
actions enables a frictionless manageability for geo-enabled databases and avoids manual human actions. To
learn about options to recover a database If there is a data center outage, see Business Continuity and Disaster
Recovery.
3. Either a shard or the ShardMapManager database is restored to an earlier point-in time. To learn about point in
time recovery using backups, see Recovery using backups.
For more information about Azure SQL Database Elastic Database tools, Geo-Replication and Restore, see the
following:
Overview: Cloud business continuity and database disaster recovery with SQL Database
Get started with elastic database tools
ShardMap Management

Retrieving RecoveryManager from a ShardMapManager

The first step is to create a RecoveryManager instance. The GetRecoveryManager method returns the recovery
manager for the current ShardMapManager instance. To address any inconsistencies in the shard map, you must
first retrieve the RecoveryManager for the particular shard map.

ShardMapManager smm = ShardMapManagerFactory.GetSqlShardMapManager(smmConnnectionString,

ShardMapManagerLoadPolicy.Lazy);
RecoveryManager rm = smm.GetRecoveryManager();

In this example, the RecoveryManager is initialized from the ShardMapManager. The ShardMapManager
containing a ShardMap is also already initialized.
Since this application code manipulates the shard map itself, the credentials used in the factory method (in the
preceding example, smmConnectionString) should be credentials that have read-write permissions on the GSM
database referenced by the connection string. These credentials are typically different from credentials used to
open connections for data-dependent routing. For more information, see Using credentials in the elastic database
client.

Removing a shard from the ShardMap after a shard is deleted

The DetachShard method detaches the given shard from the shard map and deletes mappings associated with the
shard.
The location parameter is the shard location, specifically server name and database name, of the shard being
detached.
The shardMapName parameter is the shard map name. This is only required when multiple shard maps are
managed by the same shard map manager. Optional.
IMPORTANT
Use this technique only if you are certain that the range for the updated mapping is empty. The methods above do not
check data for the range being moved, so it is best to include checks in your code.

This example removes shards from the shard map.

rm.DetachShard(s.Location, customerMap);

The shard map reflects the shard location in the GSM before the deletion of the shard. Because the shard was
deleted, it is assumed this was intentional, and the sharding key range is no longer in use. If not, you can execute
point-in time restore. to recover the shard from an earlier point-in-time. (In that case, review the following section
to detect shard inconsistencies.) To recover, see Point in time recovery.
Since it is assumed the database deletion was intentional, the final administrative cleanup action is to delete the
entry to the shard in the shard map manager. This prevents the application from inadvertently writing information
to a range that is not expected.

To detect mapping differences

The DetectMappingDifferences method selects and returns one of the shard maps (either local or global) as the
source of truth and reconciles mappings on both shard maps (GSM and LSM).

rm.DetectMappingDifferences(location, shardMapName);

The location specifies the server name and database name.

The shardMapName parameter is the shard map name. This is only required if multiple shard maps are
managed by the same shard map manager. Optional.

To resolve mapping differences

The ResolveMappingDifferences method selects one of the shard maps (either local or global) as the source of
truth and reconciles mappings on both shard maps (GSM and LSM).

ResolveMappingDifferences (RecoveryToken, MappingDifferenceResolution.KeepShardMapping);

The RecoveryToken parameter enumerates the differences in the mappings between the GSM and the LSM for
the specific shard.
The MappingDifferenceResolution enumeration is used to indicate the method for resolving the difference
between the shard mappings.
MappingDifferenceResolution.KeepShardMapping is recommended that when the LSM contains the
accurate mapping and therefore the mapping in the shard should be used. This is typically the case If there is a
failover: the shard now resides on a new server. Since the shard must first be removed from the GSM (using the
RecoveryManager.DetachShard method), a mapping no longer exists on the GSM. Therefore, the LSM must be
used to re-establish the shard mapping.

Attach a shard to the ShardMap after a shard is restored

The AttachShard method attaches the given shard to the shard map. It then detects any shard map inconsistencies
and updates the mappings to match the shard at the point of the shard restoration. It is assumed that the database
is also renamed to reflect the original database name (before the shard was restored), since the point-in time
restoration defaults to a new database appended with the timestamp.

rm.AttachShard(location, shardMapName)

The location parameter is the server name and database name, of the shard being attached.
The shardMapName parameter is the shard map name. This is only required when multiple shard maps are
managed by the same shard map manager. Optional.
This example adds a shard to the shard map that has been recently restored from an earlier point-in time. Since the
shard (namely the mapping for the shard in the LSM) has been restored, it is potentially inconsistent with the shard
entry in the GSM. Outside of this example code, the shard was restored and renamed to the original name of the
database. Since it was restored, it is assumed the mapping in the LSM is the trusted mapping.

rm.AttachShard(s.Location, customerMap);
var gs = rm.DetectMappingDifferences(s.Location);
foreach (RecoveryToken g in gs)
{
rm.ResolveMappingDifferences(g, MappingDifferenceResolution.KeepShardMapping);
}

Updating shard locations after a geo-failover (restore) of the shards

If there is a geo-failover, the secondary database is made write accessible and becomes the new primary database.
The name of the server, and potentially the database (depending on your configuration), may be different from the
original primary. Therefore the mapping entries for the shard in the GSM and LSM must be fixed. Similarly, if the
database is restored to a different name or location, or to an earlier point in time, this might cause inconsistencies
in the shard maps. The Shard Map Manager handles the distribution of open connections to the correct database.
Distribution is based on the data in the shard map and the value of the sharding key that is the target of the
application� s request. After a geo-failover, this information must be updated with the accurate server name,
database name and shard mapping of the recovered database.

Best practices
Geo-failover and recovery are operations typically managed by a cloud administrator of the application
intentionally utilizing one of Azure SQL Databases business continuity features. Business continuity planning
requires processes, procedures, and measures to ensure that business operations can continue without
interruption. The methods available as part of the RecoveryManager class should be used within this work flow to
ensure the GSM and LSM are kept up-to-date based on the recovery action taken. There are five basic steps to
properly ensuring the GSM and LSM reflect the accurate information after a failover event. The application code to
execute these steps can be integrated into existing tools and workflow.
1. Retrieve the RecoveryManager from the ShardMapManager.
2. Detach the old shard from the shard map.
3. Attach the new shard to the shard map, including the new shard location.
4. Detect inconsistencies in the mapping between the GSM and LSM.
5. Resolve differences between the GSM and the LSM, trusting the LSM.
This example performs the following steps:
1. Removes shards from the Shard Map that reflect shard locations before the failover event.
2. Attaches shards to the Shard Map reflecting the new shard locations (the parameter
"Configuration.SecondaryServer" is the new server name but the same database name).
3. Retrieves the recovery tokens by detecting mapping differences between the GSM and the LSM for each shard.
4. Resolves the inconsistencies by trusting the mapping from the LSM of each shard.

var shards = smm.GetShards();

foreach (shard s in shards)
{
if (s.Location.Server == Configuration.PrimaryServer)

{
ShardLocation slNew = new ShardLocation(Configuration.SecondaryServer, s.Location.Database);

rm.DetachShard(s.Location);

rm.AttachShard(slNew);

var gs = rm.DetectMappingDifferences(slNew);

foreach (RecoveryToken g in gs)

{
rm.ResolveMappingDifferences(g, MappingDifferenceResolution.KeepShardMapping);
}
}
}

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Migrate existing databases to scale-out
1/17/2017 • 4 min to read • Edit Online

Easily manage your existing scaled-out sharded databases using Azure SQL Database database tools (such as the
Elastic Database client library). You must first convert an existing set of databases to use the shard map manager.

Overview
To migrate an existing sharded database:
1. Prepare the shard map manager database.
2. Create the shard map.
3. Prepare the individual shards.
4. Add mappings to the shard map.
These techniques can be implemented using either the .NET Framework client library, or the PowerShell scripts
found at Azure SQL DB - Elastic Database tools scripts. The examples here use the PowerShell scripts.
For more information about the ShardMapManager, see Shard map management. For an overview of the elastic
database tools, see Elastic Database features overview.

Prepare the shard map manager database

The shard map manager is a special database that contains the data to manage scaled-out databases. You can use
an existing database, or create a new database. Note that a database acting as shard map manager should not be
the same database as a shard. Also note that the PowerShell script does not create the database for you.

Step 1: create a shard map manager

# Create a shard map manager.
New-ShardMapManager -UserName '<user_name>'
-Password '<password>'
-SqlServerName '<server_name>'
-SqlDatabaseName '<smm_db_name>'
#<server_name> and <smm_db_name> are the server name and database name
# for the new or existing database that should be used for storing
# tenant-database mapping information.

To retrieve the shard map manager

After creation, you can retrieve the shard map manager with this cmdlet. This step is needed every time you need
to use the ShardMapManager object.

# Try to get a reference to the Shard Map Manager

$ShardMapManager = Get-ShardMapManager -UserName '<user_name>'
-Password '<password>'
-SqlServerName '<server_name>'
-SqlDatabaseName '<smm_db_name>'

Step 2: create the shard map

You must select the type of shard map to create. The choice depends on the database architecture:
1. Single tenant per database (For terms, see the glossary.)
2. Multiple tenants per database (two types):
a. List mapping
b. Range mapping
For a single-tenant model, create a list mapping shard map. The single-tenant model assigns one database per
tenant. This is an effective model for SaaS developers as it simplifies management.

The multi-tenant model assigns several tenants to a single database (and you can distribute groups of tenants
across multiple databases). Use this model when you expect each tenant to have small data needs. In this model,
we assign a range of tenants to a database using range mapping.

Or you can implement a multi-tenant database model using a list mapping to assign multiple tenants to a single
database. For example, DB1 is used to store information about tenant id 1 and 5, and DB2 stores data for tenant 7
and tenant 10.
Based on your choice, choose one of these options:
Option 1: create a shard map for a list mapping
Create a shard map using the ShardMapManager object.

# $ShardMapManager is the shard map manager object.

$ShardMap = New-ListShardMap -KeyType $([int])
-ListShardMapName 'ListShardMap'
-ShardMapManager $ShardMapManager

Option 2: create a shard map for a range mapping

Note that to utilize this mapping pattern, tenant id values needs to be continuous ranges, and it is acceptable to
have gap in the ranges by simply skipping the range when creating the databases.

# $ShardMapManager is the shard map manager object

# 'RangeShardMap' is the unique identifier for the range shard map.
$ShardMap = New-RangeShardMap
-KeyType $([int])
-RangeShardMapName 'RangeShardMap'
-ShardMapManager $ShardMapManager

Option 3: List mappings on a single database

Setting up this pattern also requires creation of a list map as shown in step 2, option 1.

Step 3: Prepare individual shards

Add each shard (database) to the shard map manager. This prepares the individual databases for storing mapping
information. Execute this method on each shard.

Add-Shard
-ShardMap $ShardMap
-SqlServerName '<shard_server_name>'
-SqlDatabaseName '<shard_database_name>'
# The $ShardMap is the shard map created in step 2.

Step 4: Add mappings

The addition of mappings depends on the kind of shard map you created. If you created a list map, you add list
mappings. If you created a range map, you add range mappings.
Option 1: map the data for a list mapping
Map the data by adding a list mapping for each tenant.

# Create the mappings and associate it with the new shards

Add-ListMapping
-KeyType $([int])
-ListPoint '<tenant_id>'
-ListShardMap $ShardMap
-SqlServerName '<shard_server_name>'
-SqlDatabaseName '<shard_database_name>'

Option 2: map the data for a range mapping

Add the range mappings for all the tenant id range - database associations:

# Create the mappings and associate it with the new shards

Add-RangeMapping
-KeyType $([int])
-RangeHigh '5'
-RangeLow '1'
-RangeShardMap $ShardMap
-SqlServerName '<shard_server_name>'
-SqlDatabaseName '<shard_database_name>'

Step 4 option 3: map the data for multiple tenants on a single database
For each tenant, run the Add-ListMapping (option 1, above).

Checking the mappings

Information about the existing shards and the mappings associated with them can be queried using following
commands:

# List the shards and mappings

Get-Shards -ShardMap $ShardMap
Get-Mappings -ShardMap $ShardMap

Summary
Once you have completed the setup, you can begin to use the Elastic Database client library. You can also use data
dependent routing and multi-shard query.

Next steps
Get the PowerShell scripts from Azure SQL DB-Elastic Database tools sripts.
The tools are also on GitHub: Azure/elastic-db-tools.
Use the split-merge tool to move data to or from a multi-tenant model to a single tenant model. See Split merge
tool.

Additional resources
For information on common data architecture patterns of multi-tenant software-as-a-service (SaaS) database
applications, see Design Patterns for Multi-tenant SaaS Applications with Azure SQL Database.
Questions and Feature Requests
For questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Performance counters for shard map manager
2/2/2017 • 2 min to read • Edit Online

You can capture the performance of a shard map manager, especially when using data dependent routing.
Counters are created with methods of the Microsoft.Azure.SqlDatabase.ElasticScale.Client class.
Counters are used to track the performance of data dependent routing operations. These counters are accessible in
the Performance Monitor, under the "Elastic Database: Shard Management" category.
For the latest version: Go to Microsoft.Azure.SqlDatabase.ElasticScale.Client. See also Upgrade an app to use the
latest elastic database client library.

Prerequisites
To create the performance category and counters, the user must be a part of the local Administrators group on
the machine hosting the application.
To create a performance counter instance and update the counters, the user must be a member of either the
Administrators or Performance Monitor Users group.

Create performance category and counters

To create the counters, call the CreatePeformanceCategoryAndCounters method of the
ShardMapManagmentFactory class. Only an administrator can execute the method:

ShardMapManagerFactory.CreatePerformanceCategoryAndCounters()

You can also use this PowerShell script to execute the method. The method creates the following performance
counters:
Cached mappings: Number of mappings cached for the shard map.
DDR operations/sec: Rate of data dependent routing operations for the shard map. This counter is updated
when a call to OpenConnectionForKey() results in a successful connection to the destination shard.
Mapping lookup cache hits/sec: Rate of successful cache lookup operations for mappings in the shard map.
Mapping lookup cache misses/sec: Rate of failed cache lookup operations for mappings in the shard map.
Mappings added or updated in cache/sec: Rate at which mappings are being added or updated in cache for
the shard map.
Mappings removed from cache/sec: Rate at which mappings are being removed from cache for the shard
map.
Performance counters are created for each cached shard map per process.

Notes
The following events trigger the creation of the performance counters:
Initialization of the ShardMapManager with eager loading, if the ShardMapManager contains any shard maps.
These include the GetSqlShardMapManager and the TryGetSqlShardMapManager methods.
Successful lookup of a shard map (using GetShardMap(), GetListShardMap() or GetRangeShardMap()).
Successful creation of shard map using CreateShardMap().
The performance counters will be updated by all cache operations performed on the shard map and mappings.
Successful removal of the shard map using DeleteShardMap()reults in deletion of the performance counters
instance.

Best practices
Creation of the performance category and counters should be performed only once before the creation of
ShardMapManager object. Every execution of the command CreatePerformanceCategoryAndCounters() clears
the previous counters (losing data reported by all instances) and creates new ones.
Performance counter instances are created per process. Any application crash or removal of a shard map from
the cache will result in deletion of the performance counters instances.
See also
Elastic Database features overview

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Elastic Database client library with Entity Framework
3/7/2017 • 16 min to read • Edit Online

This document shows the changes in an Entity Framework application that are needed to integrate with the Elastic
Database tools. The focus is on composing shard map management and data-dependent routing with the Entity
Framework Code First approach. The Code First - New Database tutorial for EF serves as our running example
throughout this document. The sample code accompanying this document is part of elastic database tools' set of
samples in the Visual Studio Code Samples.

Downloading and Running the Sample Code

To download the code for this article:
Visual Studio 2012 or later is required.
Download the Elastic DB Tools for Azure SQL - Entity Framework Integration sample from MSDN. Unzip the
sample to a location of your choosing.
Start Visual Studio.
In Visual Studio, select File -> Open Project/Solution.
In the Open Project dialog, navigate to the sample you downloaded and select
EntityFrameworkCodeFirst.sln to open the sample.
To run the sample, you need to create three empty databases in Azure SQL Database:
Shard Map Manager database
Shard 1 database
Shard 2 database
Once you have created these databases, fill in the place holders in Program.cs with your Azure SQL DB server
name, the database names and your credentials to connect to the databases. Build the solution in Visual Studio.
Visual Studio will download the required NuGet packages for the elastic database client library, Entity Framework,
and Transient Fault handling as part of the build process. Make sure that restoring NuGet packages is enabled for
your solution. You can enable this setting by right-clicking on the solution file in the Visual Studio Solution
Explorer.

Entity Framework workflows

Entity Framework developers rely on one of the following four workflows to build applications and to ensure
persistence for application objects:
Code First (New Database): The EF developer creates the model in the application code and then EF generates
the database from it.
Code First (Existing Database): The developer lets EF generate the application code for the model from an
existing database.
Model First: The developer creates the model in the EF designer and then EF creates the database from the
model.
Database First: The developer uses EF tooling to infer the model from an existing database.
All these approaches rely on the DbContext class to transparently manage database connections and database
schema for an application. As we will discuss in more detail later in the document, different constructors on the
DbContext base class allow for different levels of control over connection creation, database bootstrapping and
schema creation. Challenges arise primarily from the fact that the database connection management provided by
EF intersects with the connection management capabilities of the data dependent routing interfaces provided by
the elastic database client library.

Elastic database tools assumptions

For term definitions, see Elastic Database tools glossary.
With elastic database client library, you define partitions of your application data called shardlets. Shardlets are
identified by a sharding key and are mapped to specific databases. An application may have as many databases as
needed and distribute the shardlets to provide enough capacity or performance given current business
requirements. The mapping of sharding key values to the databases is stored by a shard map provided by the
elastic database client APIs. We call this capability Shard Map Management, or SMM for short. The shard map
also serves as the broker of database connections for requests that carry a sharding key. We refer to this capability
as data-dependent routing.
The shard map manager protects users from inconsistent views into shardlet data that can occur when concurrent
shardlet management operations (such as relocating data from one shard to another) are happening. To do so, the
shard maps managed by the client library broker the database connections for an application. This allows the shard
map functionality to automatically kill a database connection when shard management operations could impact
the shardlet that the connection has been created for. This approach needs to integrate with some of EF’s
functionality, such as creating new connections from an existing one to check for database existence. In general,
our observation has been that the standard DbContext constructors only work reliably for closed database
connections that can safely be cloned for EF work. The design principle of elastic database instead is to only broker
opened connections. One might think that closing a connection brokered by the client library before handing it
over to the EF DbContext may solve this issue. However, by closing the connection and relying on EF to re-open it,
one foregoes the validation and consistency checks performed by the library. The migrations functionality in EF,
however, uses these connections to manage the underlying database schema in a way that is transparent to the
application. Ideally, we would like to retain and combine all these capabilities from both the elastic database client
library and EF in the same application. The following section discusses these properties and requirements in more
detail.

Requirements
When working with both the elastic database client library and Entity Framework APIs, we want to retain the
following properties:
Scale-out: To add or remove databases from the data tier of the sharded application as necessary for the
capacity demands of the application. This means control over the the creation and deletion of databases and
using the elastic database shard map manager APIs to manage databases, and mappings of shardlets.
Consistency: The application employs sharding, and uses the data dependent routing capabilities of the client
library. To avoid corruption or wrong query results, connections are brokered through the shard map manager.
This also retains validation and consistency.
Code First: To retain the convenience of EF’s code first paradigm. In Code First, classes in the application are
mapped transparently to the underlying database structures. The application code interacts with DbSets that
mask most aspects involved in the underlying database processing.
Schema: Entity Framework handles initial database schema creation and subsequent schema evolution
through migrations. By retaining these capabilities, adapting your app is easy as the data evolves.
The following guidance instructs how to satisfy these requirements for Code First applications using elastic
database tools.

Data dependent routing using EF DbContext

Database connections with Entity Framework are typically managed through subclasses of DbContext. Create
these subclasses by deriving from DbContext. This is where you define your DbSets that implement the
database-backed collections of CLR objects for your application. In the context of data dependent routing, we can
identify several helpful properties that do not necessarily hold for other EF code first application scenarios:
The database already exists and has been registered in the elastic database shard map.
The schema of the application has already been deployed to the database (explained below).
Data-dependent routing connections to the database are brokered by the shard map.
To integrate DbContexts with data-dependent routing for scale-out:
1. Create physical database connections through the elastic database client interfaces of the shard map manager,
2. Wrap the connection with the DbContext subclass
3. Pass the connection down into the DbContext base classes to ensure all the processing on the EF side happens
as well.
The following code example illustrates this approach. (This code is also in the accompanying Visual Studio project)

public class ElasticScaleContext<T> : DbContext

{
public DbSet<Blog> Blogs { get; set; }
…

// C'tor for data dependent routing. This call will open a validated connection
// routed to the proper shard by the shard map manager.
// Note that the base class c'tor call will fail for an open connection
// if migrations need to be done and SQL credentials are used. This is the reason for the
// separation of c'tors into the data-dependent routing case (this c'tor) and the internal c'tor for new shards.
public ElasticScaleContext(ShardMap shardMap, T shardingKey, string connectionStr)
: base(CreateDDRConnection(shardMap, shardingKey, connectionStr),
true /* contextOwnsConnection */)
{
}

// Only static methods are allowed in calls into base class c'tors.
private static DbConnection CreateDDRConnection(
ShardMap shardMap,
T shardingKey,
string connectionStr)
{
// No initialization
Database.SetInitializer<ElasticScaleContext<T>>(null);

// Ask shard map to broker a validated connection for the given key
SqlConnection conn = shardMap.OpenConnectionForKey<T>
(shardingKey, connectionStr, ConnectionOptions.Validate);
return conn;
}

Main points
A new constructor replaces the default constructor in the DbContext subclass
The new constructor takes the arguments that are required for data dependent routing through elastic
database client library:
the shard map to access the data-dependent routing interfaces,
the sharding key to identify the shardlet,
a connection string with the credentials for the data-dependent routing connection to the shard.
The call to the base class constructor takes a detour into a static method that performs all the steps
necessary for data-dependent routing.
It uses the OpenConnectionForKey call of the elastic database client interfaces on the shard map to
establish an open connection.
The shard map creates the open connection to the shard that holds the shardlet for the given sharding
key.
This open connection is passed back to the base class constructor of DbContext to indicate that this
connection is to be used by EF instead of letting EF create a new connection automatically. This way the
connection has been tagged by the elastic database client API so that it can guarantee consistency under
shard map management operations.
Use the new constructor for your DbContext subclass instead of the default constructor in your code. Here is an
example:

// Create and save a new blog.

Console.Write("Enter a name for a new blog: ");

var name = Console.ReadLine();

using (var db = new ElasticScaleContext<int>(

sharding.ShardMap,
tenantId1,
connStrBldr.ConnectionString))
{
var blog = new Blog { Name = name };
db.Blogs.Add(blog);
db.SaveChanges();

// Display all Blogs for tenant 1

var query = from b in db.Blogs
orderby b.Name
select b;
…
}

The new constructor opens the connection to the shard that holds the data for the shardlet identified by the value
of tenantid1. The code in the using block stays unchanged to access the DbSet for blogs using EF on the shard
for tenantid1. This changes semantics for the code in the using block such that all database operations are now
scoped to the one shard where tenantid1 is kept. For instance, a LINQ query over the blogs DbSet would only
return blogs stored on the current shard, but not the ones stored on other shards.
Transient faults handling
The Microsoft Patterns & Practices team published the The Transient Fault Handling Application Block. The library
is used with elastic scale client library in combination with EF. However, ensure that any transient exception returns
to a place where we can ensure that the new constructor is being used after a transient fault so that any new
connection attempt is made using the constructors we have tweaked. Otherwise, a connection to the correct shard
is not guaranteed, and there are no assurances the connection is maintained as changes to the shard map occur.
The following code sample illustrates how a SQL retry policy can be used around the new DbContext subclass
constructors:
SqlDatabaseUtils.SqlRetryPolicy.ExecuteAction(() =>
{
using (var db = new ElasticScaleContext<int>(
sharding.ShardMap,
tenantId1,
connStrBldr.ConnectionString))
{
var blog = new Blog { Name = name };
db.Blogs.Add(blog);
db.SaveChanges();
…
}
});

SqlDatabaseUtils.SqlRetryPolicy in the code above is defined as a

SqlDatabaseTransientErrorDetectionStrategy with a retry count of 10, and 5 seconds wait time between
retries. This approach is similar to the guidance for EF and user-initiated transactions (see Limitations with Retrying
Execution Strategies (EF6 onwards). Both situations require that the application program controls the scope to
which the transient exception returns: to either reopen the transaction, or (as shown) recreate the context from the
proper constructor that uses the elastic database client library.
The need to control where transient exceptions take us back in scope also precludes the use of the built-in
SqlAzureExecutionStrategy that comes with EF. SqlAzureExecutionStrategy would reopen a connection but
not use OpenConnectionForKey and therefore bypass all the validation that is performed as part of the
OpenConnectionForKey call. Instead, the code sample uses the built-in DefaultExecutionStrategy that also
comes with EF. As opposed to SqlAzureExecutionStrategy, it works correctly in combination with the retry policy
from Transient Fault Handling. The execution policy is set in the ElasticScaleDbConfiguration class. Note that we
decided not to use DefaultSqlExecutionStrategy since it suggests to use SqlAzureExecutionStrategy if
transient exceptions occur - which would lead to wrong behavior as discussed. For more information on the
different retry policies and EF, see Connection Resiliency in EF.
Constructor rewrites
The code examples above illustrate the default constructor re-writes required for your application in order to use
data dependent routing with the Entity Framework. The following table generalizes this approach to other
constructors.

REWRITTEN CONSTRUCTOR
CURRENT CONSTRUCTOR FOR DATA BASE CONSTRUCTOR NOTES

MyContext() ElasticScaleContext(ShardMa DbContext(DbConnection, The connection needs to be

p, TKey) bool) a function of the shard map
and the data-dependent
routing key. You need to by-
pass automatic connection
creation by EF and instead
use the shard map to broker
the connection.

MyContext(string) ElasticScaleContext(ShardMa DbContext(DbConnection, The connection is a function

p, TKey) bool) of the shard map and the
data-dependent routing key.
A fixed database name or
connection string will not
work as they by-pass
validation by the shard map.
REWRITTEN CONSTRUCTOR
CURRENT CONSTRUCTOR FOR DATA BASE CONSTRUCTOR NOTES

MyContext(DbCompiledMo ElasticScaleContext(ShardMa DbContext(DbConnection, The connection will get

del) p, TKey, DbCompiledModel) DbCompiledModel, bool) created for the given shard
map and sharding key with
the model provided. The
compiled model will be
passed on to the base c’tor.

MyContext(DbConnection, ElasticScaleContext(ShardMa DbContext(DbConnection, The connection needs to be

bool) p, TKey, bool) bool) inferred from the shard map
and the key. It cannot be
provided as an input (unless
that input was already using
the shard map and the key).
The Boolean will be passed
on.

MyContext(string, ElasticScaleContext(ShardMa DbContext(DbConnection, The connection needs to be

DbCompiledModel) p, TKey, DbCompiledModel) DbCompiledModel, bool) inferred from the shard map
and the key. It cannot be
provided as an input (unless
that input was using the
shard map and the key). The
compiled model will be
passed on.

MyContext(ObjectContext, ElasticScaleContext(ShardMa DbContext(ObjectContext, The new constructor needs

bool) p, TKey, ObjectContext, bool) to ensure that any
bool) connection in the
ObjectContext passed as an
input is re-routed to a
connection managed by
Elastic Scale. A detailed
discussion of ObjectContexts
is beyond the scope of this
document.

MyContext(DbConnection, ElasticScaleContext(ShardMa DbContext(DbConnection, The connection needs to be

DbCompiledModel,bool) p, TKey, DbCompiledModel, DbCompiledModel, bool); inferred from the shard map
bool) and the key. The connection
cannot be provided as an
input (unless that input was
already using the shard map
and the key). Model and
Boolean are passed on to
the base class constructor.

Shard schema deployment through EF migrations

Automatic schema management is a convenience provided by the Entity Framework. In the context of applications
using elastic database tools, we want to retain this capability to automatically provision the schema to newly
created shards when databases are added to the sharded application. The primary use case is to increase capacity
at the data tier for sharded applications using EF. Relying on EF’s capabilities for schema management reduces the
database administration effort with a sharded application built on EF.
Schema deployment through EF migrations works best on unopened connections. This is in contrast to the
scenario for data dependent routing that relies on the opened connection provided by the elastic database client
API. Another difference is the consistency requirement: While desirable to ensure consistency for all data-
dependent routing connections to protect against concurrent shard map manipulation, it is not a concern with
initial schema deployment to a new database that has not yet been registered in the shard map, and not yet been
allocated to hold shardlets. We can therefore rely on regular database connections for this scenarios, as opposed to
data-dependent routing.
This leads to an approach where schema deployment through EF migrations is tightly coupled with the registration
of the new database as a shard in the application’s shard map. This relies on the following prerequisites:
The database has already been created.
The database is empty - it holds no user schema and no user data.
The database cannot yet be accessed through the elastic database client APIs for data-dependent routing.
With these prerequisites in place, we can create a regular un-opened SqlConnection to kick off EF migrations for
schema deployment. The following code sample illustrates this approach.

// Enter a new shard - i.e. an empty database - to the shard map, allocate a first tenant to it
// and kick off EF intialization of the database to deploy schema

public void RegisterNewShard(string server, string database, string connStr, int key)
{

Shard shard = this.ShardMap.CreateShard(new ShardLocation(server, database));

SqlConnectionStringBuilder connStrBldr = new SqlConnectionStringBuilder(connStr);

connStrBldr.DataSource = server;
connStrBldr.InitialCatalog = database;

// Go into a DbContext to trigger migrations and schema deployment for the new shard.
// This requires an un-opened connection.
using (var db = new ElasticScaleContext<int>(connStrBldr.ConnectionString))
{
// Run a query to engage EF migrations
(from b in db.Blogs
select b).Count();
}

// Register the mapping of the tenant to the shard in the shard map.
// After this step, data-dependent routing on the shard map can be used

this.ShardMap.CreatePointMapping(key, shard);
}

This sample shows the method RegisterNewShard that registers the shard in the shard map, deploys the schema
through EF migrations, and stores a mapping of a sharding key to the shard. It relies on a constructor of the
DbContext subclass (ElasticScaleContext in the sample) that takes a SQL connection string as input. The code of
this constructor is straight-forward, as the following example shows:
// C'tor to deploy schema and migrations to a new shard
protected internal ElasticScaleContext(string connectionString)
: base(SetInitializerForConnection(connectionString))
{
}

// Only static methods are allowed in calls into base class c'tors
private static string SetInitializerForConnection(string connnectionString)
{
// We want existence checks so that the schema can get deployed
Database.SetInitializer<ElasticScaleContext<T>>(
new CreateDatabaseIfNotExists<ElasticScaleContext<T>>());

return connnectionString;
}

One might have used the version of the constructor inherited from the base class. But the code needs to ensure
that the default initializer for EF is used when connecting. Hence the short detour into the static method before
calling into the base class constructor with the connection string. Note that the registration of shards should run in
a different app domain or process to ensure that the initializer settings for EF do not conflict.

Limitations
The approaches outlined in this document entail a couple of limitations:
EF applications that use LocalDb first need to migrate to a regular SQL Server database before using elastic
database client library. Scaling out an application through sharding with Elastic Scale is not possible with
LocalDb. Note that development can still use LocalDb.
Any changes to the application that imply database schema changes need to go through EF migrations on all
shards. The sample code for this document does not demonstrate how to do this. Consider using Update-
Database with a ConnectionString parameter to iterate over all shards; or extract the T-SQL script for the
pending migration using Update-Database with the -Script option and apply the T-SQL script to your shards.
Given a request, it is assumed that all of its database processing is contained within a single shard as identified
by the sharding key provided by the request. However, this assumption does not always hold true. For example,
when it is not possible to make a sharding key available. To address this, the client library provides the
MultiShardQuery class that implements a connection abstraction for querying over several shards. Learning
to use the MultiShardQuery in combination with EF is beyond the scope of this document

Conclusion
Through the steps outlined in this document, EF applications can use the elastic database client library's capability
for data dependent routing by refactoring constructors of the DbContext subclasses used in the EF application.
This limits the changes required to those places where DbContext classes already exist. In addition, EF applications
can continue to benefit from automatic schema deployment by combining the steps that invoke the necessary EF
migrations with the registration of new shards and mappings in the shard map.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Upgrade an app to use the latest elastic database
client library
3/7/2017 • 3 min to read • Edit Online

New versions of the Elastic Database client library are available through NuGetand the NuGetPackage Manager
interface in Visual Studio. Upgrades contain bug fixes and support for new capabilities of the client library.
For the latest version: Go to Microsoft.Azure.SqlDatabase.ElasticScale.Client.
Rebuild your application with the new library, as well as change your existing Shard Map Manager metadata
stored in your Azure SQL Databases to support new features.
Performing these steps in order ensures that old versions of the client library are no longer present in your
environment when metadata objects are updated, which means that old-version metadata objects won’t be created
after upgrade.

Upgrade steps
1. Upgrade your applications. In Visual Studio, download and reference the latest client library version into all of
your development projects that use the library; then rebuild and deploy.
In your Visual Studio solution, select Tools --> NuGet Package Manager --> Manage NuGet Packages for
Solution.
(Visual Studio 2013) In the left panel, select Updates, and then select the Update button on the package Azure
SQL Database Elastic Scale Client Library that appears in the window.
(Visual Studio 2015) Set the Filter box to Upgrade available. Select the package to update, and click the
Update button.
(Visual Studio 2017) At the top of the dialog, select Updates. Select the package to update, and click the
Update button.
Build and Deploy.
2. Upgrade your scripts. If you are using PowerShell scripts to manage shards, download the new library
version and copy it into the directory from which you execute scripts.
3. Upgrade your split-merge service. If you use the elastic database split-merge tool to reorganize sharded data,
download and deploy the latest version of the tool. Detailed upgrade steps for the Service can be found here.
4. Upgrade your Shard Map Manager databases. Upgrade the metadata supporting your Shard Maps in Azure
SQL Database. There are two ways you can accomplish this, using PowerShell or C#. Both options are shown
below.
Option 1: Upgrade metadata using PowerShell
1. Download the latest command-line utility for NuGet from here and save to a folder.
2. Open a Command Prompt, navigate to the same folder, and issue the command:
nuget install Microsoft.Azure.SqlDatabase.ElasticScale.Client
3. Navigate to the subfolder containing the new client DLL version you have just downloaded, for example:
cd .\Microsoft.Azure.SqlDatabase.ElasticScale.Client.1.0.0\lib\net45
4. Download the elastic database client upgrade scriptlet from the Script Center, and save it into the same folder
containing the DLL.
5. From that folder, run “PowerShell .\upgrade.ps1” from the command prompt and follow the prompts.
Option 2: Upgrade metadata using C#
Alternatively, create a Visual Studio application that opens your ShardMapManager, iterates over all shards, and
performs the metadata upgrade by calling the methods UpgradeLocalStore and UpgradeGlobalStore as in this
example:

ShardMapManager smm =
ShardMapManagerFactory.GetSqlShardMapManager
(connStr, ShardMapManagerLoadPolicy.Lazy);
smm.UpgradeGlobalStore();

foreach (ShardLocation loc in

smm.GetDistinctShardLocations())
{
smm.UpgradeLocalStore(loc);
}

These techniques for metadata upgrades can be applied multiple times without harm. For example, if an older
client version inadvertently creates a shard after you have already updated, you can run upgrade again across all
shards to ensure that the latest metadata version is present throughout your infrastructure.
Note: New versions of the client library published to-date continue to work with prior versions of the Shard Map
Manager metadata on Azure SQL DB, and vice-versa. However to take advantage of some of the new features in
the latest client, metadata needs to be upgraded. Note that metadata upgrades will not affect any user-data or
application-specific data, only objects created and used by the Shard Map Manager. And applications continue to
operate through the upgrade sequence described above.

Elastic database client version history

For version history, go to Microsoft.Azure.SqlDatabase.ElasticScale.Client

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Installing Elastic Database jobs overview
2/23/2017 • 6 min to read • Edit Online

Elastic Database jobs can be installed via PowerShell or through the Azure Classic Portal.You can gain access to
create and manage jobs using the PowerShell API only if you install the PowerShell package. Additionally, the
PowerShell APIs provide significantly more functionality than the portal at this point in time.
If you have already installed Elastic Database jobs through the Portal from an existing elastic pool, the latest
Powershell preview includes scripts to upgrade your existing installation. It is highly recommended to upgrade
your installation to the latest Elastic Database jobs components in order to take advantage of new functionality
exposed via the PowerShell APIs.

Prerequisites
An Azure subscription. For a free trial, see Free trial.
Azure PowerShell. Install the latest version using the Web Platform Installer. For detailed information, see How
to install and configure Azure PowerShell.
NuGet Command-line Utility is used to install the Elastic Database jobs package. For more information, see
http://docs.nuget.org/docs/start-here/installing-nuget.

Download and import the Elastic Database jobs PowerShell package

1. Launch Microsoft Azure PowerShell command window and navigate to the directory where you downloaded
NuGet Command-line Utility (nuget.exe).
2. Download and import Elastic Database jobs package into the current directory with the following
command:

PS C:\>.\nuget install Microsoft.Azure.SqlDatabase.Jobs -prerelease

The Elastic Database jobs files are placed in the local directory in a folder named
Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x where x.x.xxxx.x reflects the version number. The
PowerShell cmdlets (including required client .dlls) are located in the tools\ElasticDatabaseJobs sub-
directory and the PowerShell scripts to install, upgrade and uninstall also reside in the tools sub-directory.
3. Navigate to the tools sub-directory under the Microsoft.Azure.SqlDatabase.Jobs.x.x.xxx.x folder by typing cd
tools, for example:

PS C:\*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*>cd tools

4. Execute the .\InstallElasticDatabaseJobsCmdlets.ps1 script to copy the ElasticDatabaseJobs directory into

$home\Documents\WindowsPowerShell\Modules. This will also automatically import the module for use,
for example:

PS C:\*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*\tools>Unblock-File .\InstallElasticDatabaseJobsCmdlets.ps1
PS C:\*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*\tools>.\InstallElasticDatabaseJobsCmdlets.ps1

Install the Elastic Database jobs components using PowerShell

1. Launch a Microsoft Azure PowerShell command window and navigate to the \tools sub-directory under the
Microsoft.Azure.SqlDatabase.Jobs.x.x.xxx.x folder: Type cd \tools

PS C:\*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*>cd tools

2. Execute the .\InstallElasticDatabaseJobs.ps1 PowerShell script and supply values for its requested variables.
This script will create the components described in Elastic Database jobs components and pricing along
with configuring the Azure Cloud Service to appropriately use the dependent components.

PS C:\*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*\tools>Unblock-File .\InstallElasticDatabaseJobs.ps1
PS C:\*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*\tools>.\InstallElasticDatabaseJobs.ps1

When you run this command a window opens asking for a user name and password. This is not your Azure
credentials, enter the user name and password that will be the administrator credentials you want to create for the
new server.
The parameters provided on this sample invocation can be modified for your desired settings. The following
provides more information on the behavior of each parameter:

PARAMETER DESCRIPTION

ResourceGroupName Provides the Azure resource group name created to contain

the newly created Azure components. This parameter defaults
to “__ElasticDatabaseJob”. It is not recommended to change
this value.

ResourceGroupLocation Provides the Azure location to be used for the newly created
Azure components. This parameter defaults to the Central US
location.

ServiceWorkerCount Provides the number of service workers to install. This

parameter defaults to 1. A higher number of workers can be
used to scale out the service and to provide high availability.
It is recommended to use “2” for deployments that require
high availability of the service.

ServiceVmSize Provides the VM size for usage within the Cloud Service. This
parameter defaults to A0. Parameters values of A0/A1/A2/A3
are accepted which cause the worker role to use an
ExtraSmall/Small/Medium/Large size, respectively. Fo more
information on worker role sizes, see Elastic Database jobs
components and pricing.

SqlServerDatabaseSlo Provides the service level objective for a Standard edition. This
parameter defaults to S0. Parameter values of S0/S1/S2/S3
are accepted which cause the Azure SQL Database to use the
respective SLO. For more information on SQL Database SLOs,
see Elastic Database jobs components and pricing.

SqlServerAdministratorUserName Provides the admin user name for the newly created Azure
SQL Database server. When not specified, a PowerShell
credentials window will open to prompt for the credentials.

SqlServerAdministratorPassword Provides the admin password for the newly created Azure
SQL Database server. When not provided, a PowerShell
credentials window will open to prompt for the credentials.
For systems that target having large numbers of jobs running in parallel against a large number of databases, it is
recommended to specify parameters such as: -ServiceWorkerCount 2 -ServiceVmSize A2 -SqlServerDatabaseSlo
S2.

PS C:\*Microsoft.Azure.SqlDatabase.Jobs.dll.x.x.xxx.x*\tools>Unblock-File .\InstallElasticDatabaseJobs.ps1
PS C:\*Microsoft.Azure.SqlDatabase.Jobs.dll.x.x.xxx.x*\tools>.\InstallElasticDatabaseJobs.ps1 -ServiceWorkerCount 2 -ServiceVmSize A2 -
SqlServerDatabaseSlo S2

Update an existing Elastic Database jobs components installation using

PowerShell
Elastic Database jobs can be updated within an existing installation for scale and high-availability. This process
allows for future upgrades of service code without having to drop and recreate the control database. This process
can also be used within the same version to modify the service VM size or the server worker count.
To update the VM size of an installation, run the following script with parameters updated to the values of your
choice.

PS C:\*Microsoft.Azure.SqlDatabase.Jobs.dll.x.x.xxx.x*\tools>Unblock-File .\UpdateElasticDatabaseJobs.ps1
PS C:\*Microsoft.Azure.SqlDatabase.Jobs.dll.x.x.xxx.x*\tools>.\UpdateElasticDatabaseJobs.ps1 -ServiceVmSize A1 -ServiceWorkerCount 2

PARAMETER DESCRIPTION

ResourceGroupName Identifies the Azure resource group name used when the
Elastic Database job components were initially installed. This
parameter defaults to “__ElasticDatabaseJob”. Since it is not
recommended to change this value, you shouldn't have to
specify this parameter.

ServiceWorkerCount Provides the number of service workers to install. This

Install the Elastic Database jobs components using the Portal

Once you have created an elastic pool, you can install Elastic Database jobs components to enable execution of
administrative tasks against each database in the elastic pool. Unlike when using the Elastic Database jobs
PowerShell APIs, the portal interface is currently restricted to only executing against an existing pool.
Estimated time to complete: 10 minutes.
1. From the dashboard view of the elastic pool via the Azure Portal , click Create job.
2. If you are creating a job for the first time, you must install Elastic Database jobs by clicking PREVIEW
TERMS.
3. Accept the terms by clicking the checkbox.
4. In the "Install services" view, click JOB CREDENTIALS.

5. Type a user name and password for a database admin. As part of the installation, a new Azure SQL
Database server is created. Within this new server, a new database, known as the control database, is
created and used to contain the meta data for Elastic Database jobs. The user name and password created
here are used for the purpose of logging in to the control database. A separate credential is used for script
execution against the databases within the pool.
6. Click the OK button. The components are created for you in a few minutes in a new Resource group. The
new resource group is pinned to the start board, as shown below. Once created, elastic database jobs
(Cloud Service, SQL Database, Service Bus, and Storage) are all created in the group.

7. If you attempt to create or manage a job while elastic database jobs is installing, when providing
Credentials you will see the following message.
If uninstallation is required, delete the resource group. See How to uninstall the Elastic Database job components.

Next steps
Ensure a credential with the appropriate rights for script execution is created on each database in the group, for
more information see Securing your SQL Database. See Creating and managing an Elastic Database jobs to get
started.
Create and manage scaled out Azure SQL Databases
using elastic jobs (preview)
2/16/2017 • 2 min to read • Edit Online

Elastic Database jobs simplify management of groups of databases by executing administrative operations such
as schema changes, credentials management, reference data updates, performance data collection or tenant
(customer) telemetry collection. Elastic Database jobs is currently available through the Azure portal and
PowerShell cmdlets. However, the Azure portal surfaces reduced functionality limited to execution across all
databases in an elastic pool (preview). To access additional features and execution of scripts across a group of
databases including a custom-defined collection or a shard set (created using Elastic Database client library), see
Creating and managing jobs using PowerShell. For more information about jobs, see Elastic Database jobs
overview.

Prerequisites
An Azure subscription. For a free trial, see Free trial.
An elastic pool. See About elastic pools.
Installation of elastic database job service components. See Installing the elastic database job service.

Creating jobs
1. Using the Azure portal, from an existing elastic database job pool, click Create job.
2. Type in the username and password of the database administrator (created at installation of Jobs) for the
jobs control database (metadata storage for jobs).
3. In the Create Job blade, type a name for the job.
4. Type the user name and password to connect to the target databases with sufficient permissions for script
execution to succeed.
5. Paste or type in the T-SQL script.
6. Click Save and then click Run.
Run idempotent jobs
When you run a script against a set of databases, you must be sure that the script is idempotent. That is, the script
must be able to run multiple times, even if it has failed before in an incomplete state. For example, when a script
fails, the job will be automatically retried until it succeeds (within limits, as the retry logic will eventually cease the
retrying). The way to do this is to use the an "IF EXISTS" clause and delete any found instance before creating a
new object. An example is shown here:

IF EXISTS (SELECT name FROM sys.indexes

WHERE name = N'IX_ProductVendor_VendorID')
DROP INDEX IX_ProductVendor_VendorID ON Purchasing.ProductVendor;
GO
CREATE INDEX IX_ProductVendor_VendorID
ON Purchasing.ProductVendor (VendorID);

Alternatively, use an "IF NOT EXISTS" clause before creating a new instance:

IF NOT EXISTS (SELECT name FROM sys.tables WHERE name = 'TestTable')

BEGIN
CREATE TABLE TestTable(
TestTableId INT PRIMARY KEY IDENTITY,
InsertionTime DATETIME2
);
END
GO

INSERT INTO TestTable(InsertionTime) VALUES (sysutcdatetime());

This script then updates the table created previously.

IF NOT EXISTS (SELECT columns.name FROM sys.columns INNER JOIN sys.tables on columns.object_id = tables.object_id WHERE
tables.name = 'TestTable' AND columns.name = 'AdditionalInformation')
BEGIN

ALTER TABLE TestTable

ADD AdditionalInformation NVARCHAR(400);

END
GO

INSERT INTO TestTable(InsertionTime, AdditionalInformation) VALUES (sysutcdatetime(), 'test');

Checking job status

After a job has begun, you can check on its progress.
1. From the elastic pool page, click Manage jobs.
2. Click on the name (a) of a job. The STATUS can be "Completed" or "Failed." The job's details appear (b) with
its date and time of creation and running. The list (c) below the that shows the progress of the script against
each database in the pool, giving its date and time details.

Checking failed jobs

If a job fails, a log of its execution can found. Click the name of the failed job to see its details.
Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Create and manage SQL Database elastic jobs using
PowerShell (preview)
2/16/2017 • 18 min to read • Edit Online

The PowerShell APIs for Elastic Database jobs (in preview), let you define a group of databases against which
scripts will execute. This article shows how to create and manage Elastic Database jobs using PowerShell
cmdlets. See Elastic jobs overview.

Prerequisites
An Azure subscription. For a free trial, see Free one-month trial.
A set of databases created with the Elastic Database tools. See Get started with Elastic Database tools.
Azure PowerShell. For detailed information, see How to install and configure Azure PowerShell.
Elastic Database jobs PowerShell package: See Installing Elastic Database jobs
Select your Azure subscription
To select the subscription you need your subscription Id (-SubscriptionId) or subscription name (-
SubscriptionName). If you have multiple subscriptions you can run the Get-AzureRmSubscription cmdlet and
copy the desired subscription information from the result set. Once you have your subscription information, run
the following commandlet to set this subscription as the default, namely the target for creating and managing
jobs:

Select-AzureRmSubscription -SubscriptionId {SubscriptionID}

The PowerShell ISE is recommended for usage to develop and execute PowerShell scripts against the Elastic
Database jobs.

Elastic Database jobs objects

The following table lists out all the object types of Elastic Database jobs along with its description and relevant
PowerShell APIs.

OBJECT TYPE DESCRIPTION RELATED POWERSHELL APIS

Credential Username and password to use when Get-AzureSqlJobCredential

connecting to databases for execution
of scripts or application of DACPACs. New-AzureSqlJobCredential
The password is encrypted before Set-AzureSqlJobCredential
sending to and storing in the Elastic
Database Jobs database. The
password is decrypted by the
Elastic Database Jobs service via the
credential created and uploaded
from the installation script.
Script Transact-SQL script to be used for Get-AzureSqlJobContent
execution across databases. The script
should be authored to be idempotent Get-AzureSqlJobContentDefinition
since the service will retry execution of New-AzureSqlJobContent
the script upon failures.
Set-AzureSqlJobContentDefinition

DACPAC Data-tier application package to be Get-AzureSqlJobContent

applied across databases.
New-AzureSqlJobContent
Set-AzureSqlJobContentDefinition

Database Target Database and server name pointing to Get-AzureSqlJobTarget

an Azure SQL Database.
New-AzureSqlJobTarget

Shard Map Target Combination of a database target and a Get-AzureSqlJobTarget

credential to be used to determine
information stored within an Elastic New-AzureSqlJobTarget
Database shard map. Set-AzureSqlJobTarget

Custom Collection Target Defined group of databases to Get-AzureSqlJobTarget

collectively use for execution.
New-AzureSqlJobTarget

Custom Collection Child Target Database target that is referenced from Add-AzureSqlJobChildTarget
a custom collection.
Remove-AzureSqlJobChildTarget

Job Definition of parameters for a job Get-AzureSqlJob

that can be used to trigger
execution or to fulfill a schedule. New-AzureSqlJob
Set-AzureSqlJob

Job Execution Container of tasks necessary to Get-AzureSqlJobExecution

fulfill either executing a script or
applying a DACPAC to a target Start-AzureSqlJobExecution
using credentials for database Stop-AzureSqlJobExecution
connections with failures handled in
accordance to an execution policy. Wait-AzureSqlJobExecution

Job Task Execution Single unit of work to fulfill a job. Get-AzureSqlJobExecution

If a job task is not able to Start-AzureSqlJobExecution
successfully execute, the resulting
exception message will be logged Stop-AzureSqlJobExecution
and a new matching job task will be Wait-AzureSqlJobExecution
created and executed in accordance
to the specified execution policy.
Job Execution Policy Controls job execution timeouts, Get-AzureSqlJobExecutionPolicy
retry limits and intervals between
retries. New-AzureSqlJobExecutionPolicy

Elastic Database jobs includes a Set-AzureSqlJobExecutionPolicy

default job execution policy which
cause essentially infinite retries of
job task failures with exponential
backoff of intervals between each
retry.

Schedule Time based specification for Get-AzureSqlJobSchedule

execution to take place either on a
reoccurring interval or at a single New-AzureSqlJobSchedule
time. Set-AzureSqlJobSchedule

Job Triggers A mapping between a job and a New-AzureSqlJobTrigger

schedule to trigger job execution
according to the schedule. Remove-AzureSqlJobTrigger

Supported Elastic Database jobs group types

The job executes Transact-SQL (T-SQL) scripts or application of DACPACs across a group of databases. When a job
is submitted to be executed across a group of databases, the job “expands” the into child jobs where each
performs the requested execution against a single database in the group.
There are two types of groups that you can create:
Shard Map group: When a job is submitted to target a shard map, the job queries the shard map to determine
its current set of shards, and then creates child jobs for each shard in the shard map.
Custom Collection group: A custom defined set of databases. When a job targets a custom collection, it creates
child jobs for each database currently in the custom collection.

To set the Elastic Database jobs connection

A connection needs to be set to the jobs control database prior to using the jobs APIs. Running this cmdlet
triggers a credential window to pop up requesting the user name and password created when installing Elastic
Database jobs. All examples provided within this topic assume that this first step has already been performed.
Open a connection to the Elastic Database jobs:

Use-AzureSqlJobConnection -CurrentAzureSubscription

Encrypted credentials within the Elastic Database jobs

Database credentials can be inserted into the jobs control database with its password encrypted. It is necessary to
store credentials to enable jobs to be executed at a later time, (using job schedules).
Encryption works through a certificate created as part of the installation script. The installation script creates and
uploads the certificate into the Azure Cloud Service for decryption of the stored encrypted passwords. The Azure
Cloud Service later stores the public key within the jobs control database which enables the PowerShell API or
Azure Classic Portal interface to encrypt a provided password without requiring the certificate to be locally
installed.
The credential passwords are encrypted and secure from users with read-only access to Elastic Database jobs
objects. But it is possible for a malicious user with read-write access to Elastic Database Jobs objects to extract a
password. Credentials are designed to be reused across job executions. Credentials are passed to target databases
when establishing connections. There are currently no restrictions on the target databases used for each
credential, malicious user could add a database target for a database under the malicious user's control. The user
could subsequently start a job targeting this database to gain the credential's password.
Security best practices for Elastic Database jobs include:
Limit usage of the APIs to trusted individuals.
Credentials should have the least privileges necessary to perform the job task. More information can be seen
within this Authorization and Permissions SQL Server MSDN article.
To create an encrypted credential for job execution across databases
To create a new encrypted credential, the Get-Credential cmdlet prompts for a user name and password that can
be passed to the New-AzureSqlJobCredential cmdlet.

$credentialName = "{Credential Name}"

$databaseCredential = Get-Credential
$credential = New-AzureSqlJobCredential -Credential $databaseCredential -CredentialName $credentialName
Write-Output $credential

To update credentials
When passwords change, use the Set-AzureSqlJobCredential cmdlet and set the CredentialName parameter.

$credentialName = "{Credential Name}"

Set-AzureSqlJobCredential -CredentialName $credentialName -Credential $credential

To define an Elastic Database shard map target

To execute a job against all databases in a shard set (created using Elastic Database client library), use a shard map
as the database target. This example requires a sharded application created using the Elastic Database client
library. See Getting started with Elastic Database tools sample.
The shard map manager database must be set as a database target and then the specific shard map must be
specified as a target.

$shardMapCredentialName = "{Credential Name}"

$shardMapDatabaseName = "{ShardMapDatabaseName}" #example: ElasticScaleStarterKit_ShardMapManagerDb
$shardMapDatabaseServerName = "{ShardMapServerName}"
$shardMapName = "{MyShardMap}" #example: CustomerIDShardMap
$shardMapDatabaseTarget = New-AzureSqlJobTarget -DatabaseName $shardMapDatabaseName -ServerName $shardMapDatabaseServerName
$shardMapTarget = New-AzureSqlJobTarget -ShardMapManagerCredentialName $shardMapCredentialName -
ShardMapManagerDatabaseName $shardMapDatabaseName -ShardMapManagerServerName $shardMapDatabaseServerName -
ShardMapName $shardMapName
Write-Output $shardMapTarget

Create a T-SQL Script for execution across databases

When creating T-SQL scripts for execution, it is highly recommended to build them to be idempotent and resilient
against failures. Elastic Database jobs will retry execution of a script whenever execution encounters a failure,
regardless of the classification of the failure.
Use the New-AzureSqlJobContent cmdlet to create and save a script for execution and set the -ContentName
and -CommandText parameters.
$scriptName = "Create a TestTable"

$scriptCommandText = "
IF NOT EXISTS (SELECT name FROM sys.tables WHERE name = 'TestTable')
BEGIN
CREATE TABLE TestTable(
TestTableId INT PRIMARY KEY IDENTITY,
InsertionTime DATETIME2
);
END
GO
INSERT INTO TestTable(InsertionTime) VALUES (sysutcdatetime());
GO"

$script = New-AzureSqlJobContent -ContentName $scriptName -CommandText $scriptCommandText

Write-Output $script

Create a new script from a file

If the T-SQL script is defined within a file, use this to import the script:

$scriptName = "My Script Imported from a File"

$scriptPath = "{Path to SQL File}"
$scriptCommandText = Get-Content -Path $scriptPath
$script = New-AzureSqlJobContent -ContentName $scriptName -CommandText $scriptCommandText
Write-Output $script

To update a T -SQL script for execution across databases

This PowerShell script updates the T-SQL command text for an existing script.
Set the following variables to reflect the desired script definition to be set:

$scriptName = "Create a TestTable"

$scriptUpdateComment = "Adding AdditionalInformation column to TestTable"
$scriptCommandText = "
IF NOT EXISTS (SELECT name FROM sys.tables WHERE name = 'TestTable')
BEGIN
CREATE TABLE TestTable(
TestTableId INT PRIMARY KEY IDENTITY,
InsertionTime DATETIME2
);
END
GO

IF NOT EXISTS (SELECT columns.name FROM sys.columns INNER JOIN sys.tables on columns.object_id = tables.object_id WHERE
tables.name = 'TestTable' AND columns.name = 'AdditionalInformation')
BEGIN
ALTER TABLE TestTable
ADD AdditionalInformation NVARCHAR(400);
END
GO

INSERT INTO TestTable(InsertionTime, AdditionalInformation) VALUES (sysutcdatetime(), 'test');

GO"

To update the definition to an existing script

Set-AzureSqlJobContentDefinition -ContentName $scriptName -CommandText $scriptCommandText -Comment $scriptUpdateComment

To create a job to execute a script across a shard map
This PowerShell script starts a job for execution of a script across each shard in an Elastic Scale shard map.
Set the following variables to reflect the desired script and target:

$jobName = "{Job Name}"

$scriptName = "{Script Name}"
$shardMapServerName = "{Shard Map Server Name}"
$shardMapDatabaseName = "{Shard Map Database Name}"
$shardMapName = "{Shard Map Name}"
$credentialName = "{Credential Name}"
$shardMapTarget = Get-AzureSqlJobTarget -ShardMapManagerDatabaseName $shardMapDatabaseName -ShardMapManagerServerName
$shardMapServerName -ShardMapName $shardMapName
$job = New-AzureSqlJob -ContentName $scriptName -CredentialName $credentialName -JobName $jobName -TargetId
$shardMapTarget.TargetId
Write-Output $job

To execute a job
This PowerShell script executes an existing job:
Update the following variable to reflect the desired job name to have executed:

$jobName = "{Job Name}"

$jobExecution = Start-AzureSqlJobExecution -JobName $jobName
Write-Output $jobExecution

To retrieve the state of a single job execution

Use the Get-AzureSqlJobExecution cmdlet and set the JobExecutionId parameter to view the state of job
execution.

$jobExecutionId = "{Job Execution Id}"

$jobExecution = Get-AzureSqlJobExecution -JobExecutionId $jobExecutionId
Write-Output $jobExecution

Use the same Get-AzureSqlJobExecution cmdlet with the IncludeChildren parameter to view the state of child
job executions, namely the specific state for each job execution against each database targeted by the job.

$jobExecutionId = "{Job Execution Id}"

$jobExecutions = Get-AzureSqlJobExecution -JobExecutionId $jobExecutionId -IncludeChildren
Write-Output $jobExecutions

To view the state across multiple job executions

The Get-AzureSqlJobExecution cmdlet has multiple optional parameters that can be used to display multiple
job executions, filtered through the provided parameters. The following demonstrates some of the possible ways
to use Get-AzureSqlJobExecution:
Retrieve all active top level job executions:

Get-AzureSqlJobExecution
Retrieve all top level job executions, including inactive job executions:

Get-AzureSqlJobExecution -IncludeInactive

Retrieve all child job executions of a provided job execution ID, including inactive job executions:

$parentJobExecutionId = "{Job Execution Id}"

Get-AzureSqlJobExecution -AzureSqlJobExecution -JobExecutionId $parentJobExecutionId -IncludeInactive -IncludeChildren

Retrieve all job executions created using a schedule / job combination, including inactive jobs:

$jobName = "{Job Name}"

$scheduleName = "{Schedule Name}"
Get-AzureSqlJobExecution -JobName $jobName -ScheduleName $scheduleName -IncludeInactive

Retrieve all jobs targeting a specified shard map, including inactive jobs:

$shardMapServerName = "{Shard Map Server Name}"

$shardMapDatabaseName = "{Shard Map Database Name}"
$shardMapName = "{Shard Map Name}"
$target = Get-AzureSqlJobTarget -ShardMapManagerDatabaseName $shardMapDatabaseName -ShardMapManagerServerName
$shardMapServerName -ShardMapName $shardMapName
Get-AzureSqlJobExecution -TargetId $target.TargetId -IncludeInactive

Retrieve all jobs targeting a specified custom collection, including inactive jobs:

$customCollectionName = "{Custom Collection Name}"

$target = Get-AzureSqlJobTarget -CustomCollectionName $customCollectionName
Get-AzureSqlJobExecution -TargetId $target.TargetId -IncludeInactive

Retrieve the list of job task executions within a specific job execution:

$jobExecutionId = "{Job Execution Id}"

$jobTaskExecutions = Get-AzureSqlJobTaskExecution -JobExecutionId $jobExecutionId
Write-Output $jobTaskExecutions

Retrieve job task execution details:

The following PowerShell script can be used to view the details of a job task execution, which is particularly useful
when debugging execution failures.

$jobTaskExecutionId = "{Job Task Execution Id}"

$jobTaskExecution = Get-AzureSqlJobTaskExecution -JobTaskExecutionId $jobTaskExecutionId
Write-Output $jobTaskExecution

To retrieve failures within job task executions

The JobTaskExecution object includes a property for the lifecycle of the task along with a message property. If a
job task execution failed, the lifecycle property will be set to Failed and the message property will be set to the
resulting exception message and its stack. If a job did not succeed, it is important to view the details of job tasks
that did not succeed for a given job.
$jobExecutionId = "{Job Execution Id}"
$jobTaskExecutions = Get-AzureSqlJobTaskExecution -JobExecutionId $jobExecutionId
Foreach($jobTaskExecution in $jobTaskExecutions)
{
if($jobTaskExecution.Lifecycle -ne 'Succeeded')
{
Write-Output $jobTaskExecution
}
}

To wait for a job execution to complete

The following PowerShell script can be used to wait for a job task to complete:

$jobExecutionId = "{Job Execution Id}"

Wait-AzureSqlJobExecution -JobExecutionId $jobExecutionId

Create a custom execution policy

Elastic Database jobs supports creating custom execution policies that can be applied when starting jobs.
Execution policies currently allow for defining:
Name: Identifier for the execution policy.
Job Timeout: Total time before a job will be canceled by Elastic Database Jobs.
Initial Retry Interval: Interval to wait before first retry.
Maximum Retry Interval: Cap of retry intervals to use.
Retry Interval Backoff Coefficient: Coefficient used to calculate the next interval between retries. The following
formula is used: (Initial Retry Interval) * Math.pow((Interval Backoff Coefficient), (Number of Retries) - 2).
Maximum Attempts: The maximum number of retry attempts to perform within a job.
The default execution policy uses the following values:
Name: Default execution policy
Job Timeout: 1 week
Initial Retry Interval: 100 milliseconds
Maximum Retry Interval: 30 minutes
Retry Interval Coefficient: 2
Maximum Attempts: 2,147,483,647
Create the desired execution policy:

$executionPolicyName = "{Execution Policy Name}"

$initialRetryInterval = New-TimeSpan -Seconds 10
$jobTimeout = New-TimeSpan -Minutes 30
$maximumAttempts = 999999
$maximumRetryInterval = New-TimeSpan -Minutes 1
$retryIntervalBackoffCoefficient = 1.5
$executionPolicy = New-AzureSqlJobExecutionPolicy -ExecutionPolicyName $executionPolicyName -InitialRetryInterval $initialRetryInterval -
JobTimeout $jobTimeout -MaximumAttempts $maximumAttempts -MaximumRetryInterval $maximumRetryInterval
-RetryIntervalBackoffCoefficient $retryIntervalBackoffCoefficient
Write-Output $executionPolicy

Update a custom execution policy

Update the desired execution policy to update:
$executionPolicyName = "{Execution Policy Name}"
$initialRetryInterval = New-TimeSpan -Seconds 15
$jobTimeout = New-TimeSpan -Minutes 30
$maximumAttempts = 999999
$maximumRetryInterval = New-TimeSpan -Minutes 1
$retryIntervalBackoffCoefficient = 1.5
$updatedExecutionPolicy = Set-AzureSqlJobExecutionPolicy -ExecutionPolicyName $executionPolicyName -InitialRetryInterval
$initialRetryInterval -JobTimeout $jobTimeout -MaximumAttempts $maximumAttempts -MaximumRetryInterval $maximumRetryInterval -
RetryIntervalBackoffCoefficient $retryIntervalBackoffCoefficient
Write-Output $updatedExecutionPolicy

Cancel a job
Elastic Database Jobs supports cancellation requests of jobs. If Elastic Database Jobs detects a cancellation request
for a job currently being executed, it will attempt to stop the job.
There are two different ways that Elastic Database Jobs can perform a cancellation:
1. Cancel currently executing tasks: If a cancellation is detected while a task is currently running, a cancellation will
be attempted within the currently executing aspect of the task. For example: If there is a long running query
currently being performed when a cancellation is attempted, there will be an attempt to cancel the query.
2. Canceling task retries: If a cancellation is detected by the control thread before a task is launched for execution,
the control thread will avoid launching the task and declare the request as canceled.
If a job cancellation is requested for a parent job, the cancellation request will be honored for the parent job and
for all of its child jobs.
To submit a cancellation request, use the Stop-AzureSqlJobExecution cmdlet and set the JobExecutionId
parameter.

$jobExecutionId = "{Job Execution Id}"

Stop-AzureSqlJobExecution -JobExecutionId $jobExecutionId

To delete a job and job history asynchronously

Elastic Database jobs supports asynchronous deletion of jobs. A job can be marked for deletion and the system
will delete the job and all its job history after all job executions have completed for the job. The system will not
automatically cancel active job executions.
Invoke Stop-AzureSqlJobExecution to cancel active job executions.
To trigger job deletion, use the Remove-AzureSqlJob cmdlet and set the JobName parameter.

$jobName = "{Job Name}"

Remove-AzureSqlJob -JobName $jobName

To create a custom database target

You can define custom database targets either for direct execution or for inclusion within a custom database
group. For example, because elastic pools are not yet directly supported using PowerShell APIs, you can create a
custom database target and custom database collection target which encompasses all the databases in the pool.
Set the following variables to reflect the desired database information:
$databaseName = "{Database Name}"
$databaseServerName = "{Server Name}"
New-AzureSqlJobDatabaseTarget -DatabaseName $databaseName -ServerName $databaseServerName

To create a custom database collection target

Use the New-AzureSqlJobTarget cmdlet to define a custom database collection target to enable execution
across multiple defined database targets. After creating a database group, databases can be associated with the
custom collection target.
Set the following variables to reflect the desired custom collection target configuration:

$customCollectionName = "{Custom Database Collection Name}"

New-AzureSqlJobTarget -CustomCollectionName $customCollectionName

To add databases to a custom database collection target

To add a database to a specific custom collection use the Add-AzureSqlJobChildTarget cmdlet.

$databaseServerName = "{Database Server Name}"

$databaseName = "{Database Name}"
$customCollectionName = "{Custom Database Collection Name}"
Add-AzureSqlJobChildTarget -CustomCollectionName $customCollectionName -DatabaseName $databaseName -ServerName
$databaseServerName

Review the databases within a custom database collection target

Use the Get-AzureSqlJobTarget cmdlet to retrieve the child databases within a custom database collection
target.

$customCollectionName = "{Custom Database Collection Name}"

$target = Get-AzureSqlJobTarget -CustomCollectionName $customCollectionName
$childTargets = Get-AzureSqlJobTarget -ParentTargetId $target.TargetId
Write-Output $childTargets

Create a job to execute a script across a custom database collection target

Use the New-AzureSqlJob cmdlet to create a job against a group of databases defined by a custom database
collection target. Elastic Database jobs will expand the job into multiple child jobs each corresponding to a
database associated with the custom database collection target and ensure that the script is executed against each
database. Again, it is important that scripts are idempotent to be resilient to retries.

$jobName = "{Job Name}"

Data collection across databases

You can use a job to execute a query across a group of databases and send the results to a specific table. The table
can be queried after the fact to see the query’s results from each database. This provides an asynchronous method
to execute a query across many databases. Failed attempts are handled automatically via retries.
The specified destination table will be automatically created if it does not yet exist. The new table matches the
schema of the returned result set. If a script returns multiple result sets, Elastic Database jobs will only send the
first to the destination table.
The following PowerShell script executes a script and collects its results into a specified table. This script assumes
that a T-SQL script has been created which outputs a single result set and that a custom database collection target
has been created.
This script uses the Get-AzureSqlJobTarget cmdlet. Set the parameters for script, credentials, and execution
target:

$jobName = "{Job Name}"

$scriptName = "{Script Name}"
$executionCredentialName = "{Execution Credential Name}"
$customCollectionName = "{Custom Collection Name}"
$destinationCredentialName = "{Destination Credential Name}"
$destinationServerName = "{Destination Server Name}"
$destinationDatabaseName = "{Destination Database Name}"
$destinationSchemaName = "{Destination Schema Name}"
$destinationTableName = "{Destination Table Name}"
$target = Get-AzureSqlJobTarget -CustomCollectionName $customCollectionName

To create and start a job for data collection scenarios

This script uses the Start-AzureSqlJobExecution cmdlet.

$job = New-AzureSqlJob -JobName $jobName

-CredentialName $executionCredentialName
-ContentName $scriptName
-ResultSetDestinationServerName $destinationServerName
-ResultSetDestinationDatabaseName $destinationDatabaseName
-ResultSetDestinationSchemaName $destinationSchemaName
-ResultSetDestinationTableName $destinationTableName
-ResultSetDestinationCredentialName $destinationCredentialName
-TargetId $target.TargetId
Write-Output $job
$jobExecution = Start-AzureSqlJobExecution -JobName $jobName
Write-Output $jobExecution

To schedule a job execution trigger

The following PowerShell script can be used to create a recurring schedule. This script uses a minute interval, but
New-AzureSqlJobSchedule also supports -DayInterval, -HourInterval, -MonthInterval, and -WeekInterval
parameters. Schedules that execute only once can be created by passing -OneTime.
Create a new schedule:

$scheduleName = "Every one minute"

$minuteInterval = 1
$startTime = (Get-Date).ToUniversalTime()
$schedule = New-AzureSqlJobSchedule
-MinuteInterval $minuteInterval
-ScheduleName $scheduleName
-StartTime $startTime
Write-Output $schedule

To trigger a job executed on a time schedule

A job trigger can be defined to have a job executed according to a time schedule. The following PowerShell script
can be used to create a job trigger.
Use New-AzureSqlJobTrigger and set the following variables to correspond to the desired job and schedule:

$jobName = "{Job Name}"

$scheduleName = "{Schedule Name}"
$jobTrigger = New-AzureSqlJobTrigger
-ScheduleName $scheduleName
-JobName $jobName
Write-Output $jobTrigger

To remove a scheduled association to stop job from executing on schedule

To discontinue reoccurring job execution through a job trigger, the job trigger can be removed. Remove a job
trigger to stop a job from being executed according to a schedule using the Remove-AzureSqlJobTrigger
cmdlet.

$jobName = "{Job Name}"

$scheduleName = "{Schedule Name}"
Remove-AzureSqlJobTrigger
-ScheduleName $scheduleName
-JobName $jobName

Retrieve job triggers bound to a time schedule

The following PowerShell script can be used to obtain and display the job triggers registered to a particular time
schedule.

$scheduleName = "{Schedule Name}"

$jobTriggers = Get-AzureSqlJobTrigger -ScheduleName $scheduleName
Write-Output $jobTriggers

To retrieve job triggers bound to a job

Use Get-AzureSqlJobTrigger to obtain and display schedules containing a registered job.

$jobName = "{Job Name}"

$jobTriggers = Get-AzureSqlJobTrigger -JobName $jobName
Write-Output $jobTriggers

To create a data-tier application (DACPAC) for execution across

databases
To create a DACPAC, see Data-Tier applications. To deploy a DACPAC, use the New-AzureSqlJobContent cmdlet.
The DACPAC must be accessible to the service. It is recommended to upload a created DACPAC to Azure Storage
and create a Shared Access Signature for the DACPAC.

$dacpacUri = "{Uri}"
$dacpacName = "{Dacpac Name}"
$dacpac = New-AzureSqlJobContent -DacpacUri $dacpacUri -ContentName $dacpacName
Write-Output $dacpac

To update a data-tier application (DACPAC ) for execution across databases

Existing DACPACs registered within Elastic Database Jobs can be updated to point to new URIs. Use the Set-
AzureSqlJobContentDefinition cmdlet to update the DACPAC URI on an existing registered DACPAC:
$dacpacName = "{Dacpac Name}"
$newDacpacUri = "{Uri}"
$updatedDacpac = Set-AzureSqlJobDacpacDefinition -ContentName $dacpacName -DacpacUri $newDacpacUri
Write-Output $updatedDacpac

To create a job to apply a data-tier application (DACPAC) across

databases
After a DACPAC has been created within Elastic Database Jobs, a job can be created to apply the DACPAC across a
group of databases. The following PowerShell script can be used to create a DACPAC job across a custom
collection of databases:

$jobName = "{Job Name}"

$dacpacName = "{Dacpac Name}"
$customCollectionName = "{Custom Collection Name}"
$credentialName = "{Credential Name}"
$target = Get-AzureSqlJobTarget
-CustomCollectionName $customCollectionName
$job = New-AzureSqlJob
-JobName $jobName
-CredentialName $credentialName
-ContentName $dacpacName -TargetId $target.TargetId
Write-Output $job

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For questions,
please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database
feedback forum.
Getting started with Elastic Database jobs
4/10/2017 • 13 min to read • Edit Online

Elastic Database jobs (preview) for Azure SQL Database allows you to reliability execute T-SQL scripts that span
multiple databases while automatically retrying and providing eventual completion guarantees. For more
information about the Elastic Database job feature, please see the feature overview page.
This topic extends the sample found in Getting started with Elastic Database tools. When completed, you will:
learn how to create and manage jobs that manage a group of related databases. It is not required to use the
Elastic Scale tools in order to take advantage of the benefits of Elastic jobs.

Prerequisites
Download and run the Getting started with Elastic Database tools sample.

Create a shard map manager using the sample app

Here you will create a shard map manager along with several shards, followed by insertion of data into the
shards. If you already have shards set up with sharded data in them, you can skip the following steps and move
to the next section.
1. Build and run the Getting started with Elastic Database tools sample application. Follow the steps until
step 7 in the section Download and run the sample app. At the end of Step 7, you will see the following
command prompt:

2. In the command window, type "1" and press Enter. This creates the shard map manager, and adds two
shards to the server. Then type "3" and press Enter; repeat this action four times. This inserts sample data
rows in your shards.
3. The Azure Portal should show three new databases:
At this point, we will create a custom database collection that reflects all the databases in the shard map.
This will allow us to create and execute a job that add a new table across shards.
Here we would usually create a shard map target, using the New-AzureSqlJobTarget cmdlet. The shard map
manager database must be set as a database target and then the specific shard map is specified as a target.
Instead, we are going to enumerate all the databases in the server and add the databases to the new custom
collection with the exception of master database.

Creates a custom collection and add all databases in the server to the
custom collection target with the exception of master.
$customCollectionName = "dbs_in_server"
New-AzureSqlJobTarget -CustomCollectionName $customCollectionName
$ResourceGroupName = "ddove_samples"
$ServerName = "samples"
$dbsinserver = Get-AzureRMSqlDatabase -ResourceGroupName $ResourceGroupName -ServerName $ServerName
$dbsinserver | %{
$currentdb = $_.DatabaseName
$ErrorActionPreference = "Stop"
Write-Output ""

Try
{
New-AzureSqlJobTarget -ServerName $ServerName -DatabaseName $currentdb | Write-Output
}
Catch
{
$ErrorMessage = $_.Exception.Message
$ErrorCategory = $_.CategoryInfo.Reason

if ($ErrorCategory -eq 'UniqueConstraintViolatedException')

{
Write-Host $currentdb "is already a database target."
}

else
{
throw $_
}

Try
{
if ($currentdb -eq "master")
{
Write-Host $currentdb "will not be added custom collection target" $CustomCollectionName "."
}

else
{
Add-AzureSqlJobChildTarget -CustomCollectionName $CustomCollectionName -ServerName $ServerName -DatabaseName $currentdb
Write-Host $currentdb "was added to" $CustomCollectionName "."
}

}
Catch
{
$ErrorMessage = $_.Exception.Message
$ErrorCategory = $_.CategoryInfo.Reason

if ($ErrorCategory -eq 'UniqueConstraintViolatedException')

{
Write-Host $currentdb "is already in the custom collection target" $CustomCollectionName"."
}

else
{
throw $_
}
}
$ErrorActionPreference = "Continue"
}

Create a T-SQL Script for execution across databases

$scriptName = "NewTable"
$scriptCommandText = "
IF NOT EXISTS (SELECT name FROM sys.tables WHERE name = 'Test')
BEGIN
CREATE TABLE Test(
TestId INT PRIMARY KEY IDENTITY,
InsertionTime DATETIME2
);
END
GO
INSERT INTO Test(InsertionTime) VALUES (sysutcdatetime());
GO"

$script = New-AzureSqlJobContent -ContentName $scriptName -CommandText $scriptCommandText

Write-Output $script

Create the job to execute a script across the custom group of

databases
$jobName = "create on server dbs"
$scriptName = "NewTable"
$customCollectionName = "dbs_in_server"
$credentialName = "ddove66"
$target = Get-AzureSqlJobTarget -CustomCollectionName $customCollectionName
$job = New-AzureSqlJob -JobName $jobName -CredentialName $credentialName -ContentName $scriptName -TargetId $target.TargetId
Write-Output $job

Execute the job

The following PowerShell script can be used to execute an existing job:
Update the following variable to reflect the desired job name to have executed:

$jobName = "create on server dbs"

$jobExecution = Start-AzureSqlJobExecution -JobName $jobName
Write-Output $jobExecution

Retrieve the state of a single job execution

Use the same Get-AzureSqlJobExecution cmdlet with the IncludeChildren parameter to view the state of
child job executions, namely the specific state for each job execution against each database targeted by the job.

$jobExecutionId = "{Job Execution Id}"

$jobExecutions = Get-AzureSqlJobExecution -JobExecutionId $jobExecutionId -IncludeChildren
Write-Output $jobExecutions

View the state across multiple job executions

Retrieve all top level job executions, including inactive job executions:

Get-AzureSqlJobExecution -IncludeInactive

Retrieve all child job executions of a provided job execution ID, including inactive job executions:

$parentJobExecutionId = "{Job Execution Id}"

Get-AzureSqlJobExecution -AzureSqlJobExecution -JobExecutionId $parentJobExecutionId -IncludeInactive -IncludeChildren

Retrieve all job executions created using a schedule / job combination, including inactive jobs:

$jobName = "{Job Name}"

$scheduleName = "{Schedule Name}"
Get-AzureSqlJobExecution -JobName $jobName -ScheduleName $scheduleName -IncludeInactive

Retrieve all jobs targeting a specified shard map, including inactive jobs:

$shardMapServerName = "{Shard Map Server Name}"

Retrieve all jobs targeting a specified custom collection, including inactive jobs:

$customCollectionName = "{Custom Collection Name}"

$target = Get-AzureSqlJobTarget -CustomCollectionName $customCollectionName
Get-AzureSqlJobExecution -TargetId $target.TargetId -IncludeInactive

Retrieve the list of job task executions within a specific job execution:

$jobExecutionId = "{Job Execution Id}"

$jobTaskExecutions = Get-AzureSqlJobTaskExecution -JobExecutionId $jobExecutionId
Write-Output $jobTaskExecutions

Retrieve job task execution details:

The following PowerShell script can be used to view the details of a job task execution, which is particularly useful
when debugging execution failures.

$jobTaskExecutionId = "{Job Task Execution Id}"

$jobTaskExecution = Get-AzureSqlJobTaskExecution -JobTaskExecutionId $jobTaskExecutionId
Write-Output $jobTaskExecution

Retrieve failures within job task executions

The JobTaskExecution object includes a property for the Lifecycle of the task along with a Message property. If a
job task execution failed, the Lifecycle property will be set to Failed and the Message property will be set to the
resulting exception message and its stack. If a job did not succeed, it is important to view the details of job tasks
that did not succeed for a given job.

$jobExecutionId = "{Job Execution Id}"

$jobTaskExecutions = Get-AzureSqlJobTaskExecution -JobExecutionId $jobExecutionId
Foreach($jobTaskExecution in $jobTaskExecutions)
{
if($jobTaskExecution.Lifecycle -ne 'Succeeded')
{
Write-Output $jobTaskExecution
}
}

Waiting for a job execution to complete

The following PowerShell script can be used to wait for a job task to complete:

$jobExecutionId = "{Job Execution Id}"

Wait-AzureSqlJobExecution -JobExecutionId $jobExecutionId

Create a custom execution policy

$executionPolicyName = "{Execution Policy Name}"

$initialRetryInterval = New-TimeSpan -Seconds 10
$jobTimeout = New-TimeSpan -Minutes 30
$maximumAttempts = 999999
$maximumRetryInterval = New-TimeSpan -Minutes 1
$retryIntervalBackoffCoefficient = 1.5
$executionPolicy = New-AzureSqlJobExecutionPolicy -ExecutionPolicyName $executionPolicyName -InitialRetryInterval $initialRetryInterval -
JobTimeout $jobTimeout -MaximumAttempts $maximumAttempts -MaximumRetryInterval $maximumRetryInterval -
RetryIntervalBackoffCoefficient $retryIntervalBackoffCoefficient
Write-Output $executionPolicy
Update a custom execution policy
Update the desired execution policy to update:

$executionPolicyName = "{Execution Policy Name}"

$initialRetryInterval = New-TimeSpan -Seconds 15
$jobTimeout = New-TimeSpan -Minutes 30
$maximumAttempts = 999999
$maximumRetryInterval = New-TimeSpan -Minutes 1
$retryIntervalBackoffCoefficient = 1.5
$updatedExecutionPolicy = Set-AzureSqlJobExecutionPolicy -ExecutionPolicyName $executionPolicyName -InitialRetryInterval
$initialRetryInterval -JobTimeout $jobTimeout -MaximumAttempts $maximumAttempts -MaximumRetryInterval $maximumRetryInterval -
RetryIntervalBackoffCoefficient $retryIntervalBackoffCoefficient
Write-Output $updatedExecutionPolicy

Cancel a job
Elastic Database Jobs supports jobs cancellation requests. If Elastic Database Jobs detects a cancellation request
for a job currently being executed, it will attempt to stop the job.
There are two different ways that Elastic Database Jobs can perform a cancellation:
1. Canceling Currently Executing Tasks: If a cancellation is detected while a task is currently running, a
cancellation will be attempted within the currently executing aspect of the task. For example: If there is a long
running query currently being performed when a cancellation is attempted, there will be an attempt to cancel
the query.
2. Canceling Task Retries: If a cancellation is detected by the control thread before a task is launched for
execution, the control thread will avoid launching the task and declare the request as canceled.
If a job cancellation is requested for a parent job, the cancellation request will be honored for the parent job and
for all of its child jobs.
To submit a cancellation request, use the Stop-AzureSqlJobExecution cmdlet and set the JobExecutionId
parameter.

$jobExecutionId = "{Job Execution Id}"

Stop-AzureSqlJobExecution -JobExecutionId $jobExecutionId

Delete a job by name and the job's history

Elastic Database jobs supports asynchronous deletion of jobs. A job can be marked for deletion and the system
will delete the job and all its job history after all job executions have completed for the job. The system will not
automatically cancel active job executions.
Instead, Stop-AzureSqlJobExecution must be invoked to cancel active job executions.
To trigger job deletion, use the Remove-AzureSqlJob cmdlet and set the JobName parameter.

$jobName = "{Job Name}"

Remove-AzureSqlJob -JobName $jobName

Create a custom database target

Custom database targets can be defined in Elastic Database jobs which can be used either for execution directly
or for inclusion within a custom database group. Since elastic pools are not yet directly supported via the
PowerShell APIs, you simply create a custom database target and custom database collection target which
encompasses all the databases in the pool.
Set the following variables to reflect the desired database information:

$databaseName = "{Database Name}"

$databaseServerName = "{Server Name}"
New-AzureSqlJobDatabaseTarget -DatabaseName $databaseName -ServerName $databaseServerName

Create a custom database collection target

A custom database collection target can be defined to enable execution across multiple defined database targets.
After a database group is created, databases can be associated to the custom collection target.
Set the following variables to reflect the desired custom collection target configuration:

$customCollectionName = "{Custom Database Collection Name}"

New-AzureSqlJobTarget -CustomCollectionName $customCollectionName

Add databases to a custom database collection target

Database targets can be associated with custom database collection targets to create a group of databases.
Whenever a job is created that targets a custom database collection target, it will be expanded to target the
databases associated to the group at the time of execution.
Add the desired database to a specific custom collection:

$serverName = "{Database Server Name}"

Review the databases within a custom database collection target

Use the Get-AzureSqlJobTarget cmdlet to retrieve the child databases within a custom database collection
target.

$customCollectionName = "{Custom Database Collection Name}"

$target = Get-AzureSqlJobTarget -CustomCollectionName $customCollectionName
$childTargets = Get-AzureSqlJobTarget -ParentTargetId $target.TargetId
Write-Output $childTargets

Create a job to execute a script across a custom database collection target

Use the New-AzureSqlJob cmdlet to create a job against a group of databases defined by a custom database
collection target. Elastic Database jobs will expand the job into multiple child jobs each corresponding to a
database associated with the custom database collection target and ensure that the script is executed against
each database. Again, it is important that scripts are idempotent to be resilient to retries.

$jobName = "{Job Name}"

$scriptName = "{Script Name}"
$customCollectionName = "{Custom Collection Name}"
$credentialName = "{Credential Name}"
$target = Get-AzureSqlJobTarget -CustomCollectionName $customCollectionName
$job = New-AzureSqlJob -JobName $jobName -CredentialName $credentialName -ContentName $scriptName -TargetId $target.TargetId
Write-Output $job
Data collection across databases
Elastic Database jobs supports executing a query across a group of databases and sends the results to a
specified database’s table. The table can be queried after the fact to see the query’s results from each database.
This provides an asynchronous mechanism to execute a query across many databases. Failure cases such as one
of the databases being temporarily unavailable are handled automatically via retries.
The specified destination table will be automatically created if it does not yet exist, matching the schema of the
returned result set. If a script execution returns multiple result sets, Elastic Database jobs will only send the first
one to the provided destination table.
The following PowerShell script can be used to execute a script collecting its results into a specified table. This
script assumes that a T-SQL script has been created which outputs a single result set and a custom database
collection target has been created.
Set the following to reflect the desired script, credentials, and execution target:

$jobName = "{Job Name}"

Create and start a job for data collection scenarios

$job = New-AzureSqlJob -JobName $jobName -CredentialName $executionCredentialName -ContentName $scriptName -

ResultSetDestinationServerName $destinationServerName -ResultSetDestinationDatabaseName $destinationDatabaseName -
ResultSetDestinationSchemaName $destinationSchemaName -ResultSetDestinationTableName $destinationTableName -
ResultSetDestinationCredentialName $destinationCredentialName -TargetId $target.TargetId
Write-Output $job
$jobExecution = Start-AzureSqlJobExecution -JobName $jobName
Write-Output $jobExecution

Create a schedule for job execution using a job trigger

The following PowerShell script can be used to create a reoccurring schedule. This script uses a one minute
interval, but New-AzureSqlJobSchedule also supports -DayInterval, -HourInterval, -MonthInterval, and -
WeekInterval parameters. Schedules that execute only once can be created by passing -OneTime.
Create a new schedule:

$scheduleName = "Every one minute"

$minuteInterval = 1
$startTime = (Get-Date).ToUniversalTime()
$schedule = New-AzureSqlJobSchedule -MinuteInterval $minuteInterval -ScheduleName $scheduleName -StartTime $startTime
Write-Output $schedule

Create a job trigger to have a job executed on a time schedule

A job trigger can be defined to have a job executed according to a time schedule. The following PowerShell script
can be used to create a job trigger.
Set the following variables to correspond to the desired job and schedule:
$jobName = "{Job Name}"
$scheduleName = "{Schedule Name}"
$jobTrigger = New-AzureSqlJobTrigger -ScheduleName $scheduleName -JobName $jobName
Write-Output $jobTrigger

Remove a scheduled association to stop job from executing on schedule

$jobName = "{Job Name}"

$scheduleName = "{Schedule Name}"
Remove-AzureSqlJobTrigger -ScheduleName $scheduleName -JobName $jobName

Import elastic database query results to Excel

You can import the results from of a query to an Excel file.
1. Launch Excel 2013.
2. Navigate to the Data ribbon.
3. Click From Other Sources and click From SQL Server.

Next steps
You can now use Excel’s data functions. Use the connection string with your server name, database name and
credentials to connect your BI and data integration tools to the elastic query database. Make sure that SQL Server
is supported as a data source for your tool. Refer to the elastic query database and external tables just like any
other SQL Server database and SQL Server tables that you would connect to with your tool.
Cost
There is no additional charge for using the Elastic Database query feature. However, at this time this feature is
available only on premium databases as an end point, but the shards can be of any service tier.
For pricing information see SQL Database Pricing Details.

Additional resources
Not using elastic database tools yet? Check out our Getting Started Guide and Documentation Map. For
questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the
SQL Database feedback forum.
Uninstall Elastic Database jobs components
1/17/2017 • 1 min to read • Edit Online

Elastic Database jobs components can be uninstalled using either the Portal or PowerShell.

Uninstall Elastic Database jobs components using the Azure portal

1. Open the Azure portal.
2. Navigate to the subscription that contains Elastic Database jobs components, namely the subscription in
which Elastic Database jobs components were installed.
3. Click Browse and click Resource groups.
4. Select the resource group named "__ElasticDatabaseJob".
5. Delete the resource group.

Uninstall Elastic Database jobs components using PowerShell

1. Launch a Microsoft Azure PowerShell command window and navigate to the tools sub-directory under the
Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x folder: Type cd tools.
PS C:*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*>cd tools
2. Execute the .\UninstallElasticDatabaseJobs.ps1 PowerShell script.
PS C:*Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x\tools>Unblock-File .\UninstallElasticDatabaseJobs.ps1 PS
C:\Microsoft.Azure.SqlDatabase.Jobs.x.x.xxxx.x*\tools>.\UninstallElasticDatabaseJobs.ps1
Or simply, execute the following script, assuming default values where used on installation of the components:

$ResourceGroupName = "__ElasticDatabaseJob"
Switch-AzureMode AzureResourceManager

$resourceGroup = Get-AzureResourceGroup -Name $ResourceGroupName

if(!$resourceGroup)
{
Write-Host "The Azure Resource Group: $ResourceGroupName has already been deleted. Elastic database job components are uninstalled."
return
}

Write-Host "Removing the Azure Resource Group: $ResourceGroupName. This may take a few minutes.”
Remove-AzureResourceGroup -Name $ResourceGroupName -Force
Write-Host "Completed removing the Azure Resource Group: $ResourceGroupName. Elastic database job compoennts are now uninstalled."

Next steps
To re-install Elastic Database jobs, see Installing the Elastic Database job service
For an overview of Elastic Database jobs, see Elastic Database jobs overview.
Report across scaled-out cloud databases (preview)
4/10/2017 • 4 min to read • Edit Online

You can create reports from multiple Azure SQL databases from a single connection point using an elastic query.
The databases must be horizontally partitioned (also known as "sharded").
If you have an existing database, see Migrating existing databases to scaled-out databases.
To understand the SQL objects needed to query, see Query across horizontally partitioned databases.

Prerequisites
Download and run the Getting started with Elastic Database tools sample.

Create a shard map manager using the sample app

Here you will create a shard map manager along with several shards, followed by insertion of data into the
shards. If you happen to already have shards setup with sharded data in them, you can skip the following steps
and move to the next section.
1. Build and run the Getting started with Elastic Database tools sample application. Follow the steps until
step 7 in the section Download and run the sample app. At the end of Step 7, you will see the following
command prompt:

2. In the command window, type "1" and press Enter. This creates the shard map manager, and adds two shards
to the server. Then type "3" and press Enter; repeat the action four times. This inserts sample data rows in your
shards.
3. The Azure portal should show three new databases in your server:
At this point, cross-database queries are supported through the Elastic Database client libraries. For
example, use option 4 in the command window. The results from a multi-shard query are always a UNION
ALL of the results from all shards.
In the next section, we create a sample database endpoint that supports richer querying of the data across
shards.

Create an elastic query database

1. Open the Azure portal and log in.
2. Create a new Azure SQL database in the same server as your shard setup. Name the database
"ElasticDBQuery."
NOTE
you can use an existing database. If you can do so, it must not be one of the shards that you would like to execute
your queries on. This database will be used for creating the metadata objects for an elastic database query.

Create database objects

Database -scoped master key and credentials
These are used to connect to the shard map manager and the shards:
1. Open SQL Server Management Studio or SQL Server Data Tools in Visual Studio.
2. Connect to ElasticDBQuery database and execute the following T-SQL commands:

CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<password>';

CREATE DATABASE SCOPED CREDENTIAL ElasticDBQueryCred

WITH IDENTITY = '<username>',
SECRET = '<password>';

"username" and "password" should be the same as login information used in step 6 of Download and run
the sample app in Getting started with elastic database tools.
External data sources
To create an external data source, execute the following command on the ElasticDBQuery database:

CREATE EXTERNAL DATA SOURCE MyElasticDBQueryDataSrc WITH

(TYPE = SHARD_MAP_MANAGER,
LOCATION = '<server_name>.database.windows.net',
DATABASE_NAME = 'ElasticScaleStarterKit_ShardMapManagerDb',
CREDENTIAL = ElasticDBQueryCred,
SHARD_MAP_NAME = 'CustomerIDShardMap'
);

"CustomerIDShardMap" is the name of the shard map, if you created the shard map and shard map manager
using the elastic database tools sample. However, if you used your custom setup for this sample, then it should be
the shard map name you chose in your application.
External tables
Create an external table that matches the Customers table on the shards by executing the following command on
ElasticDBQuery database:

CREATE EXTERNAL TABLE [dbo].[Customers]

( [CustomerId] [int] NOT NULL,
[Name] [nvarchar](256) NOT NULL,
[RegionId] [int] NOT NULL)
WITH
( DATA_SOURCE = MyElasticDBQueryDataSrc,
DISTRIBUTION = SHARDED([CustomerId])
);

Execute a sample elastic database T-SQL query

Once you have defined your external data source and your external tables you can now use full T-SQL over your
external tables.
Execute this query on the ElasticDBQuery database:

select count(CustomerId) from [dbo].[Customers]

You will notice that the query aggregates results from all the shards and gives the following output:

Import elastic database query results to Excel

You can import the results from of a query to an Excel file.
1. Launch Excel 2013.
2. Navigate to the Data ribbon.
3. Click From Other Sources and click From SQL Server.

4. In the Data Connection Wizard type the server name and login credentials. Then click Next.
5. In the dialog box Select the database that contains the data you want, select the ElasticDBQuery
database.
6. Select the Customers table in the list view and click Next. Then click Finish.
7. In the Import Data form, under Select how you want to view this data in your workbook, select Table
and click OK.
All the rows from Customers table, stored in different shards populate the Excel sheet.
You can now use Excel’s powerful data visualization functions. You can use the connection string with your server
name, database name and credentials to connect your BI and data integration tools to the elastic query database.
Make sure that SQL Server is supported as a data source for your tool. You can refer to the elastic query database
and external tables just like any other SQL Server database and SQL Server tables that you would connect to with
your tool.
Cost
There is no additional charge for using the Elastic Database Query feature.
For pricing information see SQL Database Pricing Details.

Next steps
For an overview of elastic query, see Elastic query overview.
For a vertical partitioning tutorial, see Getting started with cross-database query (vertical partitioning).
For syntax and sample queries for vertically partitioned data, see Querying vertically partitioned data)
For syntax and sample queries for horizontally partitioned data, see Querying horizontally partitioned data)
See sp_execute _remote for a stored procedure that executes a Transact-SQL statement on a single remote
Azure SQL Database or set of databases serving as shards in a horizontal partitioning scheme.
Get started with cross-database queries (vertical
partitioning) (preview)
2/21/2017 • 2 min to read • Edit Online

Elastic database query (preview) for Azure SQL Database allows you to run T-SQL queries that span multiple
databases using a single connection point. This topic applies to vertically partitioned databases.
When completed, you will: learn how to configure and use an Azure SQL Database to perform queries that span
multiple related databases.
For more information about the elastic database query feature, please see Azure SQL Database elastic database
query overview.

Prerequisites
You must possess ALTER ANY EXTERNAL DATA SOURCE permission. This permission is included with the ALTER
DATABASE permission. ALTER ANY EXTERNAL DATA SOURCE permissions are needed to refer to the underlying
data source.

Create the sample databases

To start with, we need to create two databases, Customers and Orders, either in the same or different logical
servers.
Execute the following queries on the Orders database to create the OrderInformation table and input the sample
data.

CREATE TABLE [dbo].[OrderInformation](

[OrderID] [int] NOT NULL,
[CustomerID] [int] NOT NULL
)
INSERT INTO [dbo].[OrderInformation] ([OrderID], [CustomerID]) VALUES (123, 1)
INSERT INTO [dbo].[OrderInformation] ([OrderID], [CustomerID]) VALUES (149, 2)
INSERT INTO [dbo].[OrderInformation] ([OrderID], [CustomerID]) VALUES (857, 2)
INSERT INTO [dbo].[OrderInformation] ([OrderID], [CustomerID]) VALUES (321, 1)
INSERT INTO [dbo].[OrderInformation] ([OrderID], [CustomerID]) VALUES (564, 8)

Now, execute following query on the Customers database to create the CustomerInformation table and input
the sample data.

CREATE TABLE [dbo].[CustomerInformation](

[CustomerID] [int] NOT NULL,
[CustomerName] [varchar](50) NULL,
[Company] [varchar](50) NULL
CONSTRAINT [CustID] PRIMARY KEY CLUSTERED ([CustomerID] ASC)
)
INSERT INTO [dbo].[CustomerInformation] ([CustomerID], [CustomerName], [Company]) VALUES (1, 'Jack', 'ABC')
INSERT INTO [dbo].[CustomerInformation] ([CustomerID], [CustomerName], [Company]) VALUES (2, 'Steve', 'XYZ')
INSERT INTO [dbo].[CustomerInformation] ([CustomerID], [CustomerName], [Company]) VALUES (3, 'Lylla', 'MNO')

Create database objects

Database scoped master key and credentials
1. Open SQL Server Management Studio or SQL Server Data Tools in Visual Studio.
2. Connect to the Orders database and execute the following T-SQL commands:

CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<password>';

CREATE DATABASE SCOPED CREDENTIAL ElasticDBQueryCred
WITH IDENTITY = '<username>',
SECRET = '<password>';

The "username" and "password" should be the username and password used to login into the Customers
database. Authentication using Azure Active Directory with elastic queries is not currently supported.
External data sources
To create an external data source, execute the following command on the Orders database:

CREATE EXTERNAL DATA SOURCE MyElasticDBQueryDataSrc WITH

(TYPE = RDBMS,
LOCATION = '<server_name>.database.windows.net',
DATABASE_NAME = 'Customers',
CREDENTIAL = ElasticDBQueryCred,
);

External tables
Create an external table on the Orders database, which matches the definition of the CustomerInformation table:

CREATE EXTERNAL TABLE [dbo].[CustomerInformation]

( [CustomerID] [int] NOT NULL,
[CustomerName] [varchar](50) NOT NULL,
[Company] [varchar](50) NOT NULL)
WITH
( DATA_SOURCE = MyElasticDBQueryDataSrc)

Execute a sample elastic database T-SQL query

Once you have defined your external data source and your external tables you can now use T-SQL to query your
external tables. Execute this query on the Orders database:

SELECT OrderInformation.CustomerID, OrderInformation.OrderId, CustomerInformation.CustomerName, CustomerInformation.Company

FROM OrderInformation
INNER JOIN CustomerInformation
ON CustomerInformation.CustomerID = OrderInformation.CustomerID

Cost
Currently, the elastic database query feature is included into the cost of your Azure SQL Database.
For pricing information see SQL Database Pricing.

Next steps
For an overview of elastic query, see Elastic query overview.
For syntax and sample queries for vertically partitioned data, see Querying vertically partitioned data)
For a horizontal partitioning (sharding) tutorial, see Getting started with elastic query for horizontal
partitioning (sharding).
For syntax and sample queries for horizontally partitioned data, see Querying horizontally partitioned data)
See sp_execute _remote for a stored procedure that executes a Transact-SQL statement on a single remote
Azure SQL Database or set of databases serving as shards in a horizontal partitioning scheme.
SQL Database Advisor using the Azure portal
2/16/2017 • 4 min to read • Edit Online

You can use the Azure SQL Database Advisor in the Azure portal to review and implement recommendations for
your existing SQL Databases that can improve current query performance.

Viewing recommendations
The recommendations page is where you view the top recommendations based on their potential impact to
improve performance. You can also view the status of the historical operations. Select a recommendation or status
to see more details.
To view and apply recommendations, you need the correct role-based access control permissions in Azure. Reader,
SQL DB Contributor permissions are required to view recommendations, and Owner, SQL DB Contributor
permissions are required to execute any actions; create or drop indexes and cancel index creation.
1. Sign in to the Azure portal.
2. Click More services > SQL databases, and select your database.
3. Click Performance recommendation to view available recommendations for the selected database.

NOTE
To get recommendations a database needs to have about a day of usage, and there needs to be some activity. There also
needs to be some consistent activity. The SQL Database Advisor can more easily optimize for consistent query patterns than
it can for random spotty bursts of activity. If recommendations are not available, the Performance recommendation page
should provide a message explaining why.

Here is an example of "Fix schema issue" recommendation in the Azure portal.

Recommendations are sorted by their potential impact on performance into the following four categories:

IMPACT DESCRIPTION

High High impact recommendations should provide the most

significant performance impact.

Medium Medium impact recommendations should improve

performance, but not substantially.

Low Low impact recommendations should provide better

performance than without, but improvements might not be
significant.

Removing recommendations from the list

If your list of recommendations contains items that you want to remove from the list, you can discard the
recommendation:
1. Select a recommendation in the list of Recommendations.
2. Click Discard on the Details blade.
If desired, you can add discarded items back to the Recommendations list:
1. On the Recommendations blade click View discarded.
2. Select a discarded item from the list to view its details.
3. Optionally, click Undo Discard to add the index back to the main list of Recommendations.

Applying recommendations
SQL Database Advisor gives you full control over how recommendations are enabled using any of the following
three options:
Apply individual recommendations one at a time.
Enable the advisor to automatically apply recommendations (currently applies to index recommendations only).
To implement a recommendation manually, run the recommended T-SQL script against your database .
Select any recommendation to view its details and then click View script to review the exact details of how the
recommendation is created.
The database remains online while the advisor applies the recommendation -- using SQL Database Advisor never
takes a database offline.
Apply an individual recommendation
You can review and accept recommendations one at a time.
1. On the Recommendations blade, click a recommendation.
2. On the Details blade click Apply.

Enable automatic index management

You can set the SQL Database Advisor to implement recommendations automatically. As recommendations
become available they will automatically be applied. As with all index operations managed by the service if the
performance impact is negative the recommendation will be reverted.
1. On the Recommendations blade, click Automate:

2. Set the advisor to automatically Create or Drop indexes:

Manually run the recommended T -SQL script
Select any recommendation and then click View script. Run this script against your database to manually apply
the recommendation.
Indexes that are manually executed are not monitored and validated for performance impact by the service so it is
suggested that you monitor these indexes after creation to verify they provide performance gains and adjust or
delete them if necessary. For details about creating indexes, see CREATE INDEX (Transact-SQL).
Canceling recommendations
Recommendations that are in a Pending, Verifying, or Success status can be canceled. Recommendations with a
status of Executing cannot be canceled.
1. Select a recommendation in the Tuning History area to open the recommendations details blade.
2. Click Cancel to abort the process of applying the recommendation.

Monitoring operations
Applying a recommendation might not happen instantaneously. The portal provides details regarding the status of
recommendation operations. The following are possible states that an index can be in:

STATUS DESCRIPTION

Pending Apply recommendation command has been received and is

scheduled for execution.

Executing The recommendation is being applied.

Success Recommendation was successfully applied.

Error An error occurred during the process of applying the

recommendation. This can be a transient issue, or possibly a
schema change to the table and the script is no longer valid.

Reverting The recommendation was applied, but has been deemed non-
performant and is being automatically reverted.

Reverted The recommendation was reverted.

Click an in-process recommendation from the list to see more details:

Reverting a recommendation
If you used the advisor to apply the recommendation (meaning you did not manually run the T-SQL script) it will
automatically revert it if it finds the performance impact to be negative. If for any reason you simply want to revert
a recommendation you can do the following:
1. Select a successfully applied recommendation in the Tuning history area.
2. Click Revert on the recommendation details blade.

Monitoring performance impact of index recommendations

After recommendations are successfully implemented (currently, index operations and parameterize queries
recommendations only) you can click Query Insights on the recommendation details blade to open Query
Performance Insights and see the performance impact of your top queries.

Summary
SQL Database Advisor provides recommendations for improving SQL database performance. By providing T-SQL
scripts, as well as individual and fully-automatic (currently index only), the advisor provides helpful assistance in
optimizing your database and ultimately improving query performance.

Next steps
Monitor your recommendations and continue to apply them to refine performance. Database workloads are
dynamic and change continuously. SQL Database advisor will continue to monitor and provide recommendations
that can potentially improve your database's performance.
See SQL Database Advisor for an overview of SQL Database Advisor.
See Query Performance Insights to learn about viewing the performance impact of your top queries.

Additional resources
Query Store
CREATE INDEX
Role-based access control
SQL Database Performance Insight
4/19/2017 • 1 min to read • Edit Online

Azure SQL Database provides performance tools to help you identify and improve the performance of your
databases by providing intelligent tuning actions and recommendations.
1. Browse to your database in the Azure Portal and click All settings > Performance ** > **Overview to open
the Performance page.
2. Click Recommendations to open the SQL Database Advisor, and click Queries to open Query
Performance Insight.

Performance Overview
Clicking on Overview or on the Performance tile will take you to the performance dashboard for your database.
This view provides a summary of your database performance, and helps you with performance tuning and
troubleshooting.
The Recommendations tile provides a breakdown of tuning recommendations for your database (top 3
recommendations are shown if there are more). Clicking this tile takes you to SQL Database Advisor.
The Tuning activity tile provides a summary of the ongoing and completed tuning actions for your database,
giving you a quick view into the history of tuning activity. Clicking this tile takes you to the full tuning history
view for your database.
The Auto-tuning tile shows the auto-tuning configuration for your database (which tuning actions are
configured to be automatically applied to your database). Clicking this tile opens the automation configuration
dialog.
The Database queries tile shows the summary of the query performance for your database (overall DTU usage
and top resource consuming queries). Clicking this tile takes you to Query Performance Insight.

SQL Database Advisor

SQL Database Advisor provides intelligent tuning recommendations that can help improve your database's
performance.
Recommendations on which indexes to create or drop (and an option to apply index recommendations
automatically without any user interaction and automatically rolling back recommendations that have a
negative impact on performance).
Recommendations when schema issues are identified in the database.
Recommendations when queries can benefit from parameterized queries.
Query Performance Insight
Query Performance Insight allows you to spend less time troubleshooting database performance by providing:
Deeper insight into your databases resource (DTU) consumption.
The top CPU consuming queries, which can potentially be tuned for improved performance.
The ability to drill down into the details of a query.

Additional resources
Azure SQL Database performance guidance for single databases
When should an elastic pool be used?
SQL Database performance tuning tips
4/19/2017 • 1 min to read • Edit Online

You can change the service tier of a standalone database or increase the eDTUs of an elastic pool at any time to
improve performance, but you may want to identify opportunities to improve and optimize query performance
first. Missing indexes and poorly optimized queries are common reasons for poor database performance. This
article provides guidance for performance tuning in SQL Database.
If your Azure issue is not addressed in this article, visit the Azure forums on MSDN and the Stack Overflow. You
can post your issue on these forums or to @AzureSupport on Twitter. Also, you can file an Azure support request
by selecting Get support on the Azure support site.

Steps to evaluate and tune database performance

1. In the Azure Portal, click SQL databases, select the database, and then use the Monitoring chart to look for
resources approaching their maximum. DTU consumption is shown by default. Click Edit to change the time
range and values shown.
2. Use Query Performance Insight to evaluate the queries using DTUs, and then use SQL Database Advisor to view
recommendations for creating and dropping indexes, parameterizing queries, and fixing schema issues.
3. You can use dynamic management views (DMVs), Extended Events (Xevents), and the Query Store in SSMS to
get performance parameters in real time. See the performance guidance topic for detailed monitoring and
tuning tips.

IMPORTANT
It is recommended that you always use the latest version of Management Studio to remain synchronized with updates to
Microsoft Azure and SQL Database. Update SQL Server Management Studio.

Steps to improve database performance with more resources

1. For standalone databases, you can change service tiers on-demand to improve database performance.
2. For multiple databases, consider using elastic pools to scale resources automatically.
Use Azure portal to create alerts for Azure SQL
Database
4/10/2017 • 4 min to read • Edit Online

Overview
This article shows you how to set up Azure SQL Database alerts using the Azure portal. This article also provides
best practices for values and thresholds.
You can receive an alert based on monitoring metrics for, or events on, your Azure services.
Metric values - The alert triggers when the value of a specified metric crosses a threshold you assign in either
direction. That is, it triggers both when the condition is first met and then afterwards when that condition is no
longer being met.
Activity log events - An alert can trigger on every event, or, only when a certain number of events occur.
You can configure an alert to do the following when it triggers:
send email notifications to the service administrator and co-administrators
send email to additional emails that you specify.
call a webhook
start execution of an Azure runbook (only from the Azure portal)
You can configure and get information about alert rules using
Azure portal
PowerShell
command-line interface (CLI)
Azure Monitor REST API

Create an alert rule on a metric with the Azure portal

1. In the portal, locate the resource you are interested in monitoring and select it.
2. Select Alerts or Alert rules under the MONITORING section. The text and icon may vary slightly for
different resources.

3. Select the Add alert command and fill in the fields.

4. Name your alert rule, and choose a Description, which also shows in notification emails.
5. Select the Metric you want to monitor, then choose a Condition and Threshold value for the metric. Also
chose the Period of time that the metric rule must be satisfied before the alert triggers. So for example, if you
use the period "PT5M" and your alert looks for CPU above 80%, the alert triggers when the CPU has been
consistently above 80% for 5 minutes. Once the first trigger occurs, it again triggers when the CPU stays below
80% for 5 minutes. The CPU measurement occurs every 1 minute.
6. Check Email owners... if you want administrators and co-administrators to be emailed when the alert fires.
7. If you want additional emails to receive a notification when the alert fires, add them in the Additional
Administrator email(s) field. Separate multiple emails with semi-colons -
email@contoso.com;email2@contoso.com
8. Put in a valid URI in the Webhook field if you want it called when the alert fires.
9. If you use Azure Automation, you can select a Runbook to be run when the alert fires.
10. Select OK when done to create the alert.
Within a few minutes, the alert is active and triggers as previously described.

Managing your alerts

Once you have created an alert, you can select it and:
View a graph showing the metric threshold and the actual values from the previous day.
Edit or delete it.
Disable or Enable it if you want to temporarily stop or resume receiving notifications for that alert.

SQL Database alert values and thresholds

MINIMUM ALERT TIME
RESOURCE TYPE METRIC NAME FRIENDLY NAME AGGREGATION TYPE WINDOW

SQL database cpu_percent CPU percentage Average 5 minutes

SQL database physical_data_read_pe Data IO percentage Average 5 minutes

rcent

SQL database log_write_percent Log IO percentage Average 5 minutes

SQL database dtu_consumption_per DTU percentage Average 5 minutes

cent

SQL database storage Total database size Maximum 30 minutes

SQL database connection_successful Successful Total 10 minutes

Connections

SQL database connection_failed Failed Connections Total 10 minutes

SQL database blocked_by_firewall Blocked by Firewall Total 10 minutes

SQL database deadlock Deadlocks Total 10 minutes

SQL database storage_percent Database size Maximum 30 minutes

percentage

SQL database xtp_storage_percent In-Memory OLTP Average 5 minutes

storage
percent(Preview)

SQL database workers_percent Workers percentage Average 5 minutes

SQL database sessions_percent Sessions percent Average 5 minutes

SQL database dtu_limit DTU limit Average 5 minutes

SQL database dtu_used DTU used Average 5 minutes

MINIMUM ALERT TIME
RESOURCE TYPE METRIC NAME FRIENDLY NAME AGGREGATION TYPE WINDOW

SQL data warehouse cpu_percent CPU percentage Average 10 minutes

SQL data warehouse physical_data_read_pe Data IO percentage Average 10 minutes

rcent

SQL data warehouse storage Total database size Maximum 10 minutes

SQL data warehouse connection_successful Successful Total 10 minutes

Connections

SQL data warehouse connection_failed Failed Connections Total 10 minutes

SQL data warehouse blocked_by_firewall Blocked by Firewall Total 10 minutes

SQL data warehouse service_level_objective Service level objective Total 10 minutes

of the database

SQL data warehouse dwu_limit dwu limit Maximum 10 minutes

SQL data warehouse dwu_consumption_pe DWU percentage Average 10 minutes

rcent

SQL data warehouse dwu_used DWU used Average 10 minutes

Elastic pool cpu_percent CPU percentage Average 5 minutes

Elastic pool physical_data_read_pe Data IO percentage Average 5 minutes

rcent

Elastic pool log_write_percent Log IO percentage Average 5 minutes

Elastic pool dtu_consumption_per DTU percentage Average 5 minutes

cent

Elastic pool storage_percent Storage percentage Average 5 minutes

Elastic pool workers_percent Workers percentage Average 5 minutes

Elastic pool eDTU_limit eDTU limit Average 5 minutes

Elastic pool storage_limit Storage limit Average 5 minutes

Elastic pool eDTU_used eDTU used Average 5 minutes

Elastic pool storage_used Storage used Average 5 minutes

Next steps
Get an overview of Azure monitoring including the types of information you can collect and monitor.
Learn more about configuring webhooks in alerts.
Learn more about Azure Automation Runbooks.
Get an overview of diagnostic logs and collect detailed high-frequency metrics on your service.
Get an overview of metrics collection to make sure your service is available and responsive.
Monitor In-Memory OLTP Storage
2/16/2017 • 1 min to read • Edit Online

When using In-Memory OLTP, data in memory-optimized tables and table variables resides in In-Memory OLTP
storage. Each Premium service tier has a maximum In-Memory OLTP storage size, which is documented in the
SQL Database Service Tiers article. Once this limit is exceeded, insert and update operations may start failing (with
error 41823). At that point you will need to either delete data to reclaim memory, or upgrade the performance tier
of your database.

Determine whether data will fit within the in-memory storage cap
Determine the storage cap: consult the SQL Database Service Tiers article for the storage caps of the different
Premium service tiers.
Estimating memory requirements for a memory-optimized table works the same way for SQL Server as it does in
Azure SQL Database. Take a few minutes to review that topic on MSDN.
Note that the table and table variable rows, as well as indexes, count toward the max user data size. In addition,
ALTER TABLE needs enough room to create a new version of the entire table and its indexes.

Monitoring and alerting

You can monitor in-memory storage use as a percentage of the storage cap for your performance tier in the Azure
portal:
On the Database blade, locate the Resource utilization box and click on Edit.
Then select the metric In-Memory OLTP Storage percentage .
To add an alert, click on the Resource Utilization box to open the Metric blade, then click on Add alert.
Or use the following query to show the in-memory storage utilization:

SELECT xtp_storage_percent FROM sys.dm_db_resource_stats

Correct out-of-memory situations - Error 41823

Running out-of-memory results in INSERT, UPDATE, and CREATE operations failing with error message 41823.
Error message 41823 indicates that the memory-optimized tables and table variables have exceeded the
maximum size.
To resolve this error, either:
Delete data from the memory-optimized tables, potentially offloading the data to traditional, disk-based tables;
or,
Upgrade the service tier to one with enough in-memory storage for the data you need to keep in memory-
optimized tables.

Next steps
For monitoring guidance, see Monitoring Azure SQL Database using dynamic management views.
Event File target code for extended events in SQL
Database
4/14/2017 • 9 min to read • Edit Online

You want a complete code sample for a robust way to capture and report information for an extended event.
In Microsoft SQL Server, the Event File target is used to store event outputs into a local hard drive file. But such
files are not available to Azure SQL Database. Instead we use the Azure Storage service to support the Event File
target.
This topic presents a two-phase code sample:
PowerShell, to create an Azure Storage container in the cloud.
Transact-SQL:
To assign the Azure Storage container to an Event File target.
To create and start the event session, and so on.

Prerequisites
An Azure account and subscription. You can sign up for a free trial.
Any database you can create a table in.
Optionally you can create an AdventureWorksLT demonstration database in minutes.
SQL Server Management Studio (ssms.exe), ideally its latest monthly update version. You can download the
latest ssms.exe from:
Topic titled Download SQL Server Management Studio.
A direct link to the download.
You must have the Azure PowerShell modules installed.
The modules provide commands such as - New-AzureStorageAccount.

Phase 1: PowerShell code for Azure Storage container

This PowerShell is phase 1 of the two-phase code sample.
The script starts with commands to clean up after a possible previous run, and is rerunnable.
1. Paste the PowerShell script into a simple text editor such as Notepad.exe, and save the script as a file with the
extension .ps1.
2. Start PowerShell ISE as an Administrator.
3. At the prompt, type
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser
and then press Enter.
4. In PowerShell ISE, open your .ps1 file. Run the script.
5. The script first starts a new window in which you log in to Azure.
If you rerun the script without disrupting your session, you have the convenient option of commenting
out the Add-AzureAccount command.
PowerShell code

## TODO: Before running, find all 'TODO' and make each edit!!

#--------------- 1 -----------------------

# You can comment out or skip this Add-AzureAccount

# command after the first run.
# Current PowerShell environment retains the successful outcome.

'Expect a pop-up window in which you log in to Azure.'

Add-AzureAccount

#-------------- 2 ------------------------

'
TODO: Edit the values assigned to these variables, especially the first few!
'

# Ensure the current date is between

# the Expiry and Start time values that you edit here.

$subscriptionName = 'YOUR_SUBSCRIPTION_NAME'
$policySasExpiryTime = '2016-01-28T23:44:56Z'
$policySasStartTime = '2015-08-01'

$storageAccountName = 'gmstorageaccountxevent'
$storageAccountLocation = 'West US'
$contextName = 'gmcontext'
$containerName = 'gmcontainerxevent'
$policySasToken = 'gmpolicysastoken'

# Leave this value alone, as 'rwl'.

$policySasPermission = 'rwl'

#--------------- 3 -----------------------

# The ending display lists your Azure subscriptions.

# The ending display lists your Azure subscriptions.
# One should match the $subscriptionName value you assigned
# earlier in this PowerShell script.

'Choose an existing subscription for the current PowerShell environment.'

Select-AzureSubscription -SubscriptionName $subscriptionName

#-------------- 4 ------------------------

'
Clean up the old Azure Storage Account after any previous run,
before continuing this new run.'

If ($storageAccountName)
{
Remove-AzureStorageAccount -StorageAccountName $storageAccountName
}

#--------------- 5 -----------------------

[System.DateTime]::Now.ToString()

'
Create a storage account.
This might take several minutes, will beep when ready.
...PLEASE WAIT...'

New-AzureStorageAccount `
-StorageAccountName $storageAccountName `
-Location $storageAccountLocation

[System.DateTime]::Now.ToString()

[System.Media.SystemSounds]::Beep.Play()

'
Get the primary access key for your storage account.
'

$primaryAccessKey_ForStorageAccount = `
(Get-AzureStorageKey `
-StorageAccountName $storageAccountName).Primary

"`$primaryAccessKey_ForStorageAccount = $primaryAccessKey_ForStorageAccount"

'Azure Storage Account cmdlet completed.

Remainder of PowerShell .ps1 script continues.
'

#--------------- 6 -----------------------

# The context will be needed to create a container within the storage account.

'Create a context object from the storage account and its primary access key.
'

$context = New-AzureStorageContext `
-StorageAccountName $storageAccountName `
-StorageAccountKey $primaryAccessKey_ForStorageAccount

'Create a container within the storage account.

'Create a container within the storage account.
'

$containerObjectInStorageAccount = New-AzureStorageContainer `
-Name $containerName `
-Context $context

'Create a security policy to be applied to the SAS token.

New-AzureStorageContainerStoredAccessPolicy `
-Container $containerName `
-Context $context `
-Policy $policySasToken `
-Permission $policySasPermission `
-ExpiryTime $policySasExpiryTime `
-StartTime $policySasStartTime

'
Generate a SAS token for the container.
'
Try
{
$sasTokenWithPolicy = New-AzureStorageContainerSASToken `
-Name $containerName `
-Context $context `
-Policy $policySasToken
}
Catch
{
$Error[0].Exception.ToString()
}

#-------------- 7 ------------------------

'Display the values that YOU must edit into the Transact-SQL script next!:
'

"storageAccountName: $storageAccountName"
"containerName: $containerName"
"sasTokenWithPolicy: $sasTokenWithPolicy"

'
REMINDER: sasTokenWithPolicy here might start with "?" character, which you must exclude from Transact-SQL.
'

'
(Later, return here to delete your Azure Storage account. See the preceding - Remove-AzureStorageAccount -StorageAccountName
$storageAccountName)'

'
Now shift to the Transact-SQL portion of the two-part code sample!'

# EOFile

Take note of the few named values that the PowerShell script prints when it ends. You must edit those values into
the Transact-SQL script that follows as phase 2.

Phase 2: Transact-SQL code that uses Azure Storage container

In phase 1 of this code sample, you ran a PowerShell script to create an Azure Storage container.
Next in phase 2, the following Transact-SQL script must use the container.
The script starts with commands to clean up after a possible previous run, and is rerunnable.
The PowerShell script printed a few named values when it ended. You must edit the Transact-SQL script to use
those values. Find TODO in the Transact-SQL script to locate the edit points.
1. Open SQL Server Management Studio (ssms.exe).
2. Connect to your Azure SQL Database database.
3. Click to open a new query pane.
4. Paste the following Transact-SQL script into the query pane.
5. Find every TODO in the script and make the appropriate edits.
6. Save, and then run the script.

WARNING
The SAS key value generated by the preceding PowerShell script might begin with a '?' (question mark). When you use the
SAS key in the following T-SQL script, you must remove the leading '?'. Otherwise your efforts might be blocked by security.

Transact-SQL code

---- TODO: First, run the PowerShell portion of this two-part code sample.
---- TODO: Second, find every 'TODO' in this Transact-SQL file, and edit each.

---- Transact-SQL code for Event File target on Azure SQL Database.

SET NOCOUNT ON;

---- Step 1. Establish one little table, and ---------

---- insert one row of data.

IF EXISTS
(SELECT * FROM sys.objects
WHERE type = 'U' and name = 'gmTabEmployee')
BEGIN
DROP TABLE gmTabEmployee;
END
GO

CREATE TABLE gmTabEmployee

(
EmployeeGuid uniqueIdentifier not null default newid() primary key,
EmployeeId int not null identity(1,1),
EmployeeKudosCount int not null default 0,
EmployeeDescr nvarchar(256) null
);
GO

INSERT INTO gmTabEmployee ( EmployeeDescr )

VALUES ( 'Jane Doe' );
GO

------ Step 2. Create key, and ------------

------ Create credential (your Azure Storage container must already exist).

IF NOT EXISTS
(SELECT * FROM sys.symmetric_keys
WHERE symmetric_key_id = 101)
BEGIN
CREATE MASTER KEY ENCRYPTION BY PASSWORD = '0C34C960-6621-4682-A123-C7EA08E3FC46' -- Or any newid().
END
GO

IF EXISTS
(SELECT * FROM sys.database_scoped_credentials
-- TODO: Assign AzureStorageAccount name, and the associated Container name.
WHERE name = 'https://gmstorageaccountxevent.blob.core.windows.net/gmcontainerxevent')
BEGIN
DROP DATABASE SCOPED CREDENTIAL
-- TODO: Assign AzureStorageAccount name, and the associated Container name.
[https://gmstorageaccountxevent.blob.core.windows.net/gmcontainerxevent] ;
END
GO

CREATE
DATABASE SCOPED
CREDENTIAL
-- use '.blob.', and not '.queue.' or '.table.' etc.
-- TODO: Assign AzureStorageAccount name, and the associated Container name.
[https://gmstorageaccountxevent.blob.core.windows.net/gmcontainerxevent]
WITH
IDENTITY = 'SHARED ACCESS SIGNATURE', -- "SAS" token.
-- TODO: Paste in the long SasToken string here for Secret, but exclude any leading '?'.
SECRET = 'sv=2014-02-14&sr=c&si=gmpolicysastoken&sig=EjAqjo6Nu5xMLEZEkMkLbeF7TD9v1J8DNB2t8gOKTts%3D'
;
GO

------ Step 3. Create (define) an event session. --------

------ The event session has an event with an action,
------ and a has a target.

IF EXISTS
(SELECT * from sys.database_event_sessions
WHERE name = 'gmeventsessionname240b')
BEGIN
DROP
EVENT SESSION
gmeventsessionname240b
ON DATABASE;
END
GO

CREATE
EVENT SESSION
gmeventsessionname240b
ON DATABASE

ADD EVENT
sqlserver.sql_statement_starting
(
ACTION (sqlserver.sql_text)
WHERE statement LIKE 'UPDATE gmTabEmployee%'
)
ADD TARGET
package0.event_file
(
-- TODO: Assign AzureStorageAccount name, and the associated Container name.
-- Also, tweak the .xel file name at end, if you like.
SET filename =
'https://gmstorageaccountxevent.blob.core.windows.net/gmcontainerxevent/anyfilenamexel242b.xel'
)
WITH
WITH
(MAX_MEMORY = 10 MB,
MAX_DISPATCH_LATENCY = 3 SECONDS)
;
GO

------ Step 4. Start the event session. ----------------

------ Issue the SQL Update statements that will be traced.
------ Then stop the session.

------ Note: If the target fails to attach,

------ the session must be stopped and restarted.

ALTER
EVENT SESSION
gmeventsessionname240b
ON DATABASE
STATE = START;
GO

SELECT 'BEFORE_Updates', EmployeeKudosCount, * FROM gmTabEmployee;

UPDATE gmTabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 2
WHERE EmployeeDescr = 'Jane Doe';

UPDATE gmTabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 13
WHERE EmployeeDescr = 'Jane Doe';

SELECT 'AFTER__Updates', EmployeeKudosCount, * FROM gmTabEmployee;

ALTER
EVENT SESSION
gmeventsessionname240b
ON DATABASE
STATE = STOP;
GO

-------------- Step 5. Select the results. ----------

SELECT
*, 'CLICK_NEXT_CELL_TO_BROWSE_ITS_RESULTS!' as [CLICK_NEXT_CELL_TO_BROWSE_ITS_RESULTS],
CAST(event_data AS XML) AS [event_data_XML] -- TODO: In ssms.exe results grid, double-click this cell!
FROM
sys.fn_xe_file_target_read_file
(
-- TODO: Fill in Storage Account name, and the associated Container name.
'https://gmstorageaccountxevent.blob.core.windows.net/gmcontainerxevent/anyfilenamexel242b',
null, null, null
);
GO

-------------- Step 6. Clean up. ----------

DROP
EVENT SESSION
gmeventsessionname240b
ON DATABASE;
GO

DROP DATABASE SCOPED CREDENTIAL

-- TODO: Assign AzureStorageAccount name, and the associated Container name.
[https://gmstorageaccountxevent.blob.core.windows.net/gmcontainerxevent]
[https://gmstorageaccountxevent.blob.core.windows.net/gmcontainerxevent]
;
GO

DROP TABLE gmTabEmployee;

PRINT 'Use PowerShell Remove-AzureStorageAccount to delete your Azure Storage account!';

If the target fails to attach when you run, you must stop and restart the event session:

ALTER EVENT SESSION ... STATE = STOP;

GO
ALTER EVENT SESSION ... STATE = START;
GO

Output
When the Transact-SQL script completes, click a cell under the event_data_XML column header. One element is
displayed which shows one UPDATE statement.
Here is one element that was generated during testing:

<event name="sql_statement_starting" package="sqlserver" timestamp="2015-09-22T19:18:45.420Z">

<data name="state">
<value>0</value>
<text>Normal</text>
</data>
<data name="line_number">
<value>5</value>
</data>
<data name="offset">
<value>148</value>
</data>
<data name="offset_end">
<value>368</value>
</data>
<data name="statement">
<value>UPDATE gmTabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 2
WHERE EmployeeDescr = 'Jane Doe'</value>
</data>
<action name="sql_text" package="sqlserver">
<value>

SELECT 'BEFORE_Updates', EmployeeKudosCount, * FROM gmTabEmployee;

UPDATE gmTabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 2
WHERE EmployeeDescr = 'Jane Doe';

UPDATE gmTabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 13
WHERE EmployeeDescr = 'Jane Doe';

SELECT 'AFTER__Updates', EmployeeKudosCount, * FROM gmTabEmployee;

</value>
</action>
</event>

The preceding Transact-SQL script used the following system function to read the event_file:
sys.fn_xe_file_target_read_file (Transact-SQL)
An explanation of advanced options for the viewing of data from extended events is available at:
Advanced Viewing of Target Data from Extended Events

Converting the code sample to run on SQL Server

Suppose you wanted to run the preceding Transact-SQL sample on Microsoft SQL Server.
For simplicity, you would want to completely replace use of the Azure Storage container with a simple file such
as C:\myeventdata.xel. The file would be written to the local hard drive of the computer that hosts SQL
Server.
You would not need any kind of Transact-SQL statements for CREATE MASTER KEY and CREATE
CREDENTIAL.
In the CREATE EVENT SESSION statement, in its ADD TARGET clause, you would replace the Http value
assigned made to filename= with a full path string like C:\myfile.xel.
No Azure Storage account need be involved.

More information
For more info about accounts and containers in the Azure Storage service, see:
How to use Blob storage from .NET
Naming and Referencing Containers, Blobs, and Metadata
Working with the Root Container
Lesson 1: Create a stored access policy and a shared access signature on an Azure container
Lesson 2: Create a SQL Server credential using a shared access signature
Ring Buffer target code for extended events in SQL
Database
4/14/2017 • 5 min to read • Edit Online

You want a complete code sample for the easiest quick way to capture and report information for an extended
event during a test. The easiest target for extended event data is the Ring Buffer target.
This topic presents a Transact-SQL code sample that:
1. Creates a table with data to demonstrate with.
2. Creates a session for an existing extended event, namely sqlserver.sql_statement_starting.
The event is limited to SQL statements that contain a particular Update string: statement LIKE
'%UPDATE tabEmployee%'.
Chooses to send the output of the event to a target of type Ring Buffer, namely package0.ring_buffer.
3. Starts the event session.
4. Issues a couple of simple SQL UPDATE statements.
5. Issues an SQL SELECT to retrieve event output from the Ring Buffer.
sys.dm_xe_database_session_targets and other dynamic management views (DMVs) are joined.
6. Stops the event session.
7. Drops the Ring Buffer target, to release its resources.
8. Drops the event session and the demo table.

Code sample
With very minor modification, the following Ring Buffer code sample can be run on either Azure SQL Database or
Microsoft SQL Server. The difference is the presence of the node '_database' in the name of some dynamic
management views (DMVs), used in the FROM clause in Step 5. For example:
sys.dm_xe_database_session_targets
sys.dm_xe_session_targets

GO
---- Transact-SQL.
---- Step set 1.
SET NOCOUNT ON;
GO

IF EXISTS
(SELECT * FROM sys.objects
WHERE type = 'U' and name = 'tabEmployee')
BEGIN
DROP TABLE tabEmployee;
END
GO

CREATE TABLE tabEmployee

(
EmployeeGuid uniqueIdentifier not null default newid() primary key,
EmployeeId int not null identity(1,1),
EmployeeKudosCount int not null default 0,
EmployeeDescr nvarchar(256) null
);
GO

INSERT INTO tabEmployee ( EmployeeDescr )

VALUES ( 'Jane Doe' );
GO

---- Step set 2.

IF EXISTS
(SELECT * from sys.database_event_sessions
WHERE name = 'eventsession_gm_azuresqldb51')
BEGIN
DROP EVENT SESSION eventsession_gm_azuresqldb51
ON DATABASE;
END
GO

CREATE
EVENT SESSION eventsession_gm_azuresqldb51
ON DATABASE
ADD EVENT
sqlserver.sql_statement_starting
(
ACTION (sqlserver.sql_text)
WHERE statement LIKE '%UPDATE tabEmployee%'
)
ADD TARGET
package0.ring_buffer
(SET
max_memory = 500 -- Units of KB.
);
GO

---- Step set 3.

ALTER EVENT SESSION eventsession_gm_azuresqldb51

ON DATABASE
STATE = START;
GO

---- Step set 4.

SELECT 'BEFORE_Updates', EmployeeKudosCount, * FROM tabEmployee;

UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 102;

UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 1015;

SELECT 'AFTER__Updates', EmployeeKudosCount, * FROM tabEmployee;

---- Step set 5.

SELECT
se.name AS [session-name],
ev.event_name,
ac.action_name,
st.target_name,
se.session_source,
st.target_data,
CAST(st.target_data AS XML) AS [target_data_XML]
FROM
sys.dm_xe_database_session_event_actions AS ac

INNER JOIN sys.dm_xe_database_session_events AS ev ON ev.event_name = ac.event_name

AND CAST(ev.event_session_address AS BINARY(8)) = CAST(ac.event_session_address AS BINARY(8))

INNER JOIN sys.dm_xe_database_session_object_columns AS oc

ON CAST(oc.event_session_address AS BINARY(8)) = CAST(ac.event_session_address AS BINARY(8))

INNER JOIN sys.dm_xe_database_session_targets AS st

ON CAST(st.event_session_address AS BINARY(8)) = CAST(ac.event_session_address AS BINARY(8))

INNER JOIN sys.dm_xe_database_sessions AS se

ON CAST(ac.event_session_address AS BINARY(8)) = CAST(se.address AS BINARY(8))
WHERE
oc.column_name = 'occurrence_number'
AND
se.name = 'eventsession_gm_azuresqldb51'
AND
ac.action_name = 'sql_text'
ORDER BY
se.name,
ev.event_name,
ac.action_name,
st.target_name,
se.session_source
;
GO

---- Step set 6.

ALTER EVENT SESSION eventsession_gm_azuresqldb51

ON DATABASE
STATE = STOP;
GO

---- Step set 7.

ALTER EVENT SESSION eventsession_gm_azuresqldb51

ON DATABASE
DROP TARGET package0.ring_buffer;
GO

---- Step set 8.

DROP EVENT SESSION eventsession_gm_azuresqldb51

DROP EVENT SESSION eventsession_gm_azuresqldb51
ON DATABASE;
GO

DROP TABLE tabEmployee;

Ring Buffer contents

We used ssms.exe to run the code sample.
To view the results, we clicked the cell under the column header target_data_XML.
Then in the results pane we clicked the cell under the column header target_data_XML. This click created another
file tab in ssms.exe in which the content of the result cell was displayed, as XML.
The output is shown in the following block. It looks long, but it is just two elements.

<RingBufferTarget truncated="0" processingTime="0" totalEventsProcessed="2" eventCount="2" droppedCount="0" memoryUsed="1728">

<event name="sql_statement_starting" package="sqlserver" timestamp="2015-09-22T15:29:31.317Z">
<data name="state">
<type name="statement_starting_state" package="sqlserver" />
<value>0</value>
<text>Normal</text>
</data>
<data name="line_number">
<type name="int32" package="package0" />
<value>7</value>
</data>
<data name="offset">
<type name="int32" package="package0" />
<value>184</value>
</data>
<data name="offset_end">
<type name="int32" package="package0" />
<value>328</value>
</data>
<data name="statement">
<type name="unicode_string" package="package0" />
<value>UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 102</value>
</data>
<action name="sql_text" package="sqlserver">
<type name="unicode_string" package="package0" />
<value>
---- Step set 4.

SELECT 'BEFORE_Updates', EmployeeKudosCount, * FROM tabEmployee;

UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 102;

UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 1015;

SELECT 'AFTER__Updates', EmployeeKudosCount, * FROM tabEmployee;

</value>
</action>
</event>
<event name="sql_statement_starting" package="sqlserver" timestamp="2015-09-22T15:29:31.327Z">
<data name="state">
<type name="statement_starting_state" package="sqlserver" />
<value>0</value>
<text>Normal</text>
</data>
<data name="line_number">
<type name="int32" package="package0" />
<value>10</value>
</data>
<data name="offset">
<type name="int32" package="package0" />
<value>340</value>
</data>
<data name="offset_end">
<type name="int32" package="package0" />
<value>486</value>
</data>
<data name="statement">
<type name="unicode_string" package="package0" />
<value>UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 1015</value>
</data>
<action name="sql_text" package="sqlserver">
<type name="unicode_string" package="package0" />
<value>
---- Step set 4.

SELECT 'BEFORE_Updates', EmployeeKudosCount, * FROM tabEmployee;

UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 102;

UPDATE tabEmployee
SET EmployeeKudosCount = EmployeeKudosCount + 1015;

SELECT 'AFTER__Updates', EmployeeKudosCount, * FROM tabEmployee;

</value>
</action>
</event>
</RingBufferTarget>

Release resources held by your Ring Buffer

When you are done with your Ring Buffer, you can remove it and release its resources issuing an ALTER like the
following:

ALTER EVENT SESSION eventsession_gm_azuresqldb51

ON DATABASE
DROP TARGET package0.ring_buffer;
GO

The definition of your event session is updated, but not dropped. Later you can add another instance of the Ring
Buffer to your event session:

ALTER EVENT SESSION eventsession_gm_azuresqldb51

ON DATABASE
ADD TARGET
package0.ring_buffer
(SET
max_memory = 500 -- Units of KB.
);

More information
The primary topic for extended events on Azure SQL Database is:
Extended event considerations in SQL Database, which contrasts some aspects of extended events that differ
between Azure SQL Database versus Microsoft SQL Server.
Other code sample topics for extended events are available at the following links. However, you must routinely
check any sample to see whether the sample targets Microsoft SQL Server versus Azure SQL Database. Then you
can decide whether minor changes are needed to run the sample.
Code sample for Azure SQL Database: Event File target code for extended events in SQL Database
Managing Azure SQL Databases using Azure
Automation
4/14/2017 • 1 min to read • Edit Online

This guide will introduce you to the Azure Automation service, and how it can be used to simplify management of
your Azure SQL databases.

What is Azure Automation?

Azure Automation is an Azure service for simplifying cloud management through process automation. Using Azure
Automation, long-running, manual, error-prone, and frequently repeated tasks can be automated to increase
reliability, efficiency, and time to value for your organization.
Azure Automation provides a highly-reliable and highly-available workflow execution engine that scales to meet
your needs as your organization grows. In Azure Automation, processes can be kicked off manually, by 3rd-party
systems, or at scheduled intervals so that tasks happen exactly when needed.
Lower operational overhead and free up IT / DevOps staff to focus on work that adds business value by moving
your cloud management tasks to be run automatically by Azure Automation.

How can Azure Automation help manage Azure SQL databases?

Azure SQL Database can be managed in Azure Automation by using the Azure SQL Database PowerShell cmdlets
that are available in the Azure PowerShell tools. Azure Automation has these Azure SQL Database PowerShell
cmdlets available out of the box, so that you can perform all of your SQL DB management tasks within the service.
You can also pair these cmdlets in Azure Automation with the cmdlets for other Azure services, to automate
complex tasks across Azure services and 3rd party systems.
Azure Automation also has the ability to communicate with SQL servers directly, by issuing SQL commands using
PowerShell.
The Azure Automation runbook gallery contains a variety of product team and community runbooks to get started
automating management of Azure SQL Databases, other Azure services, and 3rd party systems. Gallery runbooks
include:
Run SQL queries against a SQL Server database
Vertically scale (up or down) an Azure SQL Database on a schedule
Truncate a SQL table if its database approaches its maximum size
Index tables in an Azure SQL Database if they are highly fragmented

Next steps
Now that you've learned the basics of Azure Automation and how it can be used to manage Azure SQL databases,
follow these links to learn more about Azure Automation.
Azure Automation Overview
My first runbook
Azure Automation learning map
Azure Automation: Your SQL Agent in the Cloud
Use SQL Server Management Studio in Azure
RemoteApp to connect to SQL Database
4/14/2017 • 4 min to read • Edit Online

IMPORTANT
Azure RemoteApp is being discontinued. Read the announcement for details.

Introduction
This tutorial shows you how to use SQL Server Management Studio (SSMS) in Azure RemoteApp to connect to SQL
Database. It walks you through the process of setting up SQL Server Management Studio in Azure RemoteApp,
explains the benefits, and shows security features that you can use in Azure Active Directory.
Estimated time to complete: 45 minutes

SSMS in Azure RemoteApp

Azure RemoteApp is an RDS service in Azure that delivers applications. You can learn more about it here: What is
RemoteApp?
SSMS running in Azure RemoteApp gives you the same experience as running SSMS locally.
Benefits
There are many benefits to using SSMS in Azure RemoteApp, including:
Port 1433 on Azure SQL server does not have to be exposed externally (outside of Azure).
No need to keep adding and removing IP addresses in the Azure SQL server firewall.
All Azure RemoteApp connections occur over HTTPS on port 443 using encrypted Remote Desktop protocol
It is multi-user and can scale.
There is a performance gain from having SSMS in the same region as the SQL Database.
You can audit use of Azure RemoteApp with the Premium edition of Azure Active Directory which has user
activity reports.
You can enable multi-factor authentication (MFA).
Access SSMS anywhere when using any of the supported Azure RemoteApp clients which includes iOS, Android,
Mac, Windows Phone, and Windows PC’s.

Create the Azure RemoteApp collection

Here are the steps to create your Azure RemoteApp collection with SSMS:
1. Create a new Windows VM from Image
Use the "Windows Server Remote Desktop Session Host Windows Server 2012 R2" Image from the Gallery to
make your new VM.
2. Install SSMS from SQL Express
Go onto the new VM and navigate to this download page: Microsoft® SQL Server® 2014 Express
There is an option to only download SSMS. After download, go into the install directory and run Setup to install
SSMS.
You also need to install SQL Server 2014 Service Pack 1. You can download it here: Microsoft SQL Server 2014
Service Pack 1 (SP1)
SQL Server 2014 Service Pack 1 includes essential functionality for working with Azure SQL Database.
3. Run Validate script and Sysprep
On the desktop of the VM is a PowerShell script called Validate. Run this by double-clicking. It will verify that the
VM is ready to be used for remote hosting of applications. When verification is complete, it will ask to run sysprep -
choose to run it.
When sysprep completes, it will shut down the VM.
To learn more about creating a Azure RemoteApp image, see: How to create a RemoteApp template image in Azure
4. Capture image
When the VM has stopped running, find it in the current portal and capture it.
To learn more about capturing an image, see Capture an image of an Azure Windows virtual machine created with
the classic deployment model
5. Add to Azure RemoteApp Template images
In the Azure RemoteApp section of the current portal, go to the Template Images tab and click Add. In the pop-up
box, select "Import an image from your Virtual Machines library" and then choose the Image that you just created.
6. Create cloud collection
In the current portal, create a new Azure RemoteApp Cloud Collection. Choose the Template Image that you just
imported with SSMS installed on it.
7. Publish SSMS
On the Publishing tab of your new cloud collection, select Publish an application from the Start Menu and then
choose SSMS from the list.

8. Add users
On the User Access tab you can select the users that will have access to this Azure RemoteApp collection which only
includes SSMS.
9. Install the Azure RemoteApp client application
You can download and install a Azure RemoteApp client here: Download | Azure RemoteApp

Configure Azure SQL server

The only configuration needed is to ensure that Azure Services is enabled for the firewall. If you use this solution,
then you do not need to add any IP addresses to open the firewall. The network traffic that is allowed to the SQL
Server is from other Azure services.

Multi-Factor Authentication (MFA)

MFA can be enabled for this application specifically. Go to the Applications tab of your Azure Active Directory. You
will find an entry for Microsoft Azure RemoteApp. If you click that application and then configure, you will see the
page below where you can enable MFA for this application.
Audit user activity with Azure Active Directory Premium
If you do not have Azure AD Premium, then you have to turn it on in the Licenses section of your directory. With
Premium enabled, you can assign users to the Premium level.
When you go to a user in your Azure Active Directory, you can then go to the Activity tab to see login information
to Azure RemoteApp.

Next steps
After completing all the above steps, you will be able to run the Azure RemoteApp client and log-in with an
assigned user. You will be presented with SSMS as one of your applications, and you can run it as you would if it
were installed on your computer with access to Azure SQL server.
For more information on how to make the connection to SQL Database, see Connect to SQL Database with SQL
Server Management Studio and perform a sample T-SQL query.
That's everything for now. Enjoy!
Configure Azure SQL Database multi-factor
authentication for SQL Server Management Studio
4/14/2017 • 2 min to read • Edit Online

This topic shows you how to configure Azure SQL Database multi-factor authentication for SQL Server
Management Studio.
For an overview of Azure SQL Database multi-factor authentication, see Overview of Azure SQL Database multi-
factor authentication for SQL Server Management Studio.

Configuration steps
1. Configure an Azure Active Directory - For more information, see Integrating your on-premises identities
with Azure Active Directory, Add your own domain name to Azure AD, Microsoft Azure now supports federation
with Windows Server Active Directory, Administering your Azure AD directory, and Manage Azure AD using
Windows PowerShell.
2. Configure MFA - For step-by-step instructions, see Configuring Azure Multi-Factor Authentication.
3. Configure SQL Database or SQL Data Warehouse for Azure AD Authentication - For step-by-step
instructions, see Connecting to SQL Database or SQL Data Warehouse By Using Azure Active Directory
Authentication.
4. Download SSMS - On the client computer, download the latest SSMS (at least August 2016), from Download
SQL Server Management Studio (SSMS).

Connecting by using universal authentication with SSMS

The following steps show how to connect to SQL Database or SQL Data Warehouse by using the latest SSMS.
1. To connect using Universal Authentication, on the Connect to Server dialog box, select Active Directory
Universal Authentication.

2. As usual for SQL Database and SQL Data Warehouse you must click Options and specify the database on the
Options dialog box. Then click Connect.
3. When the Sign in to your account dialog box appears, provide the account and password of your Azure
Active Directory identity.
NOTE
For Universal Authentication with an account which does not require MFA, you connect at this point. For users
requiring MFA, continue with the following steps.

4. Two MFA setup dialog boxes might appear. This one time operation depends on the MFA administrator
setting, and therefore may be optional. For an MFA enabled domain this step is sometimes pre-defined (for
example, the domain requires users to use a smartcard and pin).
5. The second possible one time dialog box allows you to select the details of your authentication method. The
possible options are configured by your administrator.

6. The Azure Active Directory sends the confirming information to you. When you receive the verification code,
enter it into the Enter verification code box, and click Sign in.
When verification is complete, SSMS connects normally presuming valid credentials and firewall access.

Next steps
For an overview of Azure SQL Database multi-factor authentication, see Overview of Azure SQL Database
multi-factor authentication for SQL Server Management Studio.
Grant others access to your database: SQL Database Authentication and Authorization: Granting Access
Make sure others can connect through the firewall: Configure an Azure SQL Database server-level firewall rule
using the Azure portal
SQL Database FAQ
4/19/2017 • 9 min to read • Edit Online

What is the current version of SQL Database?

The current version of SQL Database is V12. Version V11 has been retired.

What is the SLA for SQL Database?

We guarantee at least 99.99% of the time customers will have connectivity between their single or elastic Basic,
Standard, or Premium Microsoft Azure SQL Database and our Internet gateway. For more information, see SLA.

How do I reset the password for the server admin?

In the Azure portal click SQL Servers, select the server from the list, and then click Reset Password.

How do I manage databases and logins?

See Managing databases and logins.

How do I make sure only authorized IP addresses are allowed to access

a server?
See How to: Configure firewall settings on SQL Database.

How does the usage of SQL Database show up on my bill?

SQL Database bills on a predictable hourly rate based on both the service tier + performance level for single
databases or eDTUs per elastic pool. Actual usage is computed and pro-rated hourly, so your bill might show
fractions of an hour. For example, if a database exists for 12 hours in a month, your bill shows usage of 0.5 days.
Additionally, service tiers + performance level and eDTUs per pool are broken out in the bill to make it easier to see
the number of database days you used for each in a single month.

What if a single database is active for less than an hour or uses a higher
service tier for less than an hour?
You are billed for each hour a database exists using the highest service tier + performance level that applied during
that hour, regardless of usage or whether the database was active for less than an hour. For example, if you create a
single database and delete it five minutes later your bill reflects a charge for one database hour.
Examples
If you create a Basic database and then immediately upgrade it to Standard S1, you are charged at the Standard
S1 rate for the first hour.
If you upgrade a database from Basic to Premium at 10:00 p.m. and upgrade completes at 1:35 a.m. on the
following day, you are charged at the Premium rate starting at 1:00 a.m.
If you downgrade a database from Premium to Basic at 11:00 a.m. and it completes at 2:15 p.m., then the
database is charged at the Premium rate until 3:00 p.m., after which it is charged at the Basic rates.
How does elastic pool usage show up on my bill and what happens
when I change eDTUs per pool?
Elastic pool charges show up on your bill as Elastic DTUs (eDTUs) in the increments shown under eDTUs per pool
on the pricing page. There is no per-database charge for elastic pools. You are billed for each hour a pool exists at
the highest eDTU, regardless of usage or whether the pool was active for less than an hour.
Examples
If you create a Standard elastic pool with 200 eDTUs at 11:18 a.m., adding five databases to the pool, you are
charged for 200 eDTUs for the whole hour, beginning at 11 a.m. through the remainder of the day.
On Day 2, at 5:05 a.m., Database 1 begins consuming 50 eDTUs and holds steady through the day. Databases 2-
5 fluctuate between 0 and 80 eDTUs. During the day, you add five other databases that consume varying eDTUs
throughout the day. Day 2 is a full day billed at 200 eDTU.
On Day 3, at 5 a.m. you add another 15 databases. Database usage increases throughout the day to the point
where you decide to increase eDTUs for the pool from 200 to 400 at 8:05 p.m. Charges at the 200 eDTU level
were in effect until 8 pm and increases to 400 eDTUs for the remaining four hours.

Elastic pool billing and pricing information

Elastic pools are billed per the following characteristics:
An elastic pool is billed upon its creation, even when there are no databases in the pool.
An elastic pool is billed hourly. This is the same metering frequency as for performance levels of single
databases.
If an elastic pool is resized to a new number of eDTUs, then the pool is not billed according to the new amount
of eDTUS until the resizing operation completes. This follows the same pattern as changing the performance
level of single databases.
The price of an elastic pool is based on the number of eDTUs of the pool. The price of an elastic pool is
independent of the number and utilization of the elastic databases within it.
Price is computed by (number of pool eDTUs)x(unit price per eDTU).
The unit eDTU price for an elastic pool is higher than the unit DTU price for a single database in the same service
tier. For details, see SQL Database pricing.
To understand the eDTUs and service tiers, see SQL Database options and performance.

How does the use of Active Geo-Replication in an elastic pool show up

on my bill?
Unlike single databases, using Active Geo-Replication with elastic databases doesn't have a direct billing impact.
You are only charged for the eDTUs provisioned for each of the pools (primary pool and secondary pool)

How does the use of the auditing feature impact my bill?

Auditing is built into the SQL Database service at no extra cost and is available to Basic, Standard, Premium, and
Premium RS databases. However, to store the audit logs, the auditing feature uses an Azure Storage account, and
rates for tables and queues in Azure Storage apply based on the size of your audit log.

How do I find the right service tier and performance level for single
databases and elastic pools?
There are a few tools available to you.
For on-premises databases, use the DTU sizing advisor to recommend the databases and DTUs required, and
evaluate multiple databases for elastic pools.
If a single database would benefit from being in a pool, Azure's intelligent engine recommends an elastic pool if
it sees a historical usage pattern that warrants it. See Monitor and manage an elastic pool with the Azure portal.
For details about how to do the math yourself, see Price and performance considerations for an elastic pool
To see whether you need to dial a single database up or down, see performance guidance for single databases.

How often can I change the service tier or performance level of a single
database?
You can change the service tier (between Basic, Standard, Premium, and Premium RS) or the performance level
within a service tier (for example, S1 to S2) as often as you want. For earlier version databases, you can change the
service tier or performance level a total of four times in a 24-hour period.

How often can I adjust the eDTUs per pool?

As often as you want.

How long does it take to change the service tier or performance level of
a single database or move a database in and out of an elastic pool?
Changing the service tier of a database and moving in and out of a pool requires the database to be copied on the
platform as a background operation. Changing the service tier can take from a few minutes to several hours
depending on the size of the databases. In both cases, the databases remain online and available during the move.
For details on changing single databases, see Change the service tier of a database.

When should I use a single database vs. elastic databases?

In general, elastic pools are designed for a typical software-as-a-service (SaaS) application pattern, where there is
one database per customer or tenant. Purchasing individual databases and overprovisioning to meet the variable
and peak demand for each database is often not cost efficient. With pools, you manage the collective performance
of the pool, and the databases scale up and down automatically. Azure's intelligent engine recommends a pool for
databases when a usage pattern warrants it. For details, see Elastic pool guidance.

What does it mean to have up to 200% of your maximum provisioned

database storage for backup storage?
Backup storage is the storage associated with your automated database backups that are used for Point-In-Time-
Restore and Geo-Restore. Microsoft Azure SQL Database provides up to 200% of your maximum provisioned
database storage of backup storage at no additional cost. For example, if you have a Standard DB instance with a
provisioned DB size of 250 GB, you are provided with 500 GB of backup storage at no additional charge. If your
database exceeds the provided backup storage, you can choose to reduce the retention period by contacting Azure
Support or pay for the extra backup storage billed at standard Read-Access Geographically Redundant Storage
(RA-GRS) rate. For more information on RA-GRS billing, see Storage Pricing Details.

I'm moving from Web/Business to the new service tiers, what do I need
to know?
Azure SQL Web and Business databases are now retired. The Basic, Standard, Premium, Premium RS, and Elastic
tiers replace the retiring Web and Business databases.
What is an expected replication lag when geo-replicating a database
between two regions within the same Azure geography?
We are currently supporting an RPO of five seconds and the replication lag has been less than that when the geo-
secondary is hosted in the Azure recommended paired region and at the same service tier.

What is an expected replication lag when geo-secondary is created in

the same region as the primary database?
Based on empirical data, there is not too much difference between intra-region and inter-region replication lag
when the Azure recommended paired region is used.

If there is a network failure between two regions, how does the retry
logic work when Geo-Replication is set up?
If there is a disconnect, we retry every 10 seconds to re-establish connections.

What can I do to guarantee that a critical change on the primary

database is replicated?
The geo-secondary is an async replica and we do not try to keep it in full sync with the primary. But we provide a
method to force synchronization to ensure the replication of critical changes (for example, password updates).
Forced synchronization impacts performance because it blocks the calling thread until all committed transactions
are replicated. For details, see sp_wait_for_database_copy_sync.

What tools are available to monitor the replication lag between the
primary database and geo-secondary?
We expose the real-time replication lag between the primary database and geo-secondary through a DMV. For
details, see sys.dm_geo_replication_link_status.

To move a database to a different server in the same subscription

In the Azure portal, click SQL databases, select a database from the list, and then click Copy. See Copy an Azure
SQL database for more detail.

To move a database between subscriptions

In the Azure portal, click SQL servers and then select the server that hosts your database from the list. Click
Move, and then pick the resources to move and the subscription to move to.
Public data sets for testing and prototyping
4/14/2017 • 4 min to read • Edit Online

Browse this list of public data sets for data that you can use to prototype and test storage and analytics services and
solutions.

U.S. Government and agency data

DATA SOURCE ABOUT THE DATA ABOUT THE FILES

US Government data Over 190,000 data sets covering Files of various sizes in various formats
agriculture, climate, consumer, including HTML, XML, CSV, JSON, Excel,
ecosystems, education, energy, finance, and many others. You can filter
health, local government, available data sets by file format.
manufacturing, maritime, ocean, public
safety, and science and research in the
U.S.

US Census data Statistical data about the population of Data sets are in various formats.
the U.S.

Earth science data from NASA Over 32,000 data collections covering Data sets are in various formats.
agriculture, atmosphere, biosphere,
climate, cryosphere, human dimensions,
hydrosphere, land surface, oceans, sun-
earth interactions, and more.

Airline flight delays and other "The U.S. Department of Files are in CSV format.
transportation data Transportation's (DOT) Bureau of
Transportation Statistics (BTS) tracks the
on-time performance of domestic flights
operated by large air carriers. Summary
information on the number of on-time,
delayed, canceled, and diverted flights
appears ... in summary tables posted on
this website."

Traffic fatalities - US Fatality Analysis "FARS is a nationwide census providing "Create your own fatality data run
Reporting System (FARS) NHTSA, Congress, and the American online by using the FARS Query System.
public yearly data regarding fatal Or download all FARS data from 1975
injuries suffered in motor vehicle traffic to present from the FTP Site."
crashes."

Toxic chemical data - EPA Toxicity "EPA's most updated, publicly available Data sets are available in various
ForeCaster (ToxCast™) data high-throughput toxicity data on formats including spreadsheets, R
thousands of chemicals. This data is packages, and MySQL database files.
generated through the EPA's ToxCast
research effort."
DATA SOURCE ABOUT THE DATA ABOUT THE FILES

Toxic chemical data - NIH Tox21 Data "The 2014 Tox21 data challenge is Data sets are available in SMILES and
Challenge 2014 designed to help scientists understand SDF formats. The data provides "assay
the potential of the chemicals and activity data and chemical structures on
compounds being tested through the the Tox21 collection of ~10,000
Toxicology in the 21st Century initiative compounds (Tox21 10K)."
to disrupt biological pathways in ways
that may result in toxic effects."

Biotechnology and genome data from Multiple data sets covering genes, Data sets are in text, XML, BLAST, and
the NCBI genomes, and proteins. other formats. A BLAST app is available.

Other statistical and scientific data

DATA SOURCE ABOUT THE DATA ABOUT THE FILES

New York City taxi data "Taxi trip records include fields capturing Data sets are in CSV files by month.
pick-up and drop-off dates/times, pick-
up and drop-off locations, trip
distances, itemized fares, rate types,
payment types, and driver-reported
passenger counts."

Microsoft Research data sets - "Data Multiple data sets covering human- Data sets are in various formats, zipped
Science for Research" computer interaction, audio/video, data for download.
mining/information retrieval,
geospatial/location, natural language
processing, and robotics/computer
vision.

Public genome data "A diverse data set of whole human Data sets, after extraction, are in UNIX
genomes are freely available for public text format. Analysis tools are also
use to enhance any genomic study..." available.
The provider, Complete Genomics, is a
private for-profit corporation.

Open Science Data Cloud data "The Open Science Data Cloud provides Data sets are in various formats.
the scientific community with resources
for storing, sharing, and analyzing
terabyte and petabyte-scale scientific
datasets."

Global climate data - WorldcLIM "WorldClim is a set of global climate These files contain geospatial data. For
layers (gridded climate data) with a more info, see Data format.
spatial resolution of about 1 km2. These
data can be used for mapping and
spatial modeling."

Data about human society - The GDELT "The GDELT Project is the largest, most The raw data files are in CSV format.
Project comprehensive, and highest resolution
open database of human society ever
created."

Advertising click prediction data for "The largest ever publicly released ML
machine learning from Criteo dataset." For more info, see Criteo's 1
TB Click Prediction Dataset.
DATA SOURCE ABOUT THE DATA ABOUT THE FILES

ClueWeb09 text mining data set from "The ClueWeb09 dataset was created to See Dataset Information.
The Lemur Project support research on information
retrieval and related human language
technologies. It consists of about 1
billion web pages in 10 languages that
were collected in January and February
2009."

Online service data

DATA SOURCE ABOUT THE DATA ABOUT THE FILES

GitHub archive "GitHub Archive is a project to record Download JSON-encloded event

the public GitHub timeline [of events], archives in .gz (Gzip) format from a web
archive it, and make it easily accessible client.
for further analysis."

GitHub activity data from The "The GHTorrent project [is] an effort to MySQL database dumps are in CSV
GHTorrent project create a scalable, queriable, offline format.
mirror of data offered through the
GitHub REST API. GHTorrent monitors
the GitHub public event time line. For
each event, it retrieves its contents and
their dependencies, exhaustively."

Stack Overflow data dump "This is an anonymized dump of all "Each site [such as Stack Overflow] is
user-contributed content on the Stack formatted as a separate archive
Exchange network [including Stack consisting of XML files zipped via 7-zip
Overflow]." using bzip2 compression. Each site
archive includes Posts, Users, Votes,
Comments, PostHistory, and PostLinks."
Azure SQL Database customer implementation
technical studies
3/10/2017 • 1 min to read • Edit Online

Daxko: Daxko/CSI Software faced a challenge: its customer base of fitness and recreation centers was
growing rapidly, thanks to the success of its comprehensive enterprise-software solution, but keeping up
with the IT-infrastructure needs for that growing customer base was testing the company’s IT staff. The
company was increasingly constrained by rising operations overhead, particularly for managing its growing
databases. Worse, that operations overhead was cutting into development resources for new initiatives, like
new mobility features for the company’s software.
GEP: GEP delivers software and services that enable procurement leaders around the world to maximize
their impact on their businesses’ operations, strategies, and financial performances. In addition to consulting
and managed services, the company offers SMART by GEP®, a cloud-based, comprehensive procurement-
software platform. However, GEP ran into limitations trying to support solutions like SMART by GEP with its
own on-premises datacenters: the investments needed were steep and regulatory requirements in other
countries would have made the necessary investments more daunting still. By moving to the cloud, GEP has
freed up IT resources, allowing it to worry less about IT operations and to focus more on developing new
sources of value for its customers across the globe.
SnelStart: SnelStart makes popular financial- and business-management software for small- and medium-
sized businesses (SMBs) in the Netherlands. Its 55,000 customers are serviced by a staff of 110 employees,
including an IT staff of 35. By moving from desktop software to a software-as-a-service (SaaS) offering built
on Azure, SnelStart made the most of built-in services, automating management using familiar environment
in C#, and optimizing performance and scalability by neither over- or under-provisioning businesses using
elastic pools. Using Azure provides SnelStart the fluidity of moving customers between on-premises and the
cloud.
Umbraco: To simplify customer deployments, Umbraco added Umbraco-as-a-Service (UaaS): a software-as-
a-service (SaaS) offering that eliminates the need for on-premises deployments, provides built-in scaling,
and removes management overhead by enabling developers to focus on product innovation rather than
solution management. Umbraco is able to provide all those benefits by relying on the flexible platform-as-a-
service (PaaS) model offered by Microsoft Azure.

Database Systems For Software Engineers SOEN 363 - Winter 2023 Assignment 1
No ratings yet
Database Systems For Software Engineers SOEN 363 - Winter 2023 Assignment 1
5 pages
Exploring Hadoop Ecosystem (Volume 2): Stream Processing
From Everand
Exploring Hadoop Ecosystem (Volume 2): Stream Processing
Wei Liu
No ratings yet
MS SQL Server Interview Questions
100% (4)
MS SQL Server Interview Questions
17 pages
SQL Server High Availability
No ratings yet
SQL Server High Availability
49 pages
SQL Server 2016 High Availability Unleashed PDF
100% (1)
SQL Server 2016 High Availability Unleashed PDF
587 pages
App Gateway PDF
No ratings yet
App Gateway PDF
296 pages
20764C ENU Companion PDF
No ratings yet
20764C ENU Companion PDF
192 pages
Microsoft Azure Cloud For Solution Architects
No ratings yet
Microsoft Azure Cloud For Solution Architects
28 pages
SnapLogic Second Edition
From Everand
SnapLogic Second Edition
Gerardus Blokdyk
No ratings yet
Ds-140 Design Specification Test
No ratings yet
Ds-140 Design Specification Test
19 pages
Microsoft® Azure™ SQL Database Step by Step PDF
No ratings yet
Microsoft® Azure™ SQL Database Step by Step PDF
48 pages
SQL DW
No ratings yet
SQL DW
596 pages
Microsoft SQL Azure Enterprise Application Development
From Everand
Microsoft SQL Azure Enterprise Application Development
Jayaram Krishnaswamy
No ratings yet
SQL Server Interview Questions You'll Most Likely Be Asked
From Everand
SQL Server Interview Questions You'll Most Likely Be Asked
Vibrant Publishers
No ratings yet
Introduction to Oracle Database Administration
From Everand
Introduction to Oracle Database Administration
Ying Wang
5/5 (1)
Microsoft Azure 2022-05-25
No ratings yet
Microsoft Azure 2022-05-25
2,106 pages
Cosmosdb: Understanding The Main Factors For Successful Deployment
No ratings yet
Cosmosdb: Understanding The Main Factors For Successful Deployment
58 pages
Ssrs Book
No ratings yet
Ssrs Book
117 pages
Azure SQL Database Migration Cookbook
No ratings yet
Azure SQL Database Migration Cookbook
41 pages
Azure Data Fundamental
No ratings yet
Azure Data Fundamental
81 pages
SQL Server
100% (1)
SQL Server
163 pages
Azure SQL Comparison Datasheet
No ratings yet
Azure SQL Comparison Datasheet
3 pages
MicrosoftAzureDataProtection Aug2014A
No ratings yet
MicrosoftAzureDataProtection Aug2014A
39 pages
MSSQL - SQL Server DBA Survival Guide PDF
100% (1)
MSSQL - SQL Server DBA Survival Guide PDF
936 pages
Microsoft Azure Platform: November 3rd, 2010 COMS W6998-6
No ratings yet
Microsoft Azure Platform: November 3rd, 2010 COMS W6998-6
71 pages
Database Partitioning With MySQL
No ratings yet
Database Partitioning With MySQL
6 pages
SQL Dba
No ratings yet
SQL Dba
275 pages
SQL Server DBA Training
100% (1)
SQL Server DBA Training
6 pages
Azure PDF
100% (4)
Azure PDF
2,267 pages
Planning SQL Server
No ratings yet
Planning SQL Server
75 pages
SQL Server Database Configuration Best Practices
No ratings yet
SQL Server Database Configuration Best Practices
19 pages
Azure
100% (1)
Azure
71 pages
Microsoft SQL 2016 Step by Step Two Node Cluster
No ratings yet
Microsoft SQL 2016 Step by Step Two Node Cluster
53 pages
Implementing An Azure Data Solution DP-200 - DumpsTool - Mansoor
No ratings yet
Implementing An Azure Data Solution DP-200 - DumpsTool - Mansoor
4 pages
Azure Storage PDF
No ratings yet
Azure Storage PDF
28 pages
SQL Server 2014 AlwaysOn AG Failover
No ratings yet
SQL Server 2014 AlwaysOn AG Failover
22 pages
An Introduction To SQL Server Clustering
No ratings yet
An Introduction To SQL Server Clustering
10 pages
Hive Succinctly
No ratings yet
Hive Succinctly
114 pages
Azure Cosmos DB
100% (1)
Azure Cosmos DB
54 pages
Lab 1 - Getting Started With Azure Data Factory
No ratings yet
Lab 1 - Getting Started With Azure Data Factory
5 pages
SQL Server Theory
No ratings yet
SQL Server Theory
2 pages
Microsoft SQL Server Interview Questions & Answers
100% (2)
Microsoft SQL Server Interview Questions & Answers
30 pages
Introduction To Mobile Development With Xamarin
No ratings yet
Introduction To Mobile Development With Xamarin
56 pages
SQL Server In-Memory OLTP PDF
No ratings yet
SQL Server In-Memory OLTP PDF
208 pages
Core Application Development
No ratings yet
Core Application Development
871 pages
RDBMS To MongoDB Migration
No ratings yet
RDBMS To MongoDB Migration
19 pages
SQL Server 2014 DBA
100% (1)
SQL Server 2014 DBA
148 pages
SQL Server 2005 Admin
No ratings yet
SQL Server 2005 Admin
1,185 pages
Microservice Using ASP Core
No ratings yet
Microservice Using ASP Core
22 pages
Basic SQL
No ratings yet
Basic SQL
147 pages
Azure Developer S Cheat Sheet
No ratings yet
Azure Developer S Cheat Sheet
2 pages
Best Practices Every SQL Server DBA Must Know Pensacola
100% (1)
Best Practices Every SQL Server DBA Must Know Pensacola
27 pages
MCTS Certification Microsoft SQL Server 2005 Database Essentials Step by Step PDF
No ratings yet
MCTS Certification Microsoft SQL Server 2005 Database Essentials Step by Step PDF
96 pages
Database Structures
No ratings yet
Database Structures
30 pages
Building Web Services With Microsoft Azure - Sample Chapter
No ratings yet
Building Web Services With Microsoft Azure - Sample Chapter
53 pages
Cloud Service Map: Microsoft Azure & AWS
No ratings yet
Cloud Service Map: Microsoft Azure & AWS
37 pages
Postgre SQL PDF
100% (1)
Postgre SQL PDF
159 pages
Monitoring and Tuning Azure SQL Database: Warner Chaves
No ratings yet
Monitoring and Tuning Azure SQL Database: Warner Chaves
25 pages
Getting Started with SQL Server 2014 Administration
From Everand
Getting Started with SQL Server 2014 Administration
Gethyn Ellis
No ratings yet
Microsoft SQL Server 2008 R2 Administration Cookbook
From Everand
Microsoft SQL Server 2008 R2 Administration Cookbook
Satya Shyam K Jayanty
5/5 (1)
SQL Server: Tips and Tricks - 1
From Everand
SQL Server: Tips and Tricks - 1
Priyanka Agarwal
5/5 (1)
Apache Spark 2.x Cookbook
From Everand
Apache Spark 2.x Cookbook
Rishi Yadav
No ratings yet
How To Implement 'Promoted Columns' On ORACLE Database For TAFJ
No ratings yet
How To Implement 'Promoted Columns' On ORACLE Database For TAFJ
25 pages
Write A Routine To Call PRODUCE - DEAL.SLIP As Shown Below
No ratings yet
Write A Routine To Call PRODUCE - DEAL.SLIP As Shown Below
3 pages
BCP Restore Guide
No ratings yet
BCP Restore Guide
6 pages
TAFJ R18 Release Notes
No ratings yet
TAFJ R18 Release Notes
10 pages
SAS Text Miner: Automate Discovery and Insights From Document Collections
No ratings yet
SAS Text Miner: Automate Discovery and Insights From Document Collections
4 pages
Program To Multiply Two 16 Bit Numbers ProjectsGeek
No ratings yet
Program To Multiply Two 16 Bit Numbers ProjectsGeek
5 pages
9.1.1.7 Lab - Encrypting and Decrypting Data Using A Hacker Tool
No ratings yet
9.1.1.7 Lab - Encrypting and Decrypting Data Using A Hacker Tool
6 pages
Service Provider Service Consumers: Service Registry Service Registry
No ratings yet
Service Provider Service Consumers: Service Registry Service Registry
34 pages
Token Keyword Identifiers in C++
No ratings yet
Token Keyword Identifiers in C++
2 pages
w5500 Ds V100e
No ratings yet
w5500 Ds V100e
65 pages
Af
No ratings yet
Af
45 pages
Lecture # 1-2-Intro
No ratings yet
Lecture # 1-2-Intro
55 pages
212 CS204 Mid-Term - Ans
No ratings yet
212 CS204 Mid-Term - Ans
8 pages
Mysql 5.5.24 Tokudb 6.5.1 Users Guide
No ratings yet
Mysql 5.5.24 Tokudb 6.5.1 Users Guide
40 pages
DBMS Normalization Normalization: Types of Normal Forms
No ratings yet
DBMS Normalization Normalization: Types of Normal Forms
17 pages
Word Count Program With MapReduce and Java
No ratings yet
Word Count Program With MapReduce and Java
6 pages
B Qradar API
No ratings yet
B Qradar API
1,756 pages
MPLAB XC16 C Compiler Users Guide DS50002071 PDF
No ratings yet
MPLAB XC16 C Compiler Users Guide DS50002071 PDF
413 pages
MP150 User Manual_V1.1.5
No ratings yet
MP150 User Manual_V1.1.5
112 pages
Multimedia Systems: CSC 461/561 Video Compression
No ratings yet
Multimedia Systems: CSC 461/561 Video Compression
8 pages
PART-B: Simulation Experiments Using MATLAB: Experiment-1
No ratings yet
PART-B: Simulation Experiments Using MATLAB: Experiment-1
11 pages
Effective Web Searching
No ratings yet
Effective Web Searching
13 pages
Part 7 - Data Tables - Participants
No ratings yet
Part 7 - Data Tables - Participants
4 pages
ADVPN Technical Deep Dive
No ratings yet
ADVPN Technical Deep Dive
51 pages
Neo4j Manual PDF
No ratings yet
Neo4j Manual PDF
334 pages
Linux Essentials Final Comprehensive Exam
No ratings yet
Linux Essentials Final Comprehensive Exam
7 pages
SRDF Replication: SRDF Synchronous (SRDF/S) SRDF Asynchronous (SRDF/A)
No ratings yet
SRDF Replication: SRDF Synchronous (SRDF/S) SRDF Asynchronous (SRDF/A)
6 pages
Insert, Update, and Delete Destination Table With SSIS - Reza Rad's Technical Blog
No ratings yet
Insert, Update, and Delete Destination Table With SSIS - Reza Rad's Technical Blog
17 pages
Sitecore Certification Exam Syllabus
No ratings yet
Sitecore Certification Exam Syllabus
5 pages
Networking
100% (1)
Networking
33 pages
Mapinfo Universal Translator
No ratings yet
Mapinfo Universal Translator
22 pages