https://www.halvorsen.

blog

Python
and SQL Server
Hans-Petter Halvorsen
 Free Textbook with lots of Practical Examples

https://www.halvorsen.blog/documents/programming/python/
 Additional Python Resources

https://www.halvorsen.blog/documents/programming/python/
 Contents
• Database Systems
• SQL Server
• SQL Server and Python
• CRUD Python Examples
• Datalogging Example
 What is a Database?
• A Database is a structured way to store lots of information.
• The information inside the database is stored in different
tables.
• - “Everything” today is stored in databases!

Examples:
• Bank/Account systems
• Information in Web pages such as Facebook, Wikipedia,
YouTube, etc.
• … lots of other examples!
 Database Systems
We communicate with the Database using a Database Management
System (DBMS). We use the Structured Query Language (SQL) in
order to communicate with the Database, i.e., Insert Data, Retrieve
Data, Update Data and Delete Data from the Database.

Database
Management
System SQL Database
(DBMS)

SQL – Structured Query Language

 Database Systems
• Oracle
• MySQL
• MariaDB
• Sybase
• Microsoft Access
• Microsoft SQL Server
• ... (we have hundreds different DBMS)
 SQL Server
• SQL Server consists of a Database Engine and a
Management Studio.
• The Database Engine has no graphical interface -
it is just a service running in the background of
your computer (preferable on the server).
• The Management Studio is graphical tool for
configuring and viewing the information in the
database. It can be installed on the server or on
the client (or both).
 SQL Server
• SQL Server Express
– Free version of SQL Server that has all we need for the
exercises in this Tutorial
• SQL Server Express consist of 2 parts (separate
installation packages):
– SQL Server Express
– SQL Server Management Studio (SSMS) – This software can
be used to create Databases, create Tables, Insert/Retrieve
or Modify Data, etc.
• SQL Server Express Installation:
https://youtu.be/hhhggAlUYo8
SQL Server Management Studio
3

Your SQL Server

1 4
Your Database Write your Query here
2

Your
Tables

5 The result from your Query

 Structured Query Language
• Structured Query Language (SQL) is used to
write, read and update data from the
Database System
• You can use SQL inside the “SQL Server
Management Studio” or inside your Python
script.
• SQL Example: select * from SCHOOL
 SQL Examples
Query Examples:
• insert into STUDENT (Name , Number, SchoolId)
values ('John Smith', '100005', 1)

• select SchoolId, Name from SCHOOL

• select * from SCHOOL where SchoolId > 100

• update STUDENT set Name='John Wayne' where StudentId=2

• delete from STUDENT where SchoolId=3

We have 4 different Query Types: INSERT, SELECT, UPDATE and DELETE
CRUD: C – Create or Insert Data, R – Retrieve (Select) Data, U – Update Data, D – Delete Data
https://www.halvorsen.blog

Python

Hans-Petter Halvorsen
 Python
• Python is a fairly old Programming Language (1991)
compared to many other Programming Languages like C#
(2000), Swift (2014), Java (1995), PHP (1995).
• Python has during the last 10 years become more and more
popular.
• Today, Python has become one of the most popular
Programming Languages.
Software used in this Tutorial:
• Anaconda Distribution (Python + most used
Libraries/Packages are included)
• Spyder Python editor (included with Anaconda Distribution)
Python Drivers for SQL Server
• There are several python SQL drivers available:
– pyodbc
– pymssql
• These Drivers are not made made Microsoft but the
Python Community.
• However, Microsoft places its testing efforts and its
confidence in pyodbc driver.
• Microsoft contributes to the pyODBC open-source
community and is an active participant in the
repository at GitHub
https://docs.microsoft.com/sql/connect/python/python-driver-for-sql-server
 pyodbc
• pyodbc is an open-source Python
module that can access ODBC
databases, e.g., SQL Server
• https://pypi.org/project/pyodbc/
• Installation: pip install pyodbc
pyodbc

pip install pyodbc

https://www.halvorsen.blog

Python Examples

Hans-Petter Halvorsen
 Database CRUD
All Database Systems supports CRUD
C – Create or Insert Data
R – Retrieve Data
U – Update Data
D – Delete Data
Let's go through some Python examples
 Python Examples
Note!
• The examples provided can be
considered as a “proof of concept”
• The sample code is simplified for
clarity and doesn't necessarily
represent best practices.
 SQL Server Database
Let's Create a simple Table called “BOOK”:
CREATE TABLE [BOOK]
(
[BookId] [int] IDENTITY(1, 1) NOT NULL PRIMARY KEY,
[Title] [varchar](50) NOT NULL UNIQUE,
[Author] [varchar](50) NOT NULL,
[Category] [varchar](50) NOT NULL
)
GO
 SQL Server Database
Let’s insert some data into the BOOK Table:
INSERT INTO BOOK (Title, Author, Category)
VALUES ('Python Program','Knut Hamsun', 'Data’)
Go

INSERT INTO BOOK (Title, Author, Category)

VALUES ('Music History','Elvis Presley', 'Music’)
GO

INSERT INTO BOOK (Title, Author, Category)

VALUES ('Et Dukkehjem','Henrik Ibsen', 'Novel')
GO
 SQL Server Database
We use SQL Server management Studio in order to create the Table and Data:
 SQL Server Database
We see that the Table and the Data have been created:
https://www.halvorsen.blog

Connect to Database
using Python
Hans-Petter Halvorsen
 Connect to Database from Python
The newest and
recommend driver
import pyodbc

driver = "{ODBC Driver 17 for SQL Server}"

server = "xxxxxx"
database = "xxxxx"
username = "xxxxx"
password = "xxxxxx"
conn = pyodbc.connect("DRIVER=" + driver
+ ";SERVER=" + server
+ ";DATABASE=" + database
+ ";UID=" + username
+ ";PWD=" + password )
 Connect to Database from Python
If Server is on your local PC,
Example: Server Name
you can use LOCALHOST
import pyodbc

driver = "{ODBC Driver 17 for SQL Server}"

server = “TESTPC\\SQLEXPRESS" Instance Name (you can have
database = "BOOKSTORE" multiple instances of SQL Server
username = "sa" on the same computer)
password = "Test123"
conn = pyodbc.connect("DRIVER=" + driver
+ ";SERVER=" + server
+ ";DATABASE=" + database
+ ";UID=" + username
+ ";PWD=" + password )

Here is the built-in “sa” user (System Administrator) used to connect to the Database. In
general, you should use another user than the sa user. The sa user is used here for
simplicity. You can easily create new user in SQL Server Management Studio
https://www.halvorsen.blog

Retrieve Data
using Python
Hans-Petter Halvorsen
 Python
import pyodbc
1 Python Program Knut Hamsun Data
driver = "{ODBC Driver 17 for SQL Server}"
server = "NUCHPH\\SQLEXPRESS" 2 Music History Elvis Presley Music
database = "BOOKSTORE" 3 Et Dukkehjem Henrik Ibsen Novel
username = "sa"
password = "xxxxxx"
conn = pyodbc.connect("DRIVER=" + driver
+ ";SERVER=" + server
+ ";DATABASE=" + database
+ ";UID=" + username
+ ";PWD=" + password )

cursor = conn.cursor()

for row in cursor.execute("select BookId, Title, Author, Category from BOOK"):

print(row.BookId, row.Title, row.Author, row.Category)
 Python - Alternative
import pyodbc

driver = "{ODBC Driver 17 for SQL Server}"

1 Python Program Knut Hamsun Data
server = "NUCHPH\\SQLEXPRESS" 2 Music History Elvis Presley Music
database = "BOOKSTORE"
username = "sa"
3 Et Dukkehjem Henrik Ibsen Novel
password = "xxxxxx"
conn = pyodbc.connect("DRIVER=" + driver
+ ";SERVER=" + server
+ ";DATABASE=" + database
+ ";UID=" + username
+ ";PWD=" + password )

cursor = conn.cursor()

cursor.execute("select BookId, Title, Author, Category from BOOK")

row = cursor.fetchone()
while row:
print(row[0], row[1], row[2], row[3])
row = cursor.fetchone()
 “Hide” Connection String
You can put the Connection string in a separate Python File, e.g., “database.py”:
def GetConnectionString():
driver = "{ODBC Driver 17 for SQL Server}"
server = "NUCHPH\\SQLEXPRESS"
database = "BOOKSTORE"
username = "sa"
password = "xxxxxx"

connectionString = "DRIVER=" + driver + ";SERVER=" + server + ";DATABASE=" + database + ";UID=" + username + ";PWD=" + password

return connectionString

import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)

cursor = conn.cursor()

for row in cursor.execute("select BookId, Title, Author, Category from BOOK"):

print(row.BookId, row.Title, row.Author, row.Category)
 SELECT … WHERE ..
Using a SELECT statement with a WHERE clause

Example:
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)

cursor = conn.cursor()

query = "select BookId, Title, Author, Category from BOOK where Category='Data'"

for row in cursor.execute(query):

print(row.BookId, row.Title, row.Author, row.Category)
Using Parameters- Avoid SQL Injection
• ODBC supports parameters using a question mark as a
place holder in the SQL. You provide the values for the
question marks by passing them after the SQL
• This is safer than putting the values into the string
because the parameters are passed to the database
separately, protecting against SQL injection attacks.
• It is also be more efficient if you execute the same SQL
repeatedly with different parameters.

https://github.com/mkleehammer/pyodbc/wiki/Getting-started
Using Parameters- Avoid SQL Injection
Example:
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)

cursor = conn.cursor()

query = "select BookId, Title, Author, Category from BOOK where Category=?"

parameters = ['Data']

for row in cursor.execute(query, parameters):

print(row.BookId, row.Title, row.Author, row.Category)
https://www.halvorsen.blog

Insert Data
using Python
Hans-Petter Halvorsen
 INSERT
Basic Example:
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)

cursor = conn.cursor()

query = "INSERT INTO BOOK (Title, Author, Category) VALUES ('Python for Experts', 'Halvorsen', 'Data')"

cursor.execute(query)
cursor.commit()
 INSERT with Row Count
With Row Count: You often want to know how many records were inserted. Then
you can use the Cursor rowcount attribute:
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)

cursor = conn.cursor()

query = "INSERT INTO BOOK (Title, Author, Category) VALUES ('Python for Fun', 'Halvorsen', 'Data')"

count = cursor.execute(query).rowcount

cursor.commit()

print('Rows inserted: ' + str(count))

 INSERT with Parameters
In this example, you see how to run an INSERT statement safely, and pass parameters. The
parameters protect your application from SQL injection.

import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)

cursor = conn.cursor()

query = "INSERT INTO BOOK (Title, Author, Category) VALUES (?,?,?)"

parameters = 'Python for Beginners', 'Hans-Petter Halvorsen', 'Data'

count = cursor.execute(query, parameters).rowcount

cursor.commit()

print('Rows inserted: ' + str(count))

https://www.halvorsen.blog

Modify Data
using Python
Hans-Petter Halvorsen
 UPDATE
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)
cursor = conn.cursor()

query = "UPDATE BOOK SET Title='Python Tutorial' WHERE BookId=5"

cursor.execute(query)
cursor.commit()
 UPDATE with Row Count
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)
cursor = conn.cursor()

query = "UPDATE BOOK SET Title='Python Tutorial' WHERE BookId=5"

count = cursor.execute(query).rowcount
cursor.commit()

print('Rows updated: ' + str(count))

 UPDATE with Parameter
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)
cursor = conn.cursor()

query = "UPDATE BOOK SET Title='Python Tutorial' WHERE BookId=?"

bookId = 5
parameters = bookId

count = cursor.execute(query, parameters).rowcount

cursor.commit()
print('Rows updated: ' + str(count))
https://www.halvorsen.blog

Delete Data
using Python
Hans-Petter Halvorsen
 DELETE
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)

cursor = conn.cursor()

query = "DELETE FROM BOOK WHERE BookId=10"

cursor.execute(query)
cursor.commit()
 DELETE with Row Count
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)
cursor = conn.cursor()

query = "DELETE FROM BOOK WHERE BookId=8"

count = cursor.execute(query).rowcount
cursor.commit()

print('Rows deleted: ' + str(count))

 DELETE with Parameter
import pyodbc
import database

connectionString = database.GetConnectionString()

conn = pyodbc.connect(connectionString)
cursor = conn.cursor()

query = "DELETE FROM BOOK WHERE BookId=?"

parameters = 12

count = cursor.execute(query, parameters).rowcount

cursor.commit()

print('Rows deleted: ' + str(count))

https://www.halvorsen.blog

Datalogging Example

Hans-Petter Halvorsen
 Datalogging Example
• We can log data from a DAQ device or
similar
• We start by creating a simple Random
Generator that simulates a Temperature
Sensor and log these data to the SQL
Server database
• Then we will in another script read the
data from the database and plot them.
 SQL Server Database
Let's create a New Database called, e.g., “LOGGINGSYSTEM”
We insert the following Table:
CREATE TABLE [MEASUREMENTDATA]
(
[MeasurmentId] [int] IDENTITY(1, 1) NOT NULL PRIMARY KEY,
[SensorName] [varchar](50) NOT NULL,
[MeasurementValue] float NOT NULL,
[MeasurementDateTime] datetime NOT NULL
)
GO
 import pyodbc
import random
import time
from datetime import datetime
import database
Logging Data
# Connect to Database
connectionString = database.GetConnectionString()
conn = pyodbc.connect(connectionString)
cursor = conn.cursor()
query = "INSERT INTO MEASUREMENTDATA (SensorName, MeasurementValue, MeasurementDateTime) VALUES (?,?,?)"

sensorName = "Temperature"
Ts = 10 # Sampling Time
N = 20
for k in range(N):
# Generate Random Data
LowLimit = 20
UpperLimit = 25
measurementValue = random.randint(LowLimit, UpperLimit)

#Find Date and Time

now = datetime.now()
datetimeformat = "%Y-%m-%d %H:%M:%S"
measurementDateTime = now.strftime(datetimeformat)

# Insert Data into Database

parameters = sensorName, measurementValue, measurementDateTime
cursor.execute(query, parameters)
cursor.commit()

# Wait
time.sleep(Ts)
Logged Data
 import pyodbc
import matplotlib.pyplot as plt
import database

sensorName = "Temperature"
Plotting Data
# Connect to Database
connectionString = database.GetConnectionString()
conn = pyodbc.connect(connectionString)
cursor = conn.cursor()
query = "SELECT MeasurementValue, MeasurementDateTime FROM MEASUREMENTDATA WHERE SensorName=?"
parameters = [sensorName]

t = []; data = []

# Retrieving and Formatting Data

for row in cursor.execute(query, parameters):
measurementValue = row.MeasurementValue
measurementDateTime = row.MeasurementDateTime

data.append(measurementValue)
t.append(measurementDateTime)

# Plotting
plt.plot(t, data, 'o-')
plt.title('Temperature')
plt.xlabel('t [s]')
plt.ylabel('Temp [degC]')
plt.grid()
plt.show()
Plotted Data
 Additional Python Resources

https://www.halvorsen.blog/documents/programming/python/
Hans-Petter Halvorsen
University of South-Eastern Norway
www.usn.no

E-mail: hans.p.halvorsen@usn.no
Web: https://www.halvorsen.blog