Python Module_5 Notes (Revised)
Python Module_5 Notes (Revised)
Module-5
Networked Programs
• Network Programming involves, writing programs that communicate with other programs
across a computer network to share the data.
• Python provides various libraries to implement the network protocols.
• Python provides two levels of access to network services
o Low-Level Access
o High-Level Access
Low-Level Access:
o At low level, we can access the socket support of the operating system using Python
libraries. It allows to implement client and server.
High-Level Access
o Python also provides libraries to implement High level (Application-level) network
protocols HTTP, FTP, etc.
Socket Programming
o Sockets are endpoints in bidirectional communication channel.
o The sockets (endpoints) can communicate within a process, between processes in the same
machine, or between the processes on the different machines.
o Single socket provides a two-way connection, ie we can both read from and write to the same
socket.
o The Python provides built-in library called socket to make network connections and to
access data over other sockets
s.close()
Explanation:
To cerate socket and send the request
1. First to import the socket library
2. Create the socket using socket.socket () method
3. Now to connect to the website using connect () method
4. To send request to webpage:
i. First create request for the web page GET method of the HTTP and store it in
cmd variable.
ii. Data sent should be in bytes format, so convert from string to byte using encode () method
and again store back in cmd variable.
iii. Now send request stored in cmd variable to the server using send () method.
Note ; HTTP/1.0: This specifies the version of the HTTP protocol i.e HTTP version 1.0.
Here \r\n means next line and \r\n\r\n represents the end of HTTP header.
5. To receive the data from webpage
i. Create the one variable new_data of type byte to store the received data
ii. We have to receive the in 512-character chunks, so extract entire data we have to use
loop
6. Close the connection using close method
3. send() : The send () method of Python's socket module is used to send data from one socket
to another socket.
4. encode() : The data sent should be in bytes format. String data can be converted to bytes by using the
encode() method
5. recv() : This function is used to receive the contents of the web page
6. close() : This function is used to close the socket connection
picture = b""
while True:
data = mysock.recv(20)
if (len(data) < 1):
break
picture = picture + data
mysock.close()
fhand = open('Image1.jpg', 'wb') #To store the image in the jpeg form
fhand.write(picture)
fhand.close()
Explanation:
To cerate socket and send the request
1. First to import the socket library
2. Create the socket using socket.socket () method
3. Now to connect to the website using connect () method
4. Create request for the web page by using GET method of the HTTP protocol and store it in
cmd variable
5. Data sent should be in bytes format, so convert from string to byte using encode () method
and again store back in cmd variable
6. Now send request stored in cmd variable to the server using send () method
Here \r\n means next line and \r\n\r\n means one complete blank line.
It means we are sending the blank line (message) to server as request.
7. Create the one variable picture of type byte to store the received data
8. To fetch or receive the data:Once we send request, then write a loop to receive data in 512-
character chunks and prints the data. This is repeated until there is no more data to read.
9. Remove the header information
10. Store the image in the jpeg form
file1 = urllib.request.urlopen('http://data.pr4e.org/romeo.txt')
Explanation
o Open the web page using urllib.urlopen.
o Then we can treat webpage like a file and read through it using a for loop.
o When we run program, output will be only contents of the file. The headers are still
sent, but the urllib code consumes the headers and only returns the data to us.
Example: To write a program to retrieve the data and compute the frequency of each word
import urllib.request,
import urllib.parse,
import urllib.error
counts = dict()
file1 = urllib.request.urlopen('http://data.pr4e.org/romeo.txt')
(Regular expression: the expression which conations the word or pattern to be searched)
Here,
<h1> and </h1> are the beginning and end of header tags
<p> and </p> are the beginning and end of paragraph tags
<a> and </a> The URL link is mentioned between these two anchor tags
href : It indicates start of hyperlink
• From above, we can understand that, if we want to extract hyperlinks from webpage, then
regular expression should the href =http.
• Now let us create a regular expression as: href=http://.+?
• Here, .+? indicates that, search all possible smallest matching strings.
import urllib.request
import urllib.parse
import re
url = “http://www.pyhton.org”
s = urllib.request.urlopen(url) #open the webpage
html = s.read() #read the the webpage
links = re.findall (b 'href="(http://.*?)’, html) #regular expression
for link in links:
print(link.decode())
• XML (eXtensible Markup Language): Best suited for exchanging document-style data
• JSON (JavaScript Object Notation):To exchange dictionaries, lists, or other international
information with each other.
• CSV
• XML is the markup language similar to HTML. It is structured and tag-based language
• This is mainly used to exchange or transmit the data across the web. It is most suitable
for exchanging document style data.
• The XML is similar to HTML, but the difference is, HTML has predefined tags, but in
XML, developer can create his own tags (user defined). That is why it is called
EXTENSIBLE
• XML is more structured than HTML.
• The most important part of XML are XML tags.
• Two important tags of XML are: start tag & end tag
• The content (data) or instructions have to be placed between these starts and stop tags.
• Start Tag: Every XML element or content begins with start-tag.
Example: <name>
• End Tag: Every element that has a start tag should end with an end-tag.
Note: end tag includes a ("/")
Example </name>
• Simple example for xml document
• We can think XML document as a tree structure. Here Player is top tag or root tag(node)
and other tags such as name, age are child elements.
Parsing XML (To retrieve or extract the node present in the XML tree)
(Parsing means extracting the data from text)
• To parse XML document, first XML document has to be converted into Tree structure (Tree
of nodes) and then required node of the tree has to be extracted(parsed).
• To convert the XML into Tree structure, inbuilt functions of Element Tree library are used.
So, first we need to import Element Tree library (xml.etree.ElementTree)
• Mainly 3 functions of Element Tree library are used.
▪ fromstring(): It is used to convert XML code into a tree-structure (tree of XML
nodes.)
▪ find () function is used extract the node (value of the tag) from XML tree
▪ get () function is used to extract the attribute value.
Example:
ET.fromstring(data) => converts the XML data (stored in the data variable) into an Element Tree structure
(tree).
tree.find('name').text =>finds the <name> tag in the XML tree and returns its content ('Dhoni')
tree.find('age').text =>finds the <age> tag in the XML tree and returns its content ('40').
tree.find('runs').text => finds the <runs> tag in the XML tree and returns its text content ('15000').
tree.find('email').get('hide')=> finds the <email> tag in the XML tree and retrieves the value of the 'hide'
attribute ('yes' ).
XML code
data = '''
<Player>
<name>Dhoni</name>
<age> 40</age>
<runs> 15000</runs>
<birthplace>Ranchi <birthplace>
<email hide="yes"/> #here hide is attribute
<Player> '''
Output:
Name:Dhoni
Age: 40
Runs: 15000
Birthplace: Ranchi
Attribute: yes
Note : JSON is simpler than XML and it directly maps to data structures of other programming
languages. This makes parsing and data extraction simpler compared to XML. So it is widely being
used in the IT industry
Parsing JSON
To parse the JSON document, first JSON data has to be converted into python dictionary. For this,
load() function of json library is used. Then we can extract the data from python dictionary
• We know that, we can exchange data between applications using HTTP, XML and JSON.
There should be some set of rules or contract between applications for smoother data
transfer.
• The contract or set of rules between application-to-application is called as Application
Program Interfaces or APIs.
• If program ‘X’ is using an API, then it publishes the APIs (ie rules). If other applications
want to access services provided by X, then these other applications must follow the rules of
X.
• When we are building programs, in our program if we include functionality, which can
access the services provided by other programs, then this approach is called Service-
Oriented Architecture or SOA.
• In SOA approach: our overall application makes use of the services of other applications.
• In non-SOA approach: The application is a single standalone application which contains all
of the code necessary to implement the application.
• When an application makes a set of services available over the web using API, we call these
web services
Example of SOA:
• We can go to a single web site and book the ticket for air-travel and also book hotel room.
• This is possible due to service-oriented Architecture.
• Similarly, the payment for the air-travel or hotel accommodation can be done using credit
card. Thus, there are various services available that are interconnected. These services are
responsible for handling air-booking-data, hotel-booking-data or credit card transaction
• Google map is very popular web service provided by Google. The web service allows us to
make use of their large database of geographic information.
• We can submit a geographic search string like “Bhagya Nagar” to their geocoding API.
Then Google returns the location details (Map containing nearby landmarks)
Database
• Database is organised collection of data
• Database is system in which data is stored, manipulated and retrieved
• We can perform various operations on data when it is stored in database
• There are various database systems:
o oracle,
o MySQL,
o Microsoft SQL Server
o SQLite.
Concept of Database
• If the data is organised in the form of table, then it is called as relational database
• The tables consists of rows and columns.
• The tables are stored inside the database file and each record is stored in particular row and
column of the table.
• In database terminology: table is called relation; row is called tuple and column is called
attribute
Command Meaning
Syntax:
This command creates a table called as Player with the attributes(columns) name, age and
runs. The name is of character data type and age, runs are of Integer type
Embedded SQL: Putting SQL queries into high-level languages (python or java) to get meaningful
& efficient output
• Python has inbuilt library sqlite3 to handle SQLite database.
• This library has methods connect (), execute (), cursor () and commit () to perform operations
on database
• To use Python to handle SQL database
i. First import the python library sqlite3
ii. To create the database: Use connect () method to Create the database (or database
object) and open (connect) it, if already existing, then directly open it.
(Note: connect means create and open)
iii. To create cursor object: Use cursor () method to create cursor object (file handle)
It acts as a file handle or intermediator
iv. To Execute: Execute the SQL commands using execute () method. By using this
method, we can execute the commands like CREATE TABLE, INSERT INTO etc
v. Confirmation: Confirm the execution by using commit method ()
vi. Close the connection (database): Use close() method to close database
import sqlite3
conn=sqlite3.connect(‘cricket.db’) #create & open database by name cricket and store in variable conn
cur=conn.cursor() # creating the cursor object : cur
cur.execute (‘CREATE TABLE Player (name TEXT, age INT, runs INT)’) #create table
conn.close() # close the database or connection
import sqlite3
conn=sqlite3.connect(‘cricket.db’) #create & open database by name cricket and store in variable conn
cur=conn.cursor() # creating the cursor object : cur
cur.execute (‘DROP TABLE Player’) # Delete table
conn.commit() #It is for confirmation
conn.close() # close the database or connection
import sqlite3
conn=sqlite3.connect(‘cricket.db’) #create & open database by name cricket and store in variable conn
cur=conn.cursor() # creating the cursor object : cur
cur.execute (‘CREATE TABLE Player (name TEXT, age INT, runs INT)’) #create table
cur.execute (‘INSERT INTO Player (name, age, runs) VALUES ('Dhoni', 35, 15000)’)
conn.close() # close the database or connection
import sqlite3
conn=sqlite3.connect(‘cricket.db’) #create & open database by name cricket and store in variable conn
cur=conn.cursor() # creating the cursor object : cur
cur.execute (‘UPDATE Player SET runs 20,000 WHERE ‘name’ = virat)
conn.commit() #It is for confirmation
conn.close() # close the database or connection
import sqlite3
conn=sqlite3.connect(‘cricket.db’) #create & open database by name cricket and store in variable conn
cur=conn.cursor() # creating the cursor object : cur
cur.execute (‘DELETE FROM Player WHERE= ‘Dhoni’)
conn.commit() #It is for confirmation
conn.close() # close the database or connection
import sqlite3
conn=sqlite3.connect(‘cricket.db’) #create & open database by name cricket and store in variable conn
cur=conn.cursor() # creating the cursor object : cur
cur.execute (‘SELECT name, age FROM Player’)
Example 7
Write a program to create a Student database with a table consisting of student name and age.
Read n records from the user and insert them into database. Write queries to display all records
and to display the students whose age is 20.
import sqlite3
# Creating Table
cursor.execute('CREATE TABLE Student (name TEXT, age INTEGER)')
#Explanation
• for-loop is used to take the user-input for student’s name and age. These data are inserted
into the table. Question mark acts as a placeholder for user-input variables.
• Later we use a method fetchall() to fetch all the values of the database. Then we display all
the records form the table in the form of tuples.