A Winter Training Report

On
Automation using Python

Submitted by :
Sandeep yadav
00496502818
ECE 6E Under the Guidance of
Mr. Al Sweigart
 Acknowledgement

I would like to thank respected Mr. Al Sweigart for giving me such a

wonderful opportunity to expand my knowledge for my own branch and giving
me guidelines to present a report. It helped me a lot to realize of what we study
for.

Secondly, I would like to thank my parents who patiently helped me as i

went through my work and helped to modify and eliminate some of the
irrelevant or unnecessary stuff.

Thirdly, I would like to thank my friends who helped me to make my

work more organized and well-stacked till the end.

Next, I would thank Microsoft for developing such a wonderful tool like MS
Word. It helped my work a lot to remain error-free.

Last but clearly not the least, I would thank The Almighty for giving me
strength to complete my report on time.
 Preface

I have made this report file on the topic Automation using Python ; I
have tried my best to elucidate all the relevant detail to the topic to be
included in the report. While in the beginning I have tried to give a general
view about this topic.

My efforts and wholehearted co-corporation of each and everyone has

ended on a successful note. I express my sincere gratitude to Mr. Al
Sweigart who assisting me throughout the preparation of this topic. I
thank him for providing me the reinforcement, confidence and most
importantly the track for the topic whenever I needed it.
CONTENT

 Introduction to Python

 History of Python

 Why Python ?

 Characteristics of Python

 Data Structures in Python

 Regular Expressions

 File Handling in Python

 Debugging in Python

 Web Scraping using Beautiful Soup

 Email

 GUI Automation

 Conclusion
INTRODUCTION TO PYTHON

Python is a widely used general-purpose, high level programming language.

It was created by Guido van Rossum in 1991 and further developed by the
Python Software Foundation. It was designed with an emphasis on code
readability, and its syntax allows programmers to express their concepts in
fewer lines of code.

Python is a programming language that lets you work quickly and integrate
systems more efficiently.

There are two major Python versions: Python 2 and Python 3. Both are quite
different.
History of Python

Python was developed in 1980 by Guido van Rossum at the National

Research Institute for Mathematics and Computer Science in the
Netherlands as a successor of ABC language capable of exception
handling and interfacing. Python features a dynamic type system and
automatic memory management. It supports multiple programming
paradigms, including object-oriented, imperative, functional and
procedural, and has a large and comprehensive standard library.

Van Rossum picked the name Python for the new language from a
TV show, Monty Python's Flying Circus.

In December 1989 the creator developed the 1st python interpreter

as a hobby and then on 16 October 2000, Python 2.0 was released
with many new features.

...In December 1989, I was looking for a "hobby" programming project

that would keep me occupied during the week around Christmas. My
office ... would be closed, but I had a home computer, and not much
else on my hands. I decided to write an interpreter for the new
scripting language I had been thinking about lately: a descendant of
ABC that would appeal to Unix/C hackers. I chose Python as a working
title for the project, being in a slightly irreverent mood (and a big fan
of Monty Python's Flying Circus)

— Guido van Rossum

Python Releases
 Python 1.0 - January 1994

 Python 1.5 - December 31, 1997

 Python 1.6 - September 5, 2000

 Python 2.0 - October 16, 2000

 Python 2.1 - April 17, 2001

 Python 2.2 - December 21, 2001

 Python 2.3 - July 29, 2003

 Python 2.4 - November 30, 2004

 Python 2.5 - September 19, 2006

 Python 2.6 - October 1, 2008

 Python 2.7 - July 3, 2010

 Python 3.0 - December 3, 2008

 Python 3.1 - June 27, 2009

 Python 3.2 - February 20, 2011

 Python 3.3 - September 29, 2012

 Python 3.4 - March 16, 2014

 Python 3.5 - September 13, 2015

 Python 3.6 - December 23, 2016

 Python 3.7 - June 27, 2018

 Python 3.8 - October 14, 2019

 Python 3.9 - October 5, 2020

Why Python ?
The language's core philosophy is summarized in the document The Zen of
Python (PEP 20), which includes aphorisms such as…

 Beautiful is better than ugly

 Simple is better than complex
 Complex is better than complicated
 Readability counts
 Explicit is better than implicit

A simple program to print “Hello, world!”

Characteristics of Python
Interpreted Language: Python is processed at runtime by Python Interpreter
Execution process of Java

Execution process of Python

• Easy to read: Python source-code is clearly defined and visible to the eyes.

• Portable: Python codes can be run on a wide variety of hardware platforms

having the same interface.

• Extendable: Users can add low level-modules to Python interpreter.

• Scalable: Python provides an improved structure for supporting large

programs than shell-scripts.

• Object-Oriented Language: It supports object-oriented features

and techniques of programming.
• Interactive Programming Language: Users can interact with the python
interpreter directly for writing programs.

• Easy language: Python is easy to learn language especially for beginners.

• Straight forward Syntax: The formation of python syntax is simple

and straightforward which also makes it popular.
Data Structures in Python
LISTS-
 Ordered collection of data.
 Supports similar slicing and indexing functionalities as in the case of Strings.
 They are mutable.
 Advantage of a list over a conventional array
• Lists have no size or type constraints(no setting restrictions
beforehand).
• They can contain different object types.
• We can delete elements from a list by using Del list_name[index_val]
 Example-
• my_list = ['one', 'two','three',4,5]
• len(my_list) would output 5.

Dictionary-
 Lists are sequences but the dictionaries are mappings.
 They are mappings between a unique key and a value pair.
 These mappings may not retain order.
 Constructing a dictionary.
 Accessing object from a dictionary.
 Nesting Dictionaries.
 Basic Dictionary Methods.
 Basic Syntax
o d={} empty dictionary will be generated and assign keys and values to
it, like d[‘animal’] = ‘Dog’
o d = {'K1':'V1', 'K2’:’V2'}
o d['K1'] outputs 'V1‘
Tuples-
 Immutable in nature, i.e they cannot be changed.
 No type restriction
 Indexing and slicing, everything's same like that in strings and lists.
 Constructing tuples.
 Basic tuple methods.
 Immutability.
 When to use tuples?
 We can use tuples to present things that shouldn’t change, such as days
of the week, or dates on a calendar, etc.

Sets-

 A set contains unique and unordered elements and we can construct them
by using a set() function.
 Convert a list into Set-
 l=[1,2,3,4,1,1,2,3,6,7]
 k = set(l)
 k becomes {1,2,3,4,6,7}
 Basic Syntax-
 x=set()
 x.add(1)
 x = {1}
 x.add(1)
 This would make no change in x now
Regular expression in Python

A RegEx, or Regular Expression, is a sequence of characters that forms a search pattern.

RegEx can be used to check if a string contains the specified search pattern.

RegEx Module
Python has a built-in package called re, which can be used to work with Regular Expressions.

Import the re module:

import re

RegEx Functions
The re module offers a set of functions that allows us to search a string for a match:

Function Description

Findall : Returns a list containing all matches

Search : Returns a Match object if there is a match anywhere in the string
Split : Returns a list where the string has been split at each match
Sub : Replaces one or many matches with a string

Metacharacters
Metacharacters are characters with a special meaning:

Character Description Example

[] A set of characters "[a-m]"

\ Signals a special sequence (can also be used "\d"

to escape special characters)

. Any character (except newline character) "he..o"

^ Starts with "^hello"

$ Ends with "world$"

* Zero or more occurrences "aix*"

+ One or more occurrences "aix+"

{} Exactly the specified number of occurrences "al{2}"

| Either or "falls|stays"

() Capture and group

Special Sequences
A special sequence is a \ followed by one of the characters in the list below, and has a special meaning:

Character Description Example

\A Returns a match if the specified characters are at the "\AThe"

beginning of the string

\b Returns a match where the specified characters are at the r"\bain"

beginning or at the end of a word r"ain\b"
(the "r" in the beginning is making sure that the string is
being treated as a "raw string")

\B Returns a match where the specified characters are r"\Bain"

present, but NOT at the beginning (or at the end) of a r"ain\B"
word
(the "r" in the beginning is making sure that the string is
being treated as a "raw string")

\d Returns a match where the string contains digits "\d"

(numbers from 0-9)

\D Returns a match where the string DOES NOT contain "\D"

digits

\s Returns a match where the string contains a white space "\s"

character

\S Returns a match where the string DOES NOT contain a "\S"

white space character

\w Returns a match where the string contains any word "\w"

characters (characters from a to Z, digits from 0-9, and
the underscore _ character)

\W Returns a match where the string DOES NOT contain "\W"

any word characters

\Z Returns a match if the specified characters are at the end "Spain\Z"

of the string

Sets
A set is a set of characters inside a pair of square brackets [] with a special meaning:

Set Description

[arn] Returns a match where one of the specified characters (a, r, or n) are present
 [a-n] Returns a match for any lower case character, alphabetically
between a and n

[^arn] Returns a match for any character EXCEPT a, r, and n

[0123] Returns a match where any of the specified digits (0, 1, 2, or 3) are present

[0-9] Returns a match for any digit between 0 and 9

[0-5][0-9] Returns a match for any two-digit numbers from 00 and 59

[a-zA-Z] Returns a match for any character alphabetically between a and z, lower

case OR upper case

[+] In sets, +, *, ., |, (), $,{} has no special meaning, so [+] means: return a

match for any + character in the string

File Handling in Python-

Python too supports file handling and allows users to handle files i.e., to read and
write files, along with many other file handling options, to operate on files. The
concept of file handling has stretched over various other languages, but the
implementation is either complicated or lengthy, but alike other concepts of
Python, this concept here is also easy and short. Python treats file differently as
text or binary and this is important. Each line of code includes a sequence of
characters and they form text file. Each line of a file is terminated with a special
character, called the EOL or End of Line characters like comma {,} or newline
character. It ends the current line and tells the interpreter a new one has begun.
Let’s start with Reading and Writing files.

We use open () function in Python to open a file in read or write mode. As explained
above, open ( ) will return a file object. To return a file object we use open ()
function along with two arguments, that accepts file name and the mode, whether
to read or write. So, the syntax being: open(filename, mode). There are three kinds
of mode, that Python provides and how files can be opened:

• “ r “, for reading.
• “ w “, for writing.
• “ a “, for appending.
• “ r+ “, for both reading and writing

Ex-It is a notepad file (101.txt)

Code in python

It read the words from 101.txt file and print the all words which are present in
the file and also tell that word occurring howmany times.
Debugging in Python
pdb — The Python Debugger

The module pdb defines an interactive source code debugger for Python programs.

It supports setting (conditional) breakpoints and single stepping at the source line
level, inspection of stack frames, source code listing, and evaluation of arbitrary
Python code in the context of any stack frame. It also supports post-mortem
debugging and can be called under program control.

The debugger is extensible – it is actually defined as the class Pdb. This is

currently undocumented but easily understood by reading the source. The
extension interface uses the modules bdb and cmd.

The debugger’s prompt is (Pdb). Typical usage to run a program under control of
the debugger is:

>>> import pdb

>>> import mymodule
>>> pdb.run('mymodule.test()')
> <string>(0)?()
(Pdb) continue
> <string>(1)?()
(Pdb) continue
NameError: 'spam'
> <string>(1)?()
(Pdb)

Debugger Commands
The commands recognized by the debugger are listed below. Most commands can
be abbreviated to one or two letters as indicated; e.g. h(elp) means that
either h or help can be used to enter the help command (but not he or hel,
nor H or Help or HELP). Arguments to commands must be separated by
whitespace (spaces or tabs). Optional arguments are enclosed in square brackets
([]) in the command syntax; the square brackets must not be typed. Alternatives in
the command syntax are separated by a vertical bar (|).

h(elp) [command]
Without argument, print the list of available commands. With a command as
argument, print help about that command. help pdb displays the full documentation
(the docstring of the pdb module). Since the command argument must be an
identifier, help exec must be entered to get help on the ! command.

w(here)
Print a stack trace, with the most recent frame at the bottom. An arrow indicates
the current frame, which determines the context of most commands.

d(own) [count]
Move the current frame count (default one) levels down in the stack trace (to a
newer frame).

u(p) [count]
Move the current frame count (default one) levels up in the stack trace (to an older
frame).

b(reak) [([filename:]lineno | function) [, condition]]

tbreak [([filename:]lineno | function) [, condition]]

Temporary breakpoint, which is removed automatically when it is first hit. The
arguments are the same as for break.

cl(ear) [filename:lineno | bpnumber ...]

With a filename:lineno argument, clear all the breakpoints at this line. With a space
separated list of breakpoint numbers, clear those breakpoints. Without argument,
clear all breaks (but first ask confirmation).

disable [bpnumber ...]
Disable the breakpoints given as a space separated list of breakpoint numbers.
Disabling a breakpoint means it cannot cause the program to stop execution, but
unlike clearing a breakpoint, it remains in the list of breakpoints and can be
(re-)enabled.

enable [bpnumber ...]
Enable the breakpoints specified.

ignore bpnumber [count]
Set the ignore count for the given breakpoint number. If count is omitted, the
ignore count is set to 0. A breakpoint becomes active when the ignore count is
zero. When non-zero, the count is decremented each time the breakpoint is reached
and the breakpoint is not disabled and any associated condition evaluates to true.

commands [bpnumber]
Specify a list of commands for breakpoint number bpnumber. The commands
themselves appear on the following lines.

s(tep)
Execute the current line, stop at the first possible occasion (either in a function that
is called or on the next line in the current function).

n(ext)
Continue execution until the next line in the current function is reached or it
returns. (The difference between next and step is that step stops inside a called
function, while next executes called functions at (nearly) full speed, only stopping
at the next line in the current function.)

unt(il) [lineno]
Without argument, continue execution until the line with a number greater than the
current one is reached.

With a line number, continue execution until a line with a number greater or equal
to that is reached. In both cases, also stop when the current frame returns.

Changed in version 3.2: Allow giving an explicit line number.

r(eturn)
Continue execution until the current function returns.

c(ont(inue))
Continue execution, only stop when a breakpoint is encountered.

j(ump) lineno
Set the next line that will be executed. Only available in the bottom-most frame.
This lets you jump back and execute code again, or jump forward to skip code that
you don’t want to run.

It should be noted that not all jumps are allowed – for instance it is not possible to
jump into the middle of a for loop or out of a finally clause.
Web Scrapping using Beautiful Soup
Steps involved in web scraping:
• Send an HTTP request to the URL of the webpage you want to access. The
server responds to the request by returning the HTML content of the webpage. For
this task, we will use a third-party HTTP library for python-requests.
• Once we have accessed the HTML content, we are left with the task of parsing
the data. Since most of the HTML data is nested, we cannot extract data simply
through string processing. One needs a parser which can create a nested/tree
structure of the HTML data. There are many HTML parser libraries available but
the most advanced one is html5lib.
• Now, all we need to do is navigating and searching the parse tree that we
created, i.e. tree traversal. For this task, we will be using another third-party
python library, Beautiful Soup. It is a Python library for pulling data out of HTML
and XML files.

Accessing the HTML content from webpage

import requests
URL = "https://www.automatetheboringstuffusingpython.org/"
r = requests.get(URL)
print(r.content)

 First of all import the requests library.

 Then, specify the URL of the webpage you want to scrape.
 Send a HTTP request to the specified URL and save the response from server in
a response object called r.
 Now, as print r.content to get the raw HTML content of the webpage. It is of
‘string’ type.

Parsing the HTML content

import requests
from bs4 import BeautifulSoup
  
URL = "http://www.values.com/inspirational-quotes"
r = requests.get(URL)
   
soup = BeautifulSoup(r.content, 'html5lib')
print(soup.prettify())

A really nice thing about the BeautifulSoup library is that it is built on the top of
the HTML parsing libraries like html5lib, lxml, html.parser, etc. So
BeautifulSoup object and specify the parser library can be created at the same
time.
In the example above,
soup = BeautifulSoup(r.content, 'html5lib')
We create a BeautifulSoup object by passing two arguments:
 r.content : It is the raw HTML content.
 html5lib : Specifying the HTML parser we want to use.

Searching and navigating through the parse tree

Now, we would like to extract some useful data from the HTML content. The
soup object contains all the data in the nested structure which could be
programmatic-ally extracted. In our example, we are scraping a webpage
consisting of some quotes. So, we would like to create a program to save those
quotes (and all relevant information about them).

import requests
from bs4 import BeautifulSoup
import csv
   
URL = "http://www.values.com/inspirational-quotes"
r = requests.get(URL)
   
soup = BeautifulSoup(r.content, 'html5lib')
   
quotes=[]  # a list to store quotes
   
table = soup.find('div', attrs = {'id':'all_quotes'}) 
   
for row in table.findAll('div',
                         attrs = {'class':'col-6 col-lg-3 text-center margin-30px-bottom sm-margin-30px-top'}):
    quote = {}
    quote['theme'] = row.h5.text
     quote['url'] = row.a['href']
    quote['img'] = row.img['src']
    quote['lines'] = row.img['alt'].split(" #")[0]
    quote['author'] = row.img['alt'].split(" #")[1]
    quotes.append(quote)
   
filename = 'inspirational_quotes.csv'
with open(filename, 'w', newline='') as f:
    w = csv.DictWriter(f,['theme','url','img','lines','author'])
    w.writeheader()
    for quote in quotes:
        w.writerow(quote)

Now, in the table element, one can notice that each quote is inside a div container
whose class is quote. So, we iterate through each div container whose class is
quote.
Here, we use findAll() method which is similar to find method in terms of
arguments but it returns a list of all matching elements. Each quote is now iterated
using a variable called row.
Here is one sample row HTML content for better understanding:
Email
Simple Mail Transfer Protocol (SMTP) is a protocol, which handles sending e-
mail and routing e-mail between mail servers.
Python provides smtplib module, which defines an SMTP client session object
that can be used to send mail to any Internet machine with an SMTP or ESMTP
listener daemon.
Here is a simple syntax to create one SMTP object, which can later be used to
send an e-mail −
import smtplib
smtpObj = smtplib.SMTP( [host [, port [, local_hostname]]] )
Here is the detail of the parameters −
 host − This is the host running your SMTP server. You can specify IP address
of the host or a domain name like tutorialspoint.com. This is optional
argument.
 port − If you are providing host argument, then you need to specify a port,
where SMTP server is listening. Usually this port would be 25.
 local_hostname − If your SMTP server is running on your local machine, then
you can specify just localhost as of this option.
An SMTP object has an instance method called sendmail, which is typically used
to do the work of mailing a message. It takes three parameters −
 The sender − A string with the address of the sender.
 The receivers − A list of strings, one for each recipient.
 The message − A message as a string formatted as specified in the various
RFCs.

Sending an HTML e-mail using Python

When you send a text message using Python, then all the content are treated as
simple text. Even if you include HTML tags in a text message, it is displayed as
simple text and HTML tags will not be formatted according to HTML syntax. But
Python provides option to send an HTML message as actual HTML message.
import smtplib

message = """From: From Person <from@fromdomain.com>

To: To Person <to@todomain.com>
MIME-Version: 1.0
Content-type: text/html
Subject: SMTP HTML e-mail test

This is an e-mail message to be sent in HTML format

<b>This is HTML message.</b>

<h1>This is headline.</h1>
"""
try:
smtpObj = smtplib.SMTP('localhost')
smtpObj.sendmail(sender, receivers, message)
print "Successfully sent email"
except SMTPException:
print "Error: unable to send email"
GUI Automation
Controlling mouse movements using pyautogui:

Python tracks and controls mouse using coordinate system of screen. Suppose the
resolution of your screen is 1023×767, then your screen’s coordinate system looks
like to know your screen resolution, type the following in a text editor:

import pyautogui

print(pyautogui.size())

This python code use size() function to output your screen resolution in x,y format:

output: (1023, 768)

To move the mouse, use function moveTo() in pyautogui module. Type:

pyautogui.moveTo(100, 100, duration= 1)

This code uses moveTo() function, which takes x and y coordinates, and an optional
duration argument. This function moves your mouse pointer from it’s current
location to x,y coordinate, and takes time as specified by duration argument to do
so.

Using moveRel() function:

moveRel() function moves the mouse pointer relative to its previous position. Type:

pyautogui.moveRel(0, 50, duration=1)

This code will move mouse pointer at (0, 50) relative to its original position. For
example, if mouse position before running the code was (1000, 1000), then this
code will move the pointer to coordinates (1000, 1050) in duration 1 second.
Getting current mouse position:

use position() function to get current position of the mouse pointer. Type:

print(pyautogui.position())

output: (500, 600)

or wherever your mouse was residing at the time of executing the program.

Clicking and dragging the mouse:

pyautogui.click(100, 100)

This code performs a typical mouse click at the location (100, 100).

We have two functions associated with drag operation of mouse, dragTo and
dragRel. they perform similar to moveTo and moveRel functions, except they hold
the left mouse button while moving, thus initiating a drag. This functionality can be
used at various places, like moving a dialog box, or drawing something
automatically using pencil tool in MS Paint.

Using scroll() function:

scroll function takes no. of pixels as argument, and scrolls the screen up to given
number of pixels.

pyautogui.scroll(200)

Controlling the keyboard:

Automate typing of a string:

You can automate typing of string by using typewrite() function. just pass the string
which you want to type as argument of this function. Ex:

pyautogui.click(100, 100);pyautogui.typewrite(“hello”)
suppose a text field was present at coordinates 100, 100 on screen, then this code
will click the text field to make it active and type “hello” in it.

Passing key names:

You can pass key names separately through typewrite() function. For example:

pyautogui.typewrite([“a”, “left”, “ctrlleft”])

This code is automatic equivalent of typing “a”, pressing left arrow key, and
pressing left control key.

Pressing hotkey combinations:

Use hotkey() function to press combination of keys like ctrl-c, ctrl-a etc. Ex:

pyautogui.hotkey(“ctrlleft”, “a”)

This code is automatic equivalent of pressing left ctrl and “a”simultaneously. Thus
in windows, this will result in selection of all text present on screen.
Conclusions
I believe the trial has shown conclusively that it is both possible and desirable
to use Python as the principal teaching language:

• It is Free (as in both cost and source code).

• It is trivial to install on a Windows PC allowing students to take their
interest further. For many the hurdle of installing a Pascal or C compiler
on a Windows machine is either too expensive or too complicated;
• It is a flexible tool that allows both the teaching of traditional procedural
programming and modern OOP.
• It can be used to teach a large number of transferable skills.
• It is a real-world programming language that can be and is used in
academia and the commercial world.
• It appears to be quicker to learn and, in combination with its many libraries,
this offers the possibility of more rapid student development allowing the
course to be made more challenging and varied.
• And most importantly, its clean syntax offers increased understanding
and enjoyment for students.

The training program having online for whole 2 weeks. In my opinion. I have
gained lots of knowledge and experience needed to be successful in great
engineering challenge as in my opinion, Engineering is after all a Challenge ,and
not a job .