Software Testing

 Software Testing is a process of identifying bugs in an application by running

the application through a series of designed tests.
 Software Testing involves multiple stake holders such as testers,
management, business, customers, consultants, and end users.

Why Software Testing Required?

Software Testing is very important for the following reasons:

 It can detect errors or defects made during development phase.

 It increases reliability and quality of the software.
 It validates or checks if software is behaving as expected based on
requirements.
 It is required for effective performance of software application.

Software Testing Life Cycle

Software Testing Life Cycle (STLC) refers to various phases involved in testing of
a software.
STLC in general includes the following phases:

 Requirements Review
 Test Planning
 Test Designing
 Test Environment Set Up
 Test Execution
 Test Reporting

Levels of Testing

Testing is generally classfied into three levels based on the component being tested.

The three levels are:

 Unit Testing
 Integration Testing
  System Testing

Unit Testing
 Unit Testing : Testing the smallest possible code snippets of a program.

 Practically, breaking down of these code snippets into further smaller pieces is
not achievable.
 For e.g: considering below function as a single unit makes sense.

def square(n):
"Returns square of a number."
return n**2

Integration Testing
 Integration Testing : It focuses on testing interactions across related units.
 These tests are still run in isolation in order to overcome inputs from outside
or other units.
 The inputs required from outside units are simulated.

System Testing
 System Testing : It checks parts of the program once all units are plugged
together.
 It is an extreme form of Integration Testing.
 System tests will not be useful, if these tests are not supported by the results
of integration tests and unit tests.

Acceptance Testing
 Unit Testing, Integration Testing and System Testing are done to verify
software product as a whole or it's individual components.
 On the other hand, the testing that you perform to confirm that the software
behaves as expected is known Acceptance Testing.
 Acceptance Testing doesn't check any core functionality of the software, it
only aims at checking the software behaviour.

Regression Testing
 Regression Testing is another type of testing which ensures that previously
built software performs the same way after making some changes to it or
interfacing it with other software.
 Regression Tests can be written before or after a bug is found.
  They provide an assurance that an increase in a program's complexity doesn't
introduce new bugs.

About the Course

 This course mainly focuses on How to perform Unit Testing of Python
programs or applications.
 The course mainly talks about four major python unit testing modules
namely : doctest, unittest, nose and pytest.
 doctest : It is the simplest framework which combines documentation and
tests.
 unittest : It is the framework having xUnit-Style architecture.
 nose : It extends unittest and makes testing easier.
 pytest : It has a rich set of useful plugins and is compatible
with unittest and nose.

Introduction to doctest
 doctest is a python testing module, which allows writing tests based on
expected output from standard python interpreter along with documentation.
 doctest is also known as "document testing" or "testable document".
 doctest allows combination of tests and documentation. This feature enables
to keep documentation up to date with reality and ensures that tests express
the expected behaviour.

A doctest specifications
 A doctest contains documentation and one or more valid python statements.
 The statements are written after the python shell's primary prompt(>>>).

 Secondary prompt (...) is used from second line on wards, when a statement
is written across multiple lines.

 Expected output is written below the python statements and it should be same
as the result obtained, when you run the statement in a python shell.

Writing Tests as Text Documentation

 Tests are written in the form of documentation in a text file.
 Let's consider the following text in a file named sample_tests.txt. The file
contains text documentation and two tests written for testing python's plus
operator.

===============================
Demonstrating usage of doctest
===============================

1. This doctest checks functionality of python's plus operator.

>>> 2 + 4
6
>>> -10.5 + 8
-2.5

 Presence of a blank line or a new line starting with primary prompt after the
expected output is seen as end of the existing test.

Running Tests
 Run the tests written in sample_tests.txt using the below command.

python3 -m doctest sample_tests.txt

 By default, no output is seen when all tests pass. To view a verbose output,
use -v option as shown in below command.

python3 -m doctest -v sample_tests.txt

Output

Trying:
2 + 4
Expecting:
6
ok
Trying:
-10.5 + 8
Expecting:
-2.5
ok
1 items passed all tests:
2 tests in sample_tests.txt
2 tests in 1 items.
2 passed and 0 failed.
Test passed.

 The verbose output shows that two tests have passed.

Using Secondary Prompt

 Let's extend the sample_tests.txt now, by appending function definition
of add2num function and two tests as shown below.

2. The below doctest checks functionality of add2num function

>>> def add2num(x, y):

... "This function returns sum of two numbers."
... return x + y
>>> add2num(6, 7)
13
>>> add2num(-8.5, 7)
-1.5

 The example shows how a function definition spanning multiple lines is

defined using main prompt (>>>) and secondary prompt(...), in a text document.

Writing Tests in docstrings

 docstring is used for documenting python modules, functions, classes and
methods.
 Now let's see an other example in which tests of function add2num are written
as a part of docstring, in a script named sample_module.py.

def add2num(x, y):

"""Adds two given numbers and returns the sum.
>>> add2num(6, 7)
13
>>> add2num(-8.5, 7)
-1.5
"""
 return x + y

 The docstring contains two tests corresponding to add2num function.

Running Tests in docstrings

 You can now run the tests with the below command

python3 -m doctest sample_module.py

 Since all tests pass, you see an empty output.

 Let's alter the expected output of add2num(-8.5, 7) as -2.5 and run the tests
again, the following output is seen.

**********************************************************************
File "D:\python 3.5\sample_module.py", line 5, in sample_module.add2num
Failed example:
add2num(-8.5, 7)
Expected:
-2.5
Got:
-1.5
**********************************************************************
1 items had failures:
1 of 2 in sample_module.add2num
***Test Failed*** 1 failures.

 One test add2num(-8.5, 7) fails.

Expecting Exceptions
 In some cases you would like to raise an exception, for example when an
invalid input is provided.
 doctest can detect raised exceptions by detecting the python exception
report and traceback displayed in a python interactive shell when the
exception is raised.
 doctest is concerned only with the first line Traceback (most recent call
last): and the last line, which tells it which exception you expect.
 doctest reports a failure only if one of these two parts doesn't match.
Expecting Exceptions
 Now add the below text, at the end of existing two tests of add2num function,
in sample_tests.txt file

>>> add2num(8, 'hello')

Traceback (most recent call last):
TypeError: unsupported operand type(s) for +: 'int' and 'str'

 Run the tests using the below command. All tests are passed.

python3 -m doctest sample_tests.txt

 If you replace TypeError with ValueError word, and run the above command

again it results in one failure case.

Expecting Blanklines
 The doctest considers the first blank line after the primary prompt (>>>) as the
end of expected output.
 However sometimes, you may expect blank lines in the output it self. This
would mislead doctest and fail the tests.
 This issue can be overcome by inserting <BLANKLINE> word instead of an
expected blank line.
 The example in next slide inserts two <BLANKLINE> words in place of two
expected blank lines.

Expecting Blank Lines

 Add the below documentation at the end of sample_tests.txt and run the
doct tests.

3. This doctest checks functionality of insertlines function

>>> def insertlines(n):
... print('Start')
... for i in range(n):
... print()
... print('End')
>>> insertlines(2)
Start
<BLANKLINE>
<BLANKLINE>
End

 The test insertlines(2) function will successfully pass.

Pros and Cons

Pros
 doctest is very simple to use and doesn't require any installation.
 It ensures code documentation, containing interactive examples are up to date.
Cons

 doctest doesn't have a proper API for testing.

 doctests are static in nature and hence cannot be parameterized.
 doctest doesn't support features like test discovery, test fixtures and test
runner. Hence it cannot be used for testing a large project.