Python Scripting for Spatial Data

Processing.

Pete Bunting and Daniel Clewley

Teaching notes on the MScs in Remote Sensing and GIS.

October 27, 2014

Aberystwyth University
Institute of Geography and Earth Sciences.

c Pete Bunting and Daniel Clewley 2013.

Copyright
This work is licensed under the Creative Commons Attribution-ShareAlike 3.0
Unported License. To view a copy of this license, visit http://creativecommons.
org/licenses/by-sa/3.0/.

Acknowledgements
The authors would like to acknowledge to the supports of others but specifically
(and in no particular order) Prof. Richard Lucas, Sam Gillingham (developer of
RIOS and TuiView) and Neil Flood (developer of RIOS) for their support and
time.

ii

Authors
Peter Bunting
Dr Pete Bunting joined the Institute of Geography and Earth Sciences (IGES),
Aberystwyth University, in September 2004 for his Ph.D. where upon completion
in the summer of 2007 he received a lectureship in remote sensing and GIS. Prior
to joining the department, Peter received a BEng(Hons) in software engineering
from the department of Computer Science at Aberystwyth University. Pete also
spent a year working for Landcare Research in New Zealand before rejoining IGES
in 2012 as a senior lecturer in remote sensing.

Contact Details
EMail: pfb@aber.ac.uk
Senior Lecturer in Remote Sensing
Institute of Geography and Earth Sciences
Aberystwyth University
Aberystwyth
Ceredigion
SY23 3DB
United Kingdom

iii

iv

Daniel Clewley
Dr Dan Clewley joined IGES in 2006 undertaking an MSc in Remote Sensing
and GIS, following his MSc Dan undertook a Ph.D. entitled Retrieval of Forest
Biomass and Structure from Radar Data using Backscatter Modelling and Inversion under the supervision of Prof. Lucas, Dr. Bunting and Prof. Mahta Moghaddam. Prior to joining the department Dan completed his BSc(Hons) in Physics
within Aberystwyth University. Dan is currently an Airborne Remote Sensing
Data Analyst at Plymouth Marine Laboratory. He writes a blog on open source
software in GIS and Remote Sensing (http://spectraldifferences.wordpress.
com/)

Contact Details
Email: daniel.clewley@gmail.com

Table of Contents
1 Introduction
1.1

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.1.1

What is Python? . . . . . . . . . . . . . . . . . . . . . . . .

1.1.2

What can it be used for? . . . . . . . . . . . . . . . . . . . .

1.1.3

A word of warning . . . . . . . . . . . . . . . . . . . . . . .

Example of Python in use . . . . . . . . . . . . . . . . . . . . . . .

1.2.1

Software in Python . . . . . . . . . . . . . . . . . . . . . . .

1.3

Python Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.4

Installing Python . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.5

Text Editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.5.1

Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.5.2

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.5.3

Mac OS X . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.5.4

Going between Windows and UNIX . . . . . . . . . . . . . .

Starting Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.6.1

Indentation . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.6.2

Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.6.3

File Naming . . . . . . . . . . . . . . . . . . . . . . . . . . .

1.2

1.6

TABLE OF CONTENTS

1.7

vi

1.6.4

Case Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . .

1.6.5

File paths in examples . . . . . . . . . . . . . . . . . . . . .

1.6.6

Independent Development of Scripts . . . . . . . . . . . . .

1.6.7

Getting Help . . . . . . . . . . . . . . . . . . . . . . . . . .

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 The Basics

10

2.1

Hello World Script . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2

Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3

Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.4

2.5

2.3.1

Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3.2

Boolean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3.3

Text (Strings) . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3.4

Example using Variables . . . . . . . . . . . . . . . . . . . . 14

Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.1

List Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4.2

n-dimensional list . . . . . . . . . . . . . . . . . . . . . . . . 20

IF-ELSE Statements . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.5.1

2.6

Logic Statements . . . . . . . . . . . . . . . . . . . . . . . . 21

Looping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.6.1

while Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.6.2

for Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.7

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.8

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Text Processing

26

TABLE OF CONTENTS

vii

3.1

Read a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.2

Write to a Text File

3.3

Programming Styles . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.4

. . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.3.1

Procedural Programming File Outline . . . . . . . . . . . 30

3.3.2

Object Orientated Programming File Outline . . . . . . . 31

Object Oriented Script . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.4.1

Object Oriented Script for Text File Processing . . . . . . . 32

3.5

Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.6

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4 File System Finding files

41

4.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

4.2

Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4.3

Checking file Extension . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.4

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.5

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5 Plotting - Matplotlib

48

5.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.2

Simple Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.3

Bar Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.4

Pie Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.5

Scatter Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.6

Line Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

5.7

Exercise: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.8

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

TABLE OF CONTENTS

viii

6 Statistics (SciPy / NumPy)

59

6.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

6.2

Simple Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.2.1

6.3

Calculate Biomass . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.3.1

6.4

Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Linear Fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.4.1

6.5

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

7 Batch Processing Command Line Tools

79

7.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

7.2

Merging ESRI Shapefiles . . . . . . . . . . . . . . . . . . . . . . . . 80

7.3

Convert Images to GeoTIFF using GDAL. . . . . . . . . . . . . . . 89

7.3.1

Passing Inputs from the Command Line into your script . . 90

7.4

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

7.5

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

8 Image Processing using GDAL and RIOS

8.1

8.2

93

Reading and Updating Header Information . . . . . . . . . . . . . . 93

8.1.1

Reading Image Headers

. . . . . . . . . . . . . . . . . . . . 93

8.1.2

Read image header example. . . . . . . . . . . . . . . . . . . 95

8.1.3

No Data Values . . . . . . . . . . . . . . . . . . . . . . . . . 97

8.1.4

Band Name . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

8.1.5

GDAL Meta-Data . . . . . . . . . . . . . . . . . . . . . . . . 102

Raster Input / Output Simplification (RIOS) Library . . . . . . . . 107

TABLE OF CONTENTS

ix

8.2.1

Getting Help Reminder . . . . . . . . . . . . . . . . . . . . 107

8.2.2

Band Maths . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

8.2.3

Multiply by a constant . . . . . . . . . . . . . . . . . . . . . 108

8.2.4

Calculate NDVI . . . . . . . . . . . . . . . . . . . . . . . . . 110

8.2.5

Calculate NDVI Using Multiple Images . . . . . . . . . . . . 113

8.3

Filtering Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

8.4

Apply a rule based classification . . . . . . . . . . . . . . . . . . . . 119

8.5

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

8.6

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

9 Raster Attribute Tables (RAT)

124

9.1

Reading Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

9.2

Writing Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

9.2.1

Calculating New Columns . . . . . . . . . . . . . . . . . . . 126

9.2.2

Add Class Name . . . . . . . . . . . . . . . . . . . . . . . . 128

9.3

Adding a colour table . . . . . . . . . . . . . . . . . . . . . . . . . . 130

9.4

Using RATs for rule based classifications. . . . . . . . . . . . . . . . 133

9.4.1

Developing a rule base . . . . . . . . . . . . . . . . . . . . . 133

9.5

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

9.6

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

10 Golden Plover Population Model

145

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

10.2 Model Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
10.3 Reading Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 145
10.4 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

TABLE OF CONTENTS

10.5 Exporting Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

10.6 Creating Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
10.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
10.8 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
A RSGISLib

167

A.1 Introduction to RSGISLib . . . . . . . . . . . . . . . . . . . . . . . 167

A.2 Using RSGISLib . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
A.3 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
A.4 Populating Segments . . . . . . . . . . . . . . . . . . . . . . . . . . 177

List of Figures
5.1

A simple plot using matplotlib. . . . . . . . . . . . . . . . . . . . . 50

5.2

A simple bar chart using matplotlib. . . . . . . . . . . . . . . . . . 51

5.3

A simple pie chart using matplotlib. . . . . . . . . . . . . . . . . . . 53

5.4

A simple scatter plot using matplotlib. . . . . . . . . . . . . . . . . 54

5.5

Rainfall data for summer and winter on the same axis. . . . . . . . 57

5.6

Rainfall data for summer and winter on different axis. . . . . . . . 58

6.1

A simple plot using matplotlib. . . . . . . . . . . . . . . . . . . . . 73

xi

List of Tables
1.1

Keywords within the Python language . . . . . . . . . . . . . . . .

2.1

The mathematical functions available within python. . . . . . . . . 13

2.2

Logic statements available within python . . . . . . . . . . . . . . . 22

3.1

Options when opening a file. . . . . . . . . . . . . . . . . . . . . . . 28

6.1

Coefficients for estimating volume and the specific gravity required

for estimating the biomass by species. . . . . . . . . . . . . . . . . . 64

Chapter 1
Introduction
1.1
1.1.1

Background
What is Python?

Python is a high level scripting language which is interpreted, interactive and

object-oriented. A key attribute of python is its clear and understandable syntax
which should allow you to quickly get up to speed and develop useful application, while the syntax is similar enough to lower level languages, for example
C/C++ and Java, to provide a background from which you can grow your expertise. Python is also a so called memory managed language, meaning that you
the developer are not directly in control of the memory usage within your application, making development much simpler. That is not saying that memory
usage does not need to be considered and you, the developer, cannot influence
the memory footprint of your scripts but these details are out of the scope of this
course. Python is cross-platform with support for Windows, Linux, Mac OS X
and most other UNIX platforms. In addition, many libraries (e.g., purpose built
and external C++ libraries) are available to python and it has become a very popular language for many applications, including on the internet and within remote
sensing and GIS.

CHAPTER 1. INTRODUCTION

1.1.2

What can it be used for?

Python can be used for almost any task from simple file operations and text
manipulation to image processing. It may also be used to extend the functionality
of other, larger applications.

1.1.3

A word of warning

There are number of different versions of python and these are not always compatible. For these worksheets we will be using version 3.X (at the time of writing
the latest version is 3.3.0). With the exception of the quiz in Chapter 2, where
raw_input must be used instead of input, the examples will also work python 2.7.
One of the most noticeable differences between python 2 and python 3 is that the
print statement is now a function. So whilst:
print "Hello World"

will work under python 2, scripts using it wont run under python 3 and must
use:
print("Hello World")

instead. As the second is backwards compatible with python 2 it is good practice

to use this, even if you are working with python 2.
It is possible to have both python 2 and python 3 installed, in this case python 3
will be called using python3, if youre system is set up like this remember to use
python3 wherever you see python in this tutorial.

1.2
1.2.1

Example of Python in use

Software in Python

Many applications have been built in python and a quick search of the web
will reveal the extent of this range. Commonly, applications solely developed
in python are web applications, run from within a web server (e.g., Apache; http:

CHAPTER 1. INTRODUCTION

//httpd.apache.org with http://www.modpython.org) but Desktop applications and data processing software such as TuiView (https://bitbucket.org/
chchrsc/tuiview) and RIOS (https://bitbucket.org/chchrsc/rios) have also
been developed.
In large standalone applications Python is often used to facilitate the development
of plugins or extensions to application. Examples of python used in this form
include ArcMap and SPSS.
For a list of applications supporting or written in python refer to the following
website http://en.wikipedia.org/wiki/Python_software.

1.3

Python Libraries

Many libraries are available to python. Libraries are collections of functions

which can be called from your script(s). Python provides extensive libraries
(http://docs.python.org/lib/lib.html) but third parties have also developed
additional libraries to provide specific functionality (e.g., plotting). A list of available libraries is available from http://wiki.python.org/moin/UsefulModules
and by following the links provides on the page.
The following sites provide links to libraries and packages specific to remote sensing
and GIS, many of which are open source with freely available software packages
and libraries for use with python.
http://freegis.org
http://opensourcegis.org
http://www.osgeo.org

1.4

Installing Python

For this tutorial Python alongside the libraries GDAL (http://www.gdal.org),

numpy (http://www.numpy.org), scipy (http://www.scipy.org), RIOS (https:

CHAPTER 1. INTRODUCTION

//bitbucket.org/chchrsc/rios) and matplotlib (http://matplotlib.sourceforge.

net) are required, built against the version of Python you are using. There are
a number of ways of installing these packages, the recomended option is through the
Anaconda Python Distribution (http://docs.continuum.io/anaconda/install.
html), versions are available for Linux, OS X and Windows. Once Anaconda has
been installed, the additional packages required can be installed using:
conda install -c osgeo \
gdal rios

1.5

Text Editors

To write your Python scripts a text editor is required. A simple text editor such
as Microsofts Notepad will do but it is recommended that you use a syntax aware
editor that will colour, and in some cases format, your code automatically. There
are many text editors available for each operating system and it is up to you
to choose one to use. The recommend editor for this course is Spyder which is
installed with Anaconda. From within Spyder you can directly run your Python
scripts (using the run button), additionally it will alert you to errors within your
scripts before you run them.

1.5.1

Windows

Under Windows, the notepad++ (http://notepad-plus.sourceforge.net) text

editor can also be used. Notepad++ is a free to use open source text editor and
can therefore be downloaded and installed onto any Windows PC. If you use this
editor it is recommended you change the settings for Python to use spaces instead
of tabs using the following steps:
1. Go to Setting Preferences
2. Select Language Menu / Tab Settings
3. Under Tab Settings for python tick Replace by space

CHAPTER 1. INTRODUCTION

1.5.2

Linux

Under Linux either the command line editor ne (nice editor), vim or its graphic
interface equivalent gvim is recommend but kdeveloper, gedit and many others are
also good choices.

1.5.3

Mac OS X

Under Mac OS X either BBEdit, SubEthaEdit or TextMate are recommended,

while the freely available TextWrangler is also a good choice. The command line
editors ne and vi are also available under OS X.

1.5.4

Going between Windows and UNIX

If you are writing your scripts on Windows and transferring them to a UNIX/Linux
machine to be executed (e.g., a High Performance Computing (HPC) environment)
then you need to be careful with the line ending (the invisible symbol defining the
end of a line within a file) as these are different between the various operating
systems. Using notepad++ line ending can be defined as UNIX and this is recommended where scripts are being composed under Windows.
Alternatively, if RSGISLib is installed then the command flip can be used to
convert the line ending, the example below converts to UNIX line endings.
flip -u InputFile.py

1.6

Starting Python

Python may be started by opening a command window and typing:

python

(Alternatively select python(x,y) Command Prompts Python interpreter from

the windows start menu).

CHAPTER 1. INTRODUCTION

This opens python in interactive mode. It is possible to perform some basic maths
try:
>>> 1 + 1
2

To exit type:
>>>exit()

To perform more complex tasks in python often a large number of commands

are required, it is therefore more convenient to create a text file containing the
commands, referred to as a script

1.6.1

Indentation

There are several basic rules and syntax which you need to know to develop scripts
within Python. The first of which is code layout. To provide the structure of the
script Python uses indentation. Indentation can be in the form of tabs or spaces
but which ever is used needs to be consistent throughout the script. The most
common and recommend is to use 4 spaces for each indentation. The example
given below shows an if-else statement where you can see that after the if part
the statement which is executed if the if-statement is true is indented from rest of
the script as with the corresponding else part of the statement. You will see this
indentation as you go through the examples and it is important that you follow
the indentation shown in the examples or your scripts will not execute.
1
2
3
4

if x == 1:
x = x + 1
else:
x = x - 1

1.6.2

Keywords

As with all scripting and programming languages python has a set of keywords,
which have special meanings to the compiler or interpreter when the code is executed. As with all python code, these keywords are case sensitive i.e., else is a

CHAPTER 1. INTRODUCTION

keyword but Else is not. A list of pythons keywords is given below:

Table 1.1: Keywords within the Python language
and
as
assert break
class
continue
def
del
elif
else
exec except
finally
for
from global
if
import
in
is
lambda
not
or
pass
print
raise
return
try
while
with
yield

1.6.3

File Naming

It is important that you use sensible and identifiable names for all the files you
generate throughout these tutorial worksheets otherwise you will not be able to
identify the script at a later date. Additionally, it is highly recommended that you
do not included spaces in file names or in the directory path you use to store the
files generated during this tutorial.

1.6.4

Case Sensitivity

Something else to remember when using python, is that the language is case sensitivity therefore if a name is in lowercase then it needs to remain in lowercase
everywhere it is used.
For example:
VariableName is not the same as variablename

1.6.5

File paths in examples

In the examples provided (in the text) file paths are given as ./PythonCourse/TutorialX/File.xxx.
When writing these scripts out for yourself you will need to update these paths to
the location on your machine where the files are located (e.g., /home/pete.bunting

CHAPTER 1. INTRODUCTION

or C:\). Please note that it is recommended that you do not have any spaces within
your file paths. In the example (answer) scripts provided no file path has been
written and you will therefore need to either save input and output files in the
same directory as the script or provide the path to the file. Please note that under
Windows you need to insert a double slash (i.e., \\) within the file path as a single
slash is an escape character (e.g., \n for new line) within strings.

1.6.6

Independent Development of Scripts

There is a significant step to be made from working your way through notes and
examples, such as those provided in this tutorial, and independently developing
your own scripts from scratch. Our recommendation for this, and when undertaking the exercises from this tutorial, is to take it slowly and think through the steps
you need to undertake to perform the operation(s) you need.
I would commonly first write the script using comments or on paper breaking the
process down into the major steps required. For example, if I were asked to write
a script to uncompress a directory of files into another directory I might write the
following outline, where I use indentation to indicate where a process is part of
the parent:
1

# Get input directory (containing the compressed files)

2
3

# Get output directory (where the files, once uncompressed, will be placed).

4
5

# Retrieve list of all files (to be uncompressed) in the input directory.

6
7
8
9
10

# Iterator through input files, uncompressing each in turn.

# Get single file from list
# create command line command for the current file
# execute command

By writing the process out in this form it makes translating this into python much
simpler as you only need to think of how to do small individual elements in python
and not how to do the whole process in one step.

CHAPTER 1. INTRODUCTION

1.6.7

Getting Help

Python provides a very useful help system through the command line. To get
access to the help run python from the terminal
> python

Then import the library want to get help on

>>> import math

and then run the help tool on the whole module

>>> import math
>>> help(math)

or on individual classes or functions within the module

>>> import osgeo.gdal
>>> help(math.cos)

To exit the help system just press the q key on the keyboard.

1.7

Further Reading

An Introduction to Python, G. van Rossum, F.L. Drake, Jr. Network Theory

ISBN 0-95-416176-9 (Also available online - http://docs.python.org/3/
tutorial/). Chapters 1 3
Python FAQ http://docs.python.org/faq/general.html
Python on Windows http://docs.python.org/faq/windows
How to think Like a Computer Scientist: Python Edition http://www.
greenteapress.com/thinkpython/

Chapter 2
The Basics
2.1

Hello World Script

To create your first python script, create a new text file using your preferred text
editor and enter the text below:
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9

#######################################
# A simple Hello World Script
# Author: <YOUR NAME>
# Emai: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11

print(Hello World)

Save your script to file (e.g., helloworld.py) and then run it either using a command
prompt (Windows) or Terminal (UNIX), using the following command:
> python helloworld.py
Hello World

10

CHAPTER 2. THE BASICS

11

To get a command prompt under Windows type cmd from the run dialog box
in the start menu (Start run), further hints for using the command prompt are
given below. Under OS X, terminal is located in within the Utilities folder in
Applications. If you are using Spyder to create your Python scripts you can run
by clicking the run button.
Hints for using the Windows command line
cd allows you to change directory, e.g.,
cd directory1\directory2
dir allows you to list the contents of a directory, e.g.,
dir
To change drives, type the drive letter followed by a colon, e.g.,
D:
If a file path has spaces, you need to use quote, e.g, to change directory:
cd "Directory with spaces in name\another directory\"

2.2

Comments

In the above script there is a heading detailing the script function, author, and
version. These lines are preceded by a hash (#), this tells the interpreter they
are comments and are not part of the code. Any line starting with a hash is a
comment. Comments are used to annotate the code, all examples in this tutorial
use comments to describe the code. It is recommended you use comments in your
own code.

CHAPTER 2. THE BASICS

2.3

12

Variables

The key building blocks within all programming languages are variables. Variables
allow data to be stored either temperately for use in a single operation or throughout the whole program (global variables). Within python the variable data type
does not need to be specified and will be defined by the first assignment. Therefore,
if the first assignment to a variable is an integer (i.e., whole number) then that
variable will be an integer for the remained of the program. Examples defining
variables are provided below:
name = Pete # String
age = 25 # Integer
height = 6.2 # Float

2.3.1

Numbers

There are three types of numbers within python:

Integers are the most basic form of number, contain only whole numbers where
calculation are automatically rounded to provide whole number answers.
Decimal or floating point numbers provide support for storing all those number
which do not form a whole number.
Complex provide support for complex numbers and are defined as a + bj where
a is the real part and b the imaginary part, e.g., 4.5 + 2.5j or 4.5 2.5j or
4.5 + 2.5j
The syntax for defining variables to store these data types is always the same as
python resolves the suitable type for the variable. Python allows a mathematical
operations to be applied to numbers, listed in Table reftab:maths

2.3.2

Boolean

The boolean data type is the simplest and just stores a true or false value, an
example of the syntax is given below:

CHAPTER 2. THE BASICS

13

Table 2.1: The mathematical functions available within python.

Function
Operation
x+y
x plus y
x-y
x minus y
x*y
x multiplied by y
x/y
x divided by y
x ** y
x to the power of y
int(obj)
convert string to int
long(obj)
convert string to long
float(obj)
convert string to float
complex(obj)
convert string to complex
complex(real, imag) create complex from real and imaginary components
abs(num)
returns absolute value
pow(num1, num2)
raises num1 to num2 power
round(float, ndig=0)
rounds float to ndig places
moveForwards = True
moveBackwards = False

2.3.3

Text (Strings)

To store text the string data type is used. Although not a base data type like
a float or int a string can be used in the same way. The difference lies in the
functions available to manipulate a string are similar to those of an object. A
comprehensive list of functions is available for a string is given in the python
documentation http://docs.python.org/lib/string-methods.html.
To access these functions the string modules needs to be imported as shown in the
example below. Copy this example out and save it as StringExamples.py. When
you run this script observe the change in the printed output and using the python
documentation to identify what each of the functions lstrip(), rstrip() and strip()
do.
1

#! /usr/bin/env python

2
3
4

#######################################
# Example with strings

CHAPTER 2. THE BASICS

5
6
7
8
9

14

# Author: <YOUR NAME>

# Emai: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11

import string

12
13

stringVariable =

Hello World

14
15

print(\ +

stringVariable + \)

16
17
18

stringVariable_lstrip = stringVariable.lstrip()
print(lstrip: \ + stringVariable_lstrip + \)

19
20
21

stringVariable_rstrip = stringVariable.rstrip()
print(rstrip: \ + stringVariable_rstrip + \)

22
23
24

stringVariable_strip = stringVariable.strip()
print(strip: \ + stringVariable_strip + \)

2.3.4

Example using Variables

An example script illustrating the use of variables is provided below. It is recommend you copy this script and execute making sure you understand each line. In
addition, try making the following changes to the script:
1. Adding your own questions.
2. Including the persons name within the questions.
3. Remove the negative marking.
1

#! /usr/bin/env python

2
3
4
5
6

#######################################
# A simple script illustrating the use of
# variables.
# Author: <YOUR NAME>

CHAPTER 2. THE BASICS

7
8
9
10
11
12

# Emai: <YOUR EMAIL>

# Date: DD/MM/YYYY
# Version: 1.1
#
# Added helper function for 2/3 compatibility
#######################################

13
14
15
16
17
18
19
20
21
22
23
24

# Compatibility function for 2/3

import sys
def getInput(question):
""" Python 2/3 helper function
for getting input.
"""
if sys.version[0] == 3:
answer = input(question)
else:
answer = raw_input(question)
return(answer)

25
26

score = 0 # A variable to store the ongoing score

27
28
29
30
31
32

# print is used to print the text to the command line

print(#################################################)
print(Sample Python program which asks the user a few \
simple questions.)
print(################################################)

33
34
35
36

# input is used to retrieve user input from the

# command line
name = getInput(What is your name?\n)

37
38
39
40

print(Hello + name + . You will now be asked a series \

of questions please answer \y\ for YES and \n\ for \
NO unless otherwise stated.)

41
42
43
44
45
46
47

print(Question 1:)
answer = getInput(ALOS PALSAR is a L band spaceborne SAR.\n)
if answer == y: # test whether the value returned was equal to y
print(Well done)
score = score + 1 # Add 1 to the score
else: # if not then the anser must be incorrect

15

CHAPTER 2. THE BASICS

48
49

print(Bad Luck)
score = score - 1 # Remove 1 from the score

50
51
52
53
54
55
56
57
58
59

print(Question 2:)
answer = getInput(CASI provides hyperspectral data in \
the Blue to NIR part of the spectrum.\n)
if answer == y:
print(Well done)
score = score + 1
else:
print(Bad Luck)
score = score - 1

60
61
62
63
64
65
66
67
68
69

print(Question 3:)
answer = getInput(HyMap also only provides data in the \
Blue to NIR part of the spectrum.\n)
if answer == y:
print(Bad Luck)
score = score - 1
else:
print(Well done)
score = score + 1

70
71
72
73
74
75
76
77
78

print(Question 4:)
answer = getInput(Landsat is a spaceborne sensor.\n)
if answer == y:
print(Well done)
score = score + 1
else:
print(Bad Luck)
score = score - 1

79
80
81
82
83
84
85
86
87
88

print(Question 5:)
answer = getInput(ADS-40 is a high resolution aerial \
sensor capturing RGB-NIR wavelengths.\n)
if answer == y:
print(Well done)
score = score + 1
else:
print(Bad Luck)
score = score - 1

16

CHAPTER 2. THE BASICS

89
90
91
92
93
94
95
96
97
98

print(Question 6:)
answer = getInput(eCognition is an object oriented \
image analysis software package.\n)
if answer == y:
print(Well done)
score = score + 1
else:
print(Bad Luck)
score = score - 1

99
100
101
102
103
104
105
106
107
108

print(Question 7:)
answer = getInput(Adobe Photoshop provides the same \
functionality as eCognition.\n)
if answer == y:
print(Bad Luck)
score = score - 1
else:
print(Well done)
score = score + 1

109
110
111
112
113
114
115
116
117
118

print(Question 8:)
answer = getInput(Python can be executed within \
the a java virtual machine.\n)
if answer == y:
print(Well done)
score = score + 1
else:
print(Bad Luck)
score = score - 1

119
120
121
122
123
124
125
126
127
128
129

print(Question 9:)
answer = getInput(Python is a scripting language \
not a programming language.\n)
if answer == y:
print(Well done)
score = score + 1
else:
print(Bad Luck)
score = score - 1

17

CHAPTER 2. THE BASICS

130
131
132
133
134
135
136
137

18

print(Question 10:)
answer = getInput(Aberystwyth is within Mid Wales.\n)
if answer == y:
print(Well done)
score = score + 1
else:
print(Bad Luck)
score = score - 1

138
139
140

# Finally print out the users final score.

print(name + you got a score of + str(score))

Note, due to differences between Python 2 and Python 3 a function is defined

called getInput, which determines uses the appropriate function depending on
the version of python being used with input used for Python 3 and raw_input
used for Python 2.

2.4

Lists

Each of the data types outlined above only store a single value at anyone time, to
store multiple values in a single variable a sequence data type is required. Python
offers the List class, which allows any data type to be stored in a sequence and even
supports the storage of objects of different types within one list. The string data
type is a sequence data type and therefore the same operations are available.
List are very flexible structures and support a number of ways to create, append
and remove content from the list, as shown below. Items in the list are numbered
consecutively from 0-n, where n is one less than the length of the list.
Additional functions are available for List data types (e.g., len(aList), aList.sort(),
aList.reverse()) and these are described in http://docs.python.org/lib/typesseq.
html and http://docs.python.org/lib/typesseq-mutable.html.

2.4.1

List Examples

CHAPTER 2. THE BASICS

#! /usr/bin/env python

2
3
4
5
6
7
8
9

#######################################
# Example with lists
# Author: <YOUR NAME>
# Emai: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11
12
13
14

# Create List:
aList = list()
anotherList = [1, 2, 3, 4]
emptyList = []

15
16
17
18

print(aList)
print(anotherList)
print(emptyList)

19
20
21
22
23

# Adding data into a List

aList.append(Pete)
aList.append(Dan)
print(aList)

24
25
26
27
28

# Updating data in the List

anotherList[2] = three
anotherList[0] = one
print(anotherList)

29
30
31
32
33

# Accessing data in the List

print(aList[0])
print(anotherList[0:2])
print(anotherList[2:3])

34
35
36
37

# Removing data from the List

del anotherList[1]
print(anotherList)

38
39
40

aList.remove(Pete)
print(aList)

19

CHAPTER 2. THE BASICS

2.4.2

20

n-dimensional list

Additionally, n-dimensional lists can be created by inserting lists into a list, a

simple example of a 2-d structure is given below. This type of structure can be
used to store images (e.g., the example given below would form a grey scale image)
and additions list dimensions could be added for additional image bands.
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9

#######################################
# Example with n-lists
# Author: <YOUR NAME>
# Emai: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# Create List:
aList = [
[1,1,1,1,1,1,1,1,1,1,1,1,1,1],
[1,1,0,0,1,1,1,1,1,0,0,1,1,1],
[1,1,0,0,1,1,1,1,1,0,0,1,1,1],
[1,1,1,1,1,1,1,1,1,1,1,1,1,1],
[1,1,1,1,1,1,0,1,1,1,1,1,1,1],
[1,1,1,1,1,1,0,1,1,1,1,1,1,1],
[1,1,1,1,1,0,0,0,1,1,1,1,1,1],
[1,0,1,1,1,1,1,1,1,1,1,1,0,1],
[1,0,1,1,1,1,1,1,1,1,1,1,0,1],
[1,1,0,0,0,0,0,0,0,0,0,0,1,1],
[1,1,1,1,1,1,1,1,1,1,1,1,1,1]
]

25
26

print(aList)

CHAPTER 2. THE BASICS

2.5

21

IF-ELSE Statements

As already illustrated in the earlier quiz example the ability to make a decision is
key to any software. The basic construct for decision making in most programming
and scripting languages are if-else statements. Python uses the following syntax
for if-else statements.
if <logic statement>:
do this if true
else:
do this
if <logic statement>:
do this if true
elif <logic statement>:
do this if true
elif <logic statement>:
do this if true
else
do this

Logic statements result in a true or false value being returned where if a value of
true is returned the contents of the if statement will be executed and remaining
parts of the statement will be ignored. If a false value is returned then the if part
of the statement will be ignored and the next logic statement will be analysis until
either one returns a true value or an else statement is reached.

2.5.1

Logic Statements

Table 2.2 outlines the main logic statements used within python in addition to
these statements functions which return a boolean value can also be used to for
decision making, although these will be described in later worksheets.

CHAPTER 2. THE BASICS

22

Table 2.2: Logic statements available within python

Function
Operation
Example
==
equals
expr1 == expr2
>
greater than
expr1 > expr2
<
less than
expr1 < expr2
>=
greater than and equal to expr1 expr2
<=
less than and equal to
expr1 expr2
not
logical not
not expr
and
logical and
expr1 and expr2
or
logical or
expr1 or expr2
is
is the same object
expr1 is expr2

2.6

Looping

In addition to the if-else statements for decision making loops provide another key
component to writing any program or script. Python offers two forms of loops,
while and for. Each can be used interchangeably given the developers preference
and available information. Both types are outlined below.

2.6.1

while Loop

The basic syntax of the while loop is very simple (shown below) where a logic
statement is used to terminate the loop, when false is returned.
while <logic statement> :
statements

Therefore, during the loop a variable in the logic statement needs to be altered
allowing the loop to terminate. Below provides an example of a while loop to
count from 0 to 10.
1

#! /usr/bin/env python

2
3
4
5
6
7

#######################################
# A simple example of a while loop
# Author: <YOUR NAME>
# Emai: <YOUR EMAIL>
# Date: DD/MM/YYYY

CHAPTER 2. THE BASICS

8
9

23

# Version: 1.0
#######################################

10
11
12
13
14

count = 0
while count <= 10:
print(count)
count = count + 1

2.6.2

for Loop

A for loop provides similar functionality to that of a while loop but it provides the
counter for termination. The syntax of the for loop is provided below:
1
2

for <iter_variable> in <iterable>:

statements

The common application of a for loop is for the iteration of a list and an example
if this is given below:
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9

#######################################
# A simple example of a for loop
# Author: <YOUR NAME>
# Emai: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11

aList = [Pete, Richard, Johanna, Philippa, Sam, Dan, Alex]

12
13
14

for name in aList:

print(Current name is: + name)

A more advance example is given below where two for loops are used to iterate
through a list of lists.
1
2

#! /usr/bin/env python

CHAPTER 2. THE BASICS

3
4
5
6
7
8
9

24

#######################################
# Example with for loop and n-lists
# Author: <YOUR NAME>
# Emai: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# Create List:
aList = [
[1,1,1,1,1,1,1,1,1,1,1,1,1,1],
[1,1,0,0,1,1,1,1,1,0,0,1,1,1],
[1,1,0,0,1,1,1,1,1,0,0,1,1,1],
[1,1,1,1,1,1,1,1,1,1,1,1,1,1],
[1,1,1,1,1,1,0,1,1,1,1,1,1,1],
[1,1,1,1,1,1,0,1,1,1,1,1,1,1],
[1,1,1,1,1,0,0,0,1,1,1,1,1,1],
[1,0,1,1,1,1,1,1,1,1,1,1,0,1],
[1,0,1,1,1,1,1,1,1,1,1,1,0,1],
[1,1,0,0,0,0,0,0,0,0,0,0,1,1],
[1,1,1,1,1,1,1,1,1,1,1,1,1,1]
]

25
26
27
28
29
30
31

for cList in aList:

for number in cList:
# Print with a space at the end
# rather than a new line
print (number,end=" ")
print()

2.7

Exercises

During this tutorial you should have followed through each of the examples and
experimented with the code to understand each of components outlined. To test
your understanding of all the material, you will now be asked to complete a series
of tasks:
1. Update the quiz so the questions and answers are stored in lists which are

CHAPTER 2. THE BASICS

25

iterated through as the script is executed.

2. Create a script that loops through the smiling face 2-d list of lists flipping it
so the face is up side down.

2.8

Further Reading

An Introduction to Python, G. van Rossum, F.L. Drake, Jr. Network Theory

ISBN 0-95-416176-9 (Also available online - http://docs.python.org/3/
tutorial/) - Chapters 4 and 5.
Spyder Documentation http://packages.python.org/spyder/
Python Documentation http://www.python.org/doc/
Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7
How to think Like a Computer Scientist: Python Edition http://www.
greenteapress.com/thinkpython/
Learn UNIX in 10 minutes http://freeengineer.org/learnUNIXin10minutes.
html (Optional, but recommended if running on OS X / Linux)

Chapter 3
Text Processing
3.1

Read a Text File

An example of a script to read a text file is given below, copy this example out
and use the numbers.txt file to test your script. Note, that the numbers.txt file
needs to be within the same directory as your python script.
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10
11
12

#######################################
# A simple example reading in a text file
# two versions of the script are provided
# to illustrate that there is not just one
# correct solution to a problem.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

13
14

import string

15
16
17
18

# 1) Splits the text file into individual characters

# to identify the commas and parsing the individual
# tokens.

26

CHAPTER 3. TEXT PROCESSING

19
20

27

numbers = list()
dataFile = open(numbers.txt, r)

21
22
23
24
25
26
27
28
29
30
31
32
33

for eachLine in dataFile:

#print(eachLine)
tmpStr =
for char in eachLine:
#print(char)
if char.isdigit():
tmpStr += char
elif char == , and tmpStr != :
numbers.append(int(tmpStr))
tmpStr =
if tmpStr.isdigit():
numbers.append(int(tmpStr))

34
35
36

print(numbers)
dataFile.close()

37
38
39
40
41

# 2) Uses the string function split to line from the file

# into a list of substrings
numbers = list()
dataFile = open(numbers.txt, r)

42
43
44
45
46
47
48
49

for eachLine in dataFile:

#print eachLine
substrs = eachLine.split(,,eachLine.count(,))
#print substrs
for strVar in substrs:
if strVar.isdigit():
numbers.append(int(strVar))

50
51
52

print(numbers)
dataFile.close()

As you can see reading a text file from within python is a simple process1 . The
first step is to open the file for reading, option r is used as the file is only going to
be read, the other options are available in Table 3.1. If the file is a text file then
1

If your data are in tabular format (e.g., CSV) the csv module in the Python Standard Library
and the genfromtxt from NumPy provide even simpler ways of reading data.

CHAPTER 3. TEXT PROCESSING

28

the contents can then be read a line at a time, if a binary file (e.g., tiff or doc)
then reading is more complicated and not covered in this tutorial.
Table 3.1: Options when opening a file.
File Mode
Operations
r
Open for read
w
Open for write (truncate)
a
Open for write (append)
r+
Open for read/write
w+
Open for read/write (truncate)
a+
Open for read/write (append)
rb
Open for binary read
wb
Open for binary write (truncate)
ab
Open for binary write (append)
rb+
Open for read/write
wb+
Open for read/write (truncate)
ab+
Open for read/write (append)
Now your need to adapt the one of the methods given in the script above to allow
numbers and words to be split into separate lists. To do this you will need to use
the isalpha() function alongside the isdigit() function. Adapt the numbers.txt file
to match the input shown below and then run your script and you should receive
the output shown below:

Input:
1,
2,pete,
3,
4,dan,5,
6,7,8,richard,10,11,12,13

Output:
>python simplereadsplit.py
[1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13]
[pete, dan, richard]

CHAPTER 3. TEXT PROCESSING

3.2

29

Write to a Text File

Writing to a text file is similar to reading from the file. When opening the file
two choices are available either to append or truncate the file. Appending to the
file leaves any content already within the file untouched while truncating the file
removes any content already within the file. An example of writing a list to a file
with each list item on a new line is given below.
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# A simple script parsing numbers of
# words from a comma seperated text file
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12
13
14

aList = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10,

one, two, three, four, five,
six, seven, eight, nine, ten]

15
16

dataFile = open(writetest.txt, w)

17
18
19

for eachitem in aList:

dataFile.write(str(eachitem)+\n)

20
21

dataFile.close()

3.3

Programming Styles

There are two main programming styles, both of which are supported by python,
and these are procedural and object oriented programming. Procedural programming preceded object oriented programming and procedural scripts provide lists
of commands which are run through sequentially.

CHAPTER 3. TEXT PROCESSING

30

Object oriented programming differs from procedural programming in that the

program is split into a series of objects, usually representing really world objects
or functionality, generally referred to as a class. Objects support the concepts
of inheritance where functionality can be used in many sub-objects. For example,
a Person class maybe written with functions such as eat, drink, beat heart etc.
and specialist sub-objects may then be created with Person as a super-object, for
example child, adult, male and female. These objects all require the functionality
of Person but it is inefficient to duplicate the functionality they share individual
rather then group this functionality into the Person class.
This course will concentrate on basic object oriented programming but below are
the basic python file outlines for both procedural and object oriented scripts.

3.3.1

Procedural Programming File Outline

When creating a procedural python script each of your files will have the same
basic format outlined below:
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9

#######################################
# Comment explaining scripts purpose
# Author: <Author Name>
# Email: <Authors Email>
# Date: <Date Last Editor>
# Version: <Version Number>
#######################################

10
11
12

# IMPORTS
# e.g., import os

13
14
15

# SCRIPT
print("Hello World")

16
17

# End of File

CHAPTER 3. TEXT PROCESSING

3.3.2

31

Object Orientated Programming File Outline

When creating an object oriented script each python file you create will have the
same basic format outlined below:
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9

#######################################
# Comment explaining scripts purpose
# Author: <Author Name>
# Emai: <Authors Email>
# Date: <Date Last Editor>
# Version: <Version Number>
#######################################

10
11
12

# IMPORTS
import os

13
14
15

# CLASS EXPRESSION - In this case class name is Person

class Person (object): # Object is the superclass

16
17
18

# CLASS ATTRIBUTES
name =

19
20
21
22

# INITIALISE THE CLASS (OFTEN EMPTY)

def __init__(self):
self.name = Dan

23
24
25
26

# METHOD TO PRINT PERSON NAME

def printName(self):
print(Name: + self.name)

27
28
29
30

# METHOD TO SET PERSON NAME

def setName(self, inputName):
self.name = inputName

31
32
33
34
35

# METHOD TO GET PERSON NAME

def getName(self):
return self.name

CHAPTER 3. TEXT PROCESSING

36
37
38
39
40

32

# METHOD TO EXECUTE CLASS

def run(self):
self.printName()
self.setName(Pete)
self.printName()

41
42
43
44
45

# IF EXECUTED FROM THE COMMAND LINE

if __name__ == __main__:
obj = Person()
obj.run()

46
47

# End of File

3.4

Object Oriented Script

For simple scripts like those demonstrated so far simple procedural scripts are all
that have been required. When creating more complex scripts the introduction
of more structured and reusable designs are preferable. To support this design
Python supports object oriented program design.

3.4.1

Object Oriented Script for Text File Processing

To illustrate the difference in implementation an example is given and explained

below. The example reads a comma separated text file (randfloats.txt) of random
floating point numbers from which the mean and standard deviation is calculated.
Create a new python script and copy the script below:
1

#! /usr/bin/env python

2
3
4
5
6
7
8

#######################################
# An python class to parse a comma
# separates text file to calculate
# the mean and standard deviation
# of the inputted floating point
# numbers.

CHAPTER 3. TEXT PROCESSING

9
10
11
12
13

# Author: <YOUR NAME>

# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

14
15
16

# import the squareroot function from python math

from math import sqrt

17
18
19

# Define a new class called CalcMeanStdDev

class CalcMeanStdDev (object):

20
21
22
23
24
25
26
27
28
29
30
31
32
33

# Define a function which parses a comma

# separated file - you should understand
# the contents of this script from the
# previous examples.
# Note: the file is passed into the
# function.
def parseCommaFile(self, file):
floatingNumbers = list()
for eachLine in file:
substrs = eachLine.split(,,eachLine.count(,))
for strVar in substrs:
floatingNumbers.append(float(strVar))
return floatingNumbers

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

# Define a function to calculate the mean

# value from a list of numbers.
# Note. The list of numbers is passed into
# the function.
def calcMean(self, numbers):
# A variable to sum all the numbers
sum = 0.0
# Iterate through the numbers list
for number in numbers:
# add each number to the sum
sum += number
# Divide the sum by the number of
# values within the numbers list
# (i.e., its length)
mean = sum/len(numbers)

33

CHAPTER 3. TEXT PROCESSING

50
51

# return the mean value calculated

return mean

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

# Define a function which calculates the

# standard deviation of a list of numbers
# Note. The list of numbers is passed into
# the function alongside a previously
# calculated mean value for the list.
def calcStdDev(self, numbers, mean):
# Varible for total deviation
deviation = 0.0
# Variable for a single deviation
singleDev = 0.0
# Iterate through the list of numbers.
for number in numbers:
# Calculate a single Deviation
singleDev = number-mean
# Add the squared single deviation to
# to the on going total.
deviation += (singleDev**2)
# Calcate the standard devaition
stddev = sqrt(deviation/(len(numbers)-1))
# return the standard deviation
return stddev

74
75
76
77
78
79
80
81
82
83

# The main thread of processing. A function

# which defines the order of processing.
# Note. The filename is passed in.
def run(self, filename):
# Open the input file
inFile = open(filename, r)
# Parse the file to retrieve a list of
# numbers
numbers = self.parseCommaFile(inFile)

84
85
86
87
88
89
90

# Calculate the mean value of the list

mean = self.calcMean(numbers)
# Calculate the standard deviation of the
# list.
stddev = self.calcStdDev(numbers, mean)

34

CHAPTER 3. TEXT PROCESSING

35

# Print the results to screen

print(Mean: + str(mean))
print(Stddev: + str(stddev))

91
92
93
94

# Close the input file

inFile.close()

95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110

# When python is executed python executes

# the code with the lowest indentation first.
#
# We can identify when python is executed from
# the command line using the following if statment.
#
# When executed we want the run() function to be
# executed therefore we create a CalcMeanStdDev
# object and call run on that object - passing
# in the file name of the file to be processed.
if __name__ == __main__:
obj = CalcMeanStdDev()
obj.run(randfloats.txt) # Update with full file path.

NOTE:
__name__
and
__main__
each have TWO underscores either side (i.e.,

).

Although, an object oriented design has been introduced making the above code,
potentially, more reusable the design does not separate more general functionality
from the application. To do this the code will be split into two files the first, named
MyMaths.py, will contain the mathematical operations calcMean and calcStdDev
while the second, named FileSummary, contains the functions run, which controls
the flow of the script, and parseCommaFile(). The code for these files is given
below but first try and split the code into the two files yourself.
1
2

#! /usr/bin/env python

CHAPTER 3. TEXT PROCESSING

3
4
5
6
7
8
9

#######################################
# An python class to hold maths operations
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11

from math import sqrt

12
13

class MyMathsClass (object):

14
15
16
17
18
19
20

def calcMean(self, numbers):

sum = 0.0
for number in numbers:
sum += number
mean = sum/len(numbers)
return mean

21
22
23
24
25
26
27
28
29

def calcStdDev(self, numbers, mean):

deviation = 0.0
singleDev = 0.0
for number in numbers:
singleDev = number-mean
deviation += (singleDev**2)
stddev = sqrt(deviation/(len(numbers)-1))
return stddev

30
31

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# An python class to parse a comma
# separates text file to calculate
# the mean and standard deviation
# of the inputted floating point
# numbers.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>

36

CHAPTER 3. TEXT PROCESSING

11
12
13

37

# Date: DD/MM/YYYY
# Version: 1.0
#######################################

14
15
16
17
18
19
20

# To import the class you have created

# you need to define the file within
# which the class is held. In this
# case MyMaths.py and the name of the
# class to be imported (i.e., MyMathsClass)
from MyMaths import MyMathsClass

21
22

class FileSummary (object):

23
24
25
26
27
28
29
30

def parseCommaFile(self, file):

floatingNumbers = list()
for eachLine in file:
substrs = eachLine.split(,,eachLine.count(,))
for strVar in substrs:
floatingNumbers.append(float(strVar))
return floatingNumbers

31
32
33
34

def run(self, filename):

inFile = open(filename, r)
numbers = self.parseCommaFile(inFile)

35
36
37
38

mathsObj = MyMathsClass()
mean = mathsObj.calcMean(numbers)
stddev = mathsObj.calcStdDev(numbers, mean)

39
40
41

print(Mean: + str(mean))
print(Stddev: + str(stddev))

42
43
44
45
46

if __name__ == __main__:
obj = FileSummary()
obj.run(randfloats.txt)

47

To allow the script to be used as a command line tool the path to the file needs
be passed into the script at runtime therefore the following changes are made to

CHAPTER 3. TEXT PROCESSING

the FileSummary script:
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10
11
12
13

#######################################
# An python class to parse a comma
# separates text file to calculate
# the mean and standard deviation
# of the inputted floating point
# numbers.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

14
15
16
17
18
19

from MyMaths import MyMathsClass

# To allow command line options to be
# retrieved the sys python library needs
# to be imported
import sys

20
21

class FileSummary (object):

22
23
24
25
26
27
28
29

def parseCommaFile(self, file):

floatingNumbers = list()
for eachLine in file:
substrs = eachLine.split(,,eachLine.count(,))
for strVar in substrs:
floatingNumbers.append(float(strVar))
return floatingNumbers

30
31
32
33
34
35
36
37
38
39

def run(self):
# To retrieve the command line arguments
# the sys.argv[X] is used where X refers to
# the argument. The argument number starts
# at 1 and is the index of a list.
filename = sys.argv[1]
inFile = open(filename, r)
numbers = self.parseCommaFile(inFile)

38

CHAPTER 3. TEXT PROCESSING

39

mathsObj = MyMathsClass()
mean = mathsObj.calcMean(numbers)
stddev = mathsObj.calcStdDev(numbers, mean)

40
41
42
43

print(Mean: + str(mean))
print(Stddev: + str(stddev))

44
45
46
47
48
49

if __name__ == __main__:
obj = FileSummary()
obj.run()

50

To read the new script the following command needs to be run from the command
prompt:
python fileSummary_commandline.py randfloats.txt

3.5

Exercise

Calculate the mean and standard deviation from only the first column of data

Hint:
You will need to replace:
substrs = eachLine.split(,,eachLine.count(,))
for strVar in substrs:
floatingNumbers.append(float(strVar))

With:
substrs = eachLine.split(,,eachLine.count(,))
# Select the column the data is stored in
column1 = substrs[0]
floatingNumbers.append(float(column1))

CHAPTER 3. TEXT PROCESSING

3.6

40

Further Reading

An Introduction to Python, G. van Rossum, F.L. Drake, Jr. Network Theory

ISBN 0-95-416176-9 (Also available online - http://docs.python.org/3/
tutorial/) - Chapter 7.
Python Documentation http://www.python.org/doc/
Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7

Chapter 4
File System Finding files
4.1

Introduction

A common task for which python is used is to batch process a task or series of
tasks. To do this the files to be processed need to be identified from within the file
system. Therefore, in this tutorial you will learn to implement code to undertake
this operation.
To start this type out the code below into a new file (save it as IterateFiles.py).
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10
11

#######################################
# A class that iterates through a directory
# or directory structure and prints out theatre
# identified files.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

12
13
14

import os.path
import sys

15

41

CHAPTER 4. FILE SYSTEM FINDING FILES

16

42

class IterateFiles (object):

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

# A function which iterates through the directory

def findFiles(self, directory):
# check whether the current directory exits
if os.path.exists(directory):
# check whether the given directory is a directory
if os.path.isdir(directory):
# list all the files within the directory
dirFileList = os.listdir(directory)
# Loop through the individual files within the directory
for filename in dirFileList:
# Check whether file is directory or file
if(os.path.isdir(os.path.join(directory,filename))):
print(os.path.join(directory,filename) + \
is a directory and therefore ignored!)
elif(os.path.isfile(os.path.join(directory,filename))):
print(os.path.join(directory,filename))
else:
print(filename + is NOT a file or directory!)
else:
print(directory + is not a directory!)
else:
print(directory + does not exist!)

40
41
42
43
44
45
46

def run(self):
# Set the folder to search
searchFolder = ./PythonCourse # Update path...
self.findFiles(searchFolder)

47
48
49
50
51

if __name__ == __main__:
obj = IterateFiles()
obj.run()

Using the online python documentation read through the section on the file system:
http://docs.python.org/library/filesys.html

CHAPTER 4. FILE SYSTEM FINDING FILES

43

http://docs.python.org/library/os.path.html
This documentation will allow you to understand the functionality which is available for manipulating the file system.

4.2

Recursion

The next stage is to add allow the function recursively go through the directory
structure. To do this add the function below to your script above:
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10
11

#######################################
# A class that iterates through a directory
# or directory structure and prints out theatre
# identified files.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

12
13
14

import os.path
import sys

15
16

class IterateFiles (object):

17
18
19
20
21
22
23
24
25
26
27
28

# A function which iterates through the directory

def findFilesRecurse(self, directory):
# check whether the current directory exits
if os.path.exists(directory):
# check whether the given directory is a directory
if os.path.isdir(directory):
# list all the files within the directory
dirFileList = os.listdir(directory)
# Loop through the individual files within the directory
for filename in dirFileList:
# Check whether file is directory or file

CHAPTER 4. FILE SYSTEM FINDING FILES

44

if(os.path.isdir(os.path.join(directory,filename))):
# If a directory is found recall this function.
self.findFilesRecurse(os.path.join(directory,filename))
elif(os.path.isfile(os.path.join(directory,filename))):
print(os.path.join(directory,filename))
else:
print(filename + is NOT a file or directory!)

29
30
31
32
33
34
35

else:
print(directory + is not a directory!)
else:
print(directory + does not exist!)

36
37
38
39
40
41
42
43
44

def run(self):
# Set the folder to search
searchFolder = ./PythonCourse # Update path...
self.findFilesRecurse(searchFolder)

45
46
47
48

if __name__ == __main__:
obj = IterateFiles()
obj.run()

Now call this function instead of the findFiles. Think and observe what effect a
function which calls itself will have on the order in which the file are found.

4.3

Checking file Extension

The next step is to include the function checkFileExtension to your class and
create two new functions which only print out the files with the file extension of
interest. This should be done for both the recursive and non-recursive functions
above.
1

#! /usr/bin/env python

2
3
4
5
6

#######################################
# A class that iterates through a directory
# or directory structure and prints out theatre
# identified files.

CHAPTER 4. FILE SYSTEM FINDING FILES

7
8
9
10
11

45

# Author: <YOUR NAME>

# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

12
13
14

import os.path
import sys

15
16

class IterateFiles (object):

17
18
19
20
21
22
23
24
25
26
27
28
29
30

# A function which checks a file extension and returns

def checkFileExtension(self, filename, extension):
# Boolean variable to be returned by the function
foundExtension = False;
# Split the filename into two parts (name + ext)
filenamesplit = os.path.splitext(filename)
# Get the file extension into a varaiable
fileExtension = filenamesplit[1].strip()
# Decide whether extensions are equal
if(fileExtension == extension):
foundExtension = True
# Return result
return foundExtension

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

# A function which iterates through the directory and checks file extensions
def findFilesExtRecurse(self, directory, extension):
# check whether the current directory exits
if os.path.exists(directory):
# check whether the given directory is a directory
if os.path.isdir(directory):
# list all the files within the directory
dirFileList = os.listdir(directory)
# Loop through the individual files within the directory
for filename in dirFileList:
# Check whether file is directory or file
if(os.path.isdir(os.path.join(directory,filename))):
# If a directory is found recall this function.
self.findFilesRecurse(os.path.join(directory,filename))
elif(os.path.isfile(os.path.join(directory,filename))):
if(self.checkFileExtension(filename, extension)):

CHAPTER 4. FILE SYSTEM FINDING FILES

48
49
50
51
52
53
54

46

print(os.path.join(directory,filename))
else:
print(filename + is NOT a file or directory!)
else:
print(directory + is not a directory!)
else:
print(directory + does not exist!)

55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

# A function which iterates through the directory and checks file extensions
def findFilesExt(self, directory, extension):
# check whether the current directory exits
if os.path.exists(directory):
# check whether the given directory is a directory
if os.path.isdir(directory):
# list all the files within the directory
dirFileList = os.listdir(directory)
# Loop through the individual files within the directory
for filename in dirFileList:
# Check whether file is directory or file
if(os.path.isdir(os.path.join(directory,filename))):
print(os.path.join(directory,filename) + \
is a directory and therefore ignored!)
elif(os.path.isfile(os.path.join(directory,filename))):
if(self.checkFileExtension(filename, extension)):
print(os.path.join(directory,filename))
else:
print(filename + is NOT a file or directory!)
else:
print(directory + is not a directory!)
else:
print(directory + does not exist!)

80
81
82
83
84

def run(self):
# Set the folder to search
searchFolder = ./PythonCourse # Update path...
self.findFilesExt(searchFolder, .txt)

85
86
87
88

if __name__ == __main__:
obj = IterateFiles()
obj.run()

CHAPTER 4. FILE SYSTEM FINDING FILES

4.4

47

Exercises

1. Rather than print the file paths to screen add them to a list and return them
from the function. This would be useful for applications where the files to
be process need to be known up front and creates a more generic piece of
python which can be called from other scripts.
2. Using the return list add code to loop through the returned list and print
out the file information in the following comma separated format.
[FILE NAME], [EXTENSION], [PATH], [DRIVE LETTER (On Windows)], [MODIFICATION TIME]

4.5

Further Reading

Python Documentation http://www.python.org/doc/

Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7

Chapter 5
Plotting - Matplotlib
5.1

Introduction

Many open source libraries are available from within python. These significantly
increase the available functionality, decreasing your development time. One such
library is matplotlib (http://matplotlib.sourceforge.net), which provides a
plotting library with a similar interface to those available within Matlab. The matplotlib website provides a detailed tutorial and documentation for all the different
options available within the library but this worksheet provides some examples of
the common plot types and a more complex example continuing on from previous
examples.

5.2

Simple Script

Below is your first script using the matplotlib library. The script demonstrates
the plotting of a mathematical function, in this case a sine function. The plot
function requires two lists of numbers to be provided, which provides the x and
y locations of the points which go to create the displayed function. The axis can
be labelled using the xlabel() and ylabel() functions while the title is set using
the title() function. Finally, the show() function is used to reveal the interface

48

CHAPTER 5. PLOTTING - MATPLOTLIB

49

displaying the plot.

1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# A simple python script to display a
# sine function
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12
13

# import the matplotlib libraries

from matplotlib import pylab as plt

14
15
16
17
18
19
20

#
#
t
#
#
s

Create a list with values from

0 to 3 with 0.01 intervals
= arange(0.0, 3, 0.01)
Calculate the sin curve for
the values within t
= sin(pi*t)

21
22
23
24
25
26
27
28

# Plot the values in s and t

plt.plot(t, s)
plt.xlabel(X Axis)
plt.ylabel(Y Axis)
plt.title(Simple Plot)
# save plot to disk.
plt.savefig(simpleplot.pdf, dpi=200, format=PDF)

5.3

Bar Chart

The creation of a bar chart is equally simply where two lists are provided, the
first contains the locations on the X axis at which the bars start and the second
the heights of the bars. The width of the bars can also be specified and their
colour. More options are available in the documentation (http://matplotlib.

CHAPTER 5. PLOTTING - MATPLOTLIB

50

Simple Plot

1.0

Y Axis

0.5

0.0

0.5

1.00.0

0.5

1.0

1.5
X Axis

2.0

2.5

Figure 5.1: A simple plot using matplotlib.

sourceforge.net/matplotlib.pylab.html#-bar)
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# A simple python script to display a
# bar chart.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12

from matplotlib import pylab as plt

13
14
15
16
17

# Values for the Y axis (i.e., height of bars)

height = [5, 6, 7, 8, 12, 13, 9, 5, 7, 4, 3, 1]
# Values for the x axis
x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

3.0

CHAPTER 5. PLOTTING - MATPLOTLIB

51

18
19
20
21
22

# create plot with colour grey

plt.bar(x, height, width=1, color=gray)
# save plot to disk.
plt.savefig(simplebar.pdf, dpi=200, format=PDF)

14
12
10
8
6
4
2
00

10

12

14

Figure 5.2: A simple bar chart using matplotlib.

5.4

Pie Chart

A pie chart is similar to the previous scripts where a list of the fractions making
up the pie chart is given alongside a list of labels and if required a list of fractions to explode the pie chart. Other options including colour and shadow are
available and outlined in the documentation (http://matplotlib.sourceforge.
net/matplotlib.pylab.html#-pie) This script also demonstrates the use of the
savefig() function allowing the plot to be saved to file rather than simply displayed
on screen.

CHAPTER 5. PLOTTING - MATPLOTLIB

52

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# A simple python script to display a
# pie chart.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12

from matplotlib import pylab as plt

13
14
15
16

frac = [25, 33, 17, 10, 15]

labels = [25, 33, 17, 10, 15]
explode = [0, 0.25, 0, 0, 0]

17
18
19
20
21
22
23

# Create pie chart

plt.pie(frac, explode, labels, shadow=True)
# Give it a title
plt.title(A Sample Pie Chart)
# save the plot to a PDF file
plt.savefig(pichart.pdf, dpi=200, format=PDF)

5.5

Scatter Plot

The following script demonstrates the production of a scatter plot (http://matplotlib.

sourceforge.net/matplotlib.pylab.html#-scatter) where the lists x and y
provide the locations of the points in the X and Y axis and Z provides the third
dimension used to colour the points.
1

#! /usr/bin/env python

2
3
4
5
6

#######################################
# A simple python script to display a
# scatter plot.
# Author: <YOUR NAME>

CHAPTER 5. PLOTTING - MATPLOTLIB

53

A Sample Pie Chart

25

33

15
17

10

Figure 5.3: A simple pie chart using matplotlib.

7
8
9
10

# Email: <YOUR EMAIL>

# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12
13
14

from matplotlib import pylab as plt

# Import a random number generator
from random import random

15
16
17
18

x = []
y = []
z = []

19
20
21
22
23
24
25

# Create data values for X, Y, Z axis

for i in range(5000):
x.append(random() * 100)
y.append(random() * 100)
z.append(x[i]-y[i])

CHAPTER 5. PLOTTING - MATPLOTLIB

27
28
29
30
31
32
33
34
35
36
37
38
39

# Create figure
plt.figure()
# Create scatter plot where the plots are coloured using the
# Z values.
plt.scatter(x, y, c=z, marker=o, cmap=cm.jet, vmin=-100, vmax=100)
# Display colour bar
colorbar()
# Make axis tight to the data
plt.axis(tight)
plt.xlabel(X Axis)
plt.ylabel(Y Axis)
plt.title(Simple Scatter Plot)
# save plot to disk.
plt.savefig(simplescatter.pdf, dpi=200, format=PDF)

Simple Scatter Plot

100

100

80
60

80

40
20

60
Y Axis

26

54

0
40

20
40

20

60
80

0
0

20

40

X Axis

60

80

100

100

Figure 5.4: A simple scatter plot using matplotlib.

CHAPTER 5. PLOTTING - MATPLOTLIB

5.6

55

Line Plot

A more complicated example is now given building on the previous tutorial where
the data is read in from a text file before being plotted. In this case data was downloaded from the Environment Agency and converted from columns to rows. The
dataset provides the five year average rainfall for the summer (June - August) and
winter (December - February) from 1766 to 2006. Two examples of plotting this
data are given where the first plots the two datasets onto the same axis (Figure 5.5)
while the second plots them onto individual axis (Figure 5.6). Information on the
use of the subplot() function can be found in the matplotlib documentation (http:
//matplotlib.sourceforge.net/matplotlib.pylab.html#-subplot).
1
2
3
4
5
6
7
8
9

#######################################
# A python script to read in a text file
# of rainfall data for summer and winter
# within the UK and display as a plot.
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

10
11
12
13

from matplotlib import pylab as plt

import os.path
import sys

14
15

class PlotRainfall (object):

16
17
18
19
20
21
22
23
24
25
26

# Parse the input file - Three columns year, summer, winter

def parseDataFile(self, dataFile, year, summer, winter):
line = 0
for eachLine in dataFile:
commaSplit = eachLine.split(,, eachLine.count(,))
first = True
for token in commaSplit:
if first:
first = False
else:

CHAPTER 5. PLOTTING - MATPLOTLIB

27
28
29
30
31
32
33

if line == 0:
year.append(int(token))
elif line == 1:
summer.append(float(token))
elif line == 2:
winter.append(float(token))
line += 1

34
35
36
37
38
39
40
41
42
43
44
45

# Plot data onto the same axis

def plotData(self, year, summer, winter, outFile):
plt.figure()
plt.plot(year, summer)
plt.plot(year, winter)
plt.legend([Summer, Winter])
plt.xlabel(Year)
plt.ylabel(Rainfall (5 Year Mean))
plt.title(Summer and Winter rainfall across the UK)
# save plot to disk.
plt.savefig(outFile, dpi=200, format=PDF)

46
47
48
49
50
51
52
53
54

# Plot the data onto separate axis, using subplot.

def plotDataSeparate(self, year, summer, winter, outFile):
plt.figure()
plt.subplot(2,1,1)
plt.plot(year, summer)
plt.ylabel(Rainfall (5 Year Mean))
plt.title(Summer rainfall across the UK)
plt.axis(tight)

55
56
57
58
59
60
61
62
63

plt.subplot(2,1,2)
plt.plot(year, winter)
plt.xlabel(Year)
plt.ylabel(Rainfall (5 Year Mean))
plt.title(Winter rainfall across the UK)
plt.axis(tight)
# save plot to disk.
plt.savefig(outFile, dpi=200, format=PDF)

64
65
66
67

def run(self):
filename = ukweatheraverage.csv
if os.path.exists(filename):

56

CHAPTER 5. PLOTTING - MATPLOTLIB

68
69
70
71
72
73
74
75
76
77
78
79
80
81

57

year = list()
summer = list()
winter = list()
try:
dataFile = open(filename, r)
except IOError as e:
print(\nCould not open file:\n, e)
return
self.parseDataFile(dataFile, year, summer, winter)
dataFile.close()
self.plotData(year, summer, winter, "Rainfall_SinglePlot.pdf")
self.plotDataSeparate(year, summer, winter, "Rainfall_MultiplePlots.pdf")
else:
print(File \ + filename + \ does not exist.)

82

84
85

if __name__ == __main__:
obj = PlotRainfall()
obj.run()

350

Summer and Winter rainfall across the UK

Summer
Winter

300
Rainfall (5 Year Mean)

83

250
200
150
100
1750

1800

1850

1900
Year

1950

2000

2050

Figure 5.5: Rainfall data for summer and winter on the same axis.

CHAPTER 5. PLOTTING - MATPLOTLIB

58

Rainfall (5 Year Mean)

Rainfall (5 Year Mean)

Summer rainfall across the UK

300
280
260
240
220
200
180
160

1800

300

1850

1900

1950

2000

1850

1900
Year

1950

2000

Winter rainfall across the UK

250
200
150
1800

Figure 5.6: Rainfall data for summer and winter on different axis.

5.7

Exercise:

Based on the available data is there a correlation between summer and winter rainfall? Use the lists read in of summer and winter rainfall and produce a scatterplot
to answer this question.

5.8

Further Reading

Matplotlib http://matplotlib.sourceforge.net
Python Documentation http://www.python.org/doc/
Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7

Chapter 6
Statistics (SciPy / NumPy)
6.1

Introduction

NumPy is a library for storing and manipulating multi-dimensional arrays. NumPy

arrays are similar to lists, however they have a lot more functionality and allow
faster operations. SciPy is a library for maths and science using NumPy arrays and
includes routines for statistics, optimisation and numerical integration. A comprehensive list is available from the SciPy website (http://docs.scipy.org/doc/
scipy/reference). The combination of NumPy, SciPy and MatPlotLib provides
similar functionality to that available in packages such as MatLab and Mathematica and allows for complex numerical analysis.
This tutorial will introduce some of the statistical functionality of NumPy / SciPy
by calculating statistics from forest inventory data, read in from a text file. Linear regression will also be used to calculate derive relationships between parameters.
There are a number of ways to create NumPy arrays, one of the easiest (and
the method that will be used in this tutorial) is to convert a python list to an
array:
import numpy
pythonList = [1 , 4 , 2 , 5, 3]
numpyArray = numpy.array(pythonList)

59

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

6.2

60

Simple Statistics

Forest inventory data have been collected for a number of plots within Penglais
woods (Aberystwyth, Wales). For each tree, the diameter, species height, crown
size and position have been recorded. An example script is provided to read the
diameters into a separate list for each species. The lists are then converted to
NumPy arrays, from which statistics are calculated and written out to a text
file.
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# A script to calculate statistics from
# a text file using NumPy
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12
13
14
15

import numpy
import scipy
# Import scipy stats functions we need
import scipy.stats as spstats

16
17

class CalculateStatistics (object):

18
19
20
21
22
23
24
25
26
27

def run(self):
# Set up lists to hold input diameters
# A seperate list is used for each species
beechDiameter = list()
ashDiameter = list()
birchDiameter = list()
oakDiameter = list()
sycamoreDiameter = list()
otherDiameter = list()

28
29
30

# Open input and output files

inFileName = PenglaisWoodsData.csv

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

31
32
33

outFileName = PenglaisWoodsStats.csv
inFile = open(inFileName, r)
outFile = open(outFileName,w)

34
35
36
37
38
39
40
41
42
43

# Iterate through the input file and save diameter into

# lists, based on species
header = True
for eachLine in inFile:
if header: # Skip header row
print(Skipping header row)
header = False
else:
substrs = eachLine.split(,,eachLine.count(,))

44
45
46
47

species = substrs[3]
if substrs[4].isdigit: # Check diameter is a number
diameter = float(substrs[4])

48
49
50
51
52
53
54
55
56
57
58
59
60

if species == BEECH:
beechDiameter.append(diameter)
elif species == ASH:
ashDiameter.append(diameter)
elif species == BIRCH:
birchDiameter.append(diameter)
elif species == OAK:
oakDiameter.append(diameter)
elif species == SYC:
sycamoreDiameter.append(diameter)
else:
otherDiameter.append(diameter)

61
62
63
64
65
66
67
68

# Convert input lists to NumPy arrays

beechDiameter = numpy.array(beechDiameter)
ashDiameter = numpy.array(ashDiameter)
birchDiameter = numpy.array(birchDiameter)
oakDiameter = numpy.array(oakDiameter)
sycamoreDiameter = numpy.array(sycamoreDiameter)
otherDiameter = numpy.array(otherDiameter)

69
70
71

# Write header line to output file

headerLine = species, meanDiameter, medianDiameter, stDevDiameter\n

61

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

72

62

outFile.write(headerLine)

73
74
75
76
77
78
79
80
81
82
83
84
85
86

# Calculate statistics for each species and write to file

outLine = Beech, + self.createStatsLine(beechDiameter) + \n
outFile.write(outLine)
outLine = Ash, + self.createStatsLine(ashDiameter) + \n
outFile.write(outLine)
outLine = Birch, + self.createStatsLine(birchDiameter) + \n
outFile.write(outLine)
outLine = Oak, + self.createStatsLine(oakDiameter) + \n
outFile.write(outLine)
outLine = Sycamore, + self.createStatsLine(sycamoreDiameter) + \n
outFile.write(outLine)
outLine = Other, + self.createStatsLine(otherDiameter) + \n
outFile.write(outLine)

87
88

print(Statistics written to: + outFileName)

89
90
91
92
93
94
95
96

def createStatsLine(self, inArray):

# Calculate statsistics for NumPy array and return output line.
meanArray = numpy.mean(inArray)
medianArray = numpy.median(inArray)
stDevArray = numpy.std(inArray)
skewArray = spstats.skew(inArray)

97
98
99
100

# Create output line with stats

statsLine = str(meanArray) + , + str(medianArray) + , + str(stDevArray)
return statsLine

101
102
103
104

if __name__ == __main__:
obj = CalculateStatistics()
obj.run()

Note in tutorial three, functions were written to calculate the mean and standard
deviation a list, in this tutorial the same result is accomplished using the built in
functionality of NumPy.

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

6.2.1

63

Exercises

1. Based on the example script also calculate mean, median and standard deviation for tree heights and add to the output file.
2. Look at other statistics functions available in SciPy and calculate for height
and density.

6.3

Calculate Biomass

One of the features of NumPy arrays is the ability to perform mathematical operation on all elements of an array.
For example, for NumPy array a:
a = numpy.array([1,2,3,4])

Performing
b = 2 * a

Gives
b = array([2,4,6,8])

Some special versions of functions are available to work on arrays. To calculate

the natural log of a single number log may be used, to perform the natural log of
an array np.log may be used (where NumPy has been imported as np).
Tree volume may be calculated from height and stem diameter using:
Volume = a + bD2 h0.75

(6.1)

Where D is diameter and h is height. The coefficients a and b vary according

to species (see Table 6.1). From volume, it is possible to calculate biomass by
multiplying by the specific gravity.

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

Biomass = Volume SpecificGravity

64

(6.2)

The specific gravity also varies by species, values for each species are given in
Table 6.1.
Table 6.1: Coefficients for estimating volume and the specific gravity required for
estimating the biomass by species.
Species
a-coefficient b-coefficient Specific gravity
Beech
0.014306
0.0000748
0.56
Ash
0.012107
0.0000777
0.54
Beech
0.009184
0.0000673
0.53
Oak
0.011724
0.0000765
0.56
Sycamore
0.012668
0.0000737
0.54
The following function takes two arrays containing height and density, and a string
for species. From these biomass is calculated.
1
2
3
4
5

def calcBiomass(self, inDiameterArray, inHeightArray, inSpecies):

if inSpecies == BEECH:
a = 0.014306
b = 0.0000748
specificGravity = 0.56

6
7
8
9
10
11
12

# Calculate Volume
volume = a + ((b*(inDiameterArray / 100)**2) * (inHeightArray**0.75))
# Calculate biomass
biomass = volume * specificGravity
# Return biomass
return biomass

Note only the coefficients for BEECH have been included therefore, if a different
species is passed in, the program will produce an error (try to think about what
the error would be). A neater way of dealing with the error would be to throw an
exception if the species was not recognised. Exceptions form the basis of controlling
errors in a number of programming languages (including C++ and Java) the simple
concept is that as a program is running, if an error occurs an exception is thrown,
at which point processing stops until the exception is caught and dealt with. If the

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

65

exception is never caught, then the software crashes and stops. Python provides
the following syntax for exception programming,
try:
< Perform operations during which
an error is likely to occur >
except <ExceptionName>:
< If error occurs do something
appropriate >

where the code you wish to run is written inside the try statement and the except
statement is executed only when a named exception (within the except statement)
is produced within the try block. It is good practise you use exceptions where
possible as when used properly they provide more robust code which can provide
more feedback to the user.
The function to calculate biomass may be rewritten to throw an exception if the
species is not recognised.
1
2
3
4
5
6
7

def calcBiomass(self, inDiameterArray, inHeightArray, inSpecies):

if inSpecies == BEECH:
a = 0.014306
b = 0.0000748
specificGravity = 0.56
else: # Raise exception if species is not recognised
raise Exception(Species not recognised)

8
9
10
11
12
13
14

# Calculate Volume
volume = a + ((b*(inDiameterArray / 100)**2) * (inHeightArray**0.75))
# Calculate biomass
biomass = volume * specificGravity
# Return biomass
return biomass

The function below, calls calcBiomass to calculate biomass for an array. From
this mean, median and standard deviation are calculated and an output array is
returned. By calling the function from within a try and except block if the species
is not recognised, it will not try to calculate stats and will return the string na
(not available) for all values in the output line.

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

1
2
3
4
5
6
7
8
9
10

66

def calcBiomassStatsLine(self, inDiameterArray, inHeightArray, inSpecies):

# Calculates biomass, calculates stats from biomass and returns output line
biomassStatsLine =
try:
# Calculate biomass
biomass = self.calcBiomass(inDiameterArray, inHeightArray, inSpecies)
# Calculate stats from biomass
meanBiomass = numpy.mean(biomass)
medianBiomass = numpy.median(biomass)
stDevBiomass = numpy.std(biomass)

11
12
13
14

# Create output line

biomassStatsLine = str(meanBiomass) + , + str(medianBiomass) + , + \
str(stDevBiomass)

15
16
17
18

except Exception:
# Catch exception and write na for all values
biomassStatsLine = na,na,na

19
20

return biomassStatsLine

Therefore, the final script should result in the following:

1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# A script to calculate statistics from
# a text file using NumPy
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12
13
14
15

import numpy
import scipy
# Import scipy stats functions we need
import scipy.stats as spstats

16
17
18

class CalculateStatistics (object):

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

def run(self):
# Set up lists to hold input diameters and heights
# A seperate list is used for each species
beechDiameter = list()
beechHeight = list()
ashDiameter = list()
ashHeight = list()
birchDiameter = list()
birchHeight = list()
oakDiameter = list()
oakHeight = list()
sycamoreDiameter = list()
sycamoreHeight = list()
otherDiameter = list()
otherHeight = list()

34
35
36
37
38
39

# Open input and output files

inFileName = PenglaisWoodsData.csv
outFileName = PenglaisWoodsStats.csv
inFile = open(inFileName, r)
outFile = open(outFileName,w)

40
41
42
43
44
45
46
47
48
49

# Iterate through the input file and save diameter and height
# into lists, based on species
header = True
for eachLine in inFile:
if header: # Skip header row
print(Skipping header row)
header = False
else:
substrs = eachLine.split(,,eachLine.count(,))

50
51
52
53
54

species = substrs[3]
if substrs[4].isdigit: # Check diameter is a number
diameter = float(substrs[4])
height = float(substrs[10])

55
56
57
58
59

if species == BEECH:
beechDiameter.append(diameter)
beechHeight.append(height)
elif species == ASH:

67

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

60
61
62
63
64
65
66
67
68
69
70
71
72
73

ashDiameter.append(diameter)
ashHeight.append(height)
elif species == BIRCH:
birchDiameter.append(diameter)
birchHeight.append(height)
elif species == OAK:
oakDiameter.append(diameter)
oakHeight.append(height)
elif species == SYC:
sycamoreDiameter.append(diameter)
sycamoreHeight.append(height)
else:
otherDiameter.append(diameter)
otherHeight.append(height)

74
75
76
77
78
79
80
81

# Convert to NumPy arrays

beechDiameter = numpy.array(beechDiameter)
ashDiameter = numpy.array(ashDiameter)
birchDiameter = numpy.array(birchDiameter)
oakDiameter = numpy.array(oakDiameter)
sycamoreDiameter = numpy.array(sycamoreDiameter)
otherDiameter = numpy.array(otherDiameter)

82
83
84
85
86
87
88

beechHeight = numpy.array(beechHeight)
ashHeight = numpy.array(ashHeight)
birchHeight = numpy.array(birchHeight)
oakHeight = numpy.array(oakHeight)
sycamoreHeight = numpy.array(sycamoreHeight)
otherHeight = numpy.array(otherHeight)

89
90
91
92
93
94

# Write header line to output file

headerLine = species,meanDiameter,medianDiameter,stDevDiameter,\
meanHeight,medianHeight,stDevHeight,\
meanBiomass,medianBiomass,stDevBiomass\n
outFile.write(headerLine)

95
96
97
98
99
100

# Calculate statistics and biomass for each species and write to file
outLine = Beech, + self.createStatsLine(beechDiameter) + , + \
self.createStatsLine(beechHeight) + , + \
self.calcBiomassStatsLine(beechDiameter, beechHeight, BEECH) + \n
outFile.write(outLine)

68

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

69

outLine = Ash, + self.createStatsLine(ashDiameter) + , + \

self.createStatsLine(ashHeight) + , + \
self.calcBiomassStatsLine(ashDiameter, ashHeight, ASH) + \n
outFile.write(outLine)
outLine = Birch, + self.createStatsLine(birchDiameter) + , + \
self.createStatsLine(birchHeight) + , + \
self.calcBiomassStatsLine(birchDiameter, birchHeight, BIRCH) + \n
outFile.write(outLine)
outLine = Oak, + self.createStatsLine(oakDiameter) + , + \
self.createStatsLine(oakHeight) + , + \
self.calcBiomassStatsLine(oakDiameter, oakHeight, OAK) + \n
outFile.write(outLine)
outLine = Sycamore, + self.createStatsLine(sycamoreDiameter) + , + \
self.createStatsLine(sycamoreHeight) + , + \
self.calcBiomassStatsLine(sycamoreDiameter, sycamoreHeight, SYC) + \n
outFile.write(outLine)
outLine = Other, + self.createStatsLine(otherDiameter) + , + \
self.createStatsLine(otherHeight) + , + \
self.calcBiomassStatsLine(otherDiameter, otherHeight, Other) + \n
outFile.write(outLine)

121
122

print(Statistics written to: + outFileName)

123
124
125
126
127
128

def createStatsLine(self, inArray):

# Calculate statsistics for array and return output line.
meanArray = numpy.mean(inArray)
medianArray = numpy.median(inArray)
stDevArray = numpy.std(inArray)

129
130
131
132

# Create output line with stats

statsLine = str(meanArray) + , + str(medianArray) + , + str(stDevArray)
return statsLine

133
134
135
136
137
138
139
140
141

def calcBiomassStatsLine(self, inDiameterArray, inHeightArray, inSpecies):

# Calculates biomass, calculates stats from biomass and returns output line
biomassStatsLine =
try:
# Calculate biomass
biomass = self.calcBiomass(inDiameterArray, inHeightArray, inSpecies)
# Calculate stats from biomass
meanBiomass = numpy.mean(biomass)

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

70

medianBiomass = numpy.median(biomass)
stDevBiomass = numpy.std(biomass)

142
143
144
145
146
147

# Create output line

biomassStatsLine = str(meanBiomass) + , + str(medianBiomass) + , + \
str(stDevBiomass)

148

except Exception:
# Catch exception and write na for all values
biomassStatsLine = na,na,na

149
150
151
152

return biomassStatsLine

153
154
155
156
157
158
159
160
161

def calcBiomass(self, inDiameterArray, inHeightArray, inSpecies):

if inSpecies == BEECH:
a = 0.014306
b = 0.0000748
specificGravity = 0.56
else: # Raise exception is species is not recognised
raise Exception(Species not recognised)

162

# Calcualte volume
volume = a + ((b*(inDiameterArray)**2) * (inHeightArray**0.75))
# Calculate biomass
biomass = volume * specificGravity
# Return biomass
return biomass

163
164
165
166
167
168
169
170
171
172

if __name__ == __main__:
obj = CalculateStatistics()
obj.run()

6.3.1

Exercise

1. Add in the coefficients to calculate biomass for the other species

2. Write the statistics for biomass out to the text file. Remember to change the
header line.

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

6.4

71

Linear Fitting

One of the built in feature of SciPy is the ability to perform fits. Using the linear
regression function (linregress) it is possible to fit equations of the form:

y = ax + b

(6.3)

to two NumPy arrays (x and y) using:

(aCoeff,bCoeff,rVal,pVal,stdError) = linregress(x, y)

Where aCoeff and bCoeff are the coefficients rVal is the r value (r**2 gives R2 ),
pVal is the p value and stdError is the standard error.
It is possible to fit the following equation to the collected data expressing height
as a function of diameter.

height = a log (diameter) + b

(6.4)

To fit an equation of this form an array must be created containing log diameter
. Linear regression may then be performed using:
linregress(np.log(inDiameterArray), inHeightArray)

To test the fit it may be plotted against the original data using MatPlotLib. The
following code first performs the linear regression then creates a plot showing the
fit against the original data.
1
2
3
4

def plotLinearRegression(self, inDiameterArray, inHeightArray, outPlotName):

# Perform fit
(aCoeff,bCoeff,rVal,pVal,stdError) = linregress(np.log(inDiameterArray), \
inHeightArray)

5
6
7
8

# Use fits to predict height for a range of diameters

testDiameter = arange(min(inDiameterArray), max(inDiameterArray), 1)
predictHeight = (aCoeff * np.log(testDiameter)) + bCoeff

9
10

# Create a string, showing the form of the equation (with fitted coefficients)

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

11
12
13
14

72

# and r squared value

# Coefficients are rounded to two decimal places.
equation = str(round(aCoeff,2)) + log(D) + + str(round(bCoeff,2)) + \
(r$^2$ = + str(round(rVal**2,2)) + )

15
16
17
18
19
20
21

# Plot fit against origional data

plt.plot(inDiameterArray, inHeightArray,.)
plt.plot(testDiameter, predictHeight)
plt.xlabel(Diameter (cm))
plt.ylabel(Height (m))
plt.legend([measured data,equation])

22
23
24

# Save plot
plt.savefig(outPlotName, dpi=200, format=PDF)

The coefficients and r2 of the fit are displayed in the legend. To display the
superscript 2 in the data it is possible to use LaTeX syntax. So r2 is written as:
r$2$.
The function may be called using:
# Set output directory for plots
outDIR = ./output/directory/
self.plotLinearRegression(beechDiameter, beechHeight, outDIR + beech.pdf)

Produce a plot similar to the one shown in Figure 6.1 and save as a PDF.
The final script should result in the following:
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10
11

#######################################
# A script to calculate statistics from
# a text file using NumPy
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

45

73

measured data
5.54log(D) -3.7 (r2 = 0.37)

40
35

Height (m)

30
25
20
15
10
5
00

20

40
60
Diameter (cm)

80

Figure 6.1: A simple plot using matplotlib.

12
13
14
15
16
17

import numpy
import scipy
# Import scipy stats functions we need
import scipy.stats as spstats
# Import plotting library as plt
import matplotlib.pyplot as plt

18
19

class CalculateStatistics (object):

20
21
22
23
24
25
26
27
28
29
30

def run(self):
# Set up lists to hold input diameters and heights
# A seperate list is used for each species
beechDiameter = list()
beechHeight = list()
ashDiameter = list()
ashHeight = list()
birchDiameter = list()
birchHeight = list()
oakDiameter = list()

100

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

31
32
33
34
35

oakHeight = list()
sycamoreDiameter = list()
sycamoreHeight = list()
otherDiameter = list()
otherHeight = list()

36
37
38
39
40
41

# Open input and output files

inFileName = PenglaisWoodsData.csv
outFileName = PenglaisWoodsStats.csv
inFile = open(inFileName, r)
outFile = open(outFileName,w)

42
43
44
45
46
47
48
49
50
51

# Iterate through the input file and save diameter and height
# into lists, based on species
header = True
for eachLine in inFile:
if header: # Skip header row
print(Skipping header row)
header = False
else:
substrs = eachLine.split(,,eachLine.count(,))

52
53
54
55
56

species = substrs[3]
if substrs[4].isdigit: # Check diameter is a number
diameter = float(substrs[4])
height = float(substrs[10])

57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

if species == BEECH:
beechDiameter.append(diameter)
beechHeight.append(height)
elif species == ASH:
ashDiameter.append(diameter)
ashHeight.append(height)
elif species == BIRCH:
birchDiameter.append(diameter)
birchHeight.append(height)
elif species == OAK:
oakDiameter.append(diameter)
oakHeight.append(height)
elif species == SYC:
sycamoreDiameter.append(diameter)

74

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

72
73
74
75

sycamoreHeight.append(height)
else:
otherDiameter.append(diameter)
otherHeight.append(height)

76
77
78
79
80
81
82
83

# Convert to NumPy arrays

beechDiameter = numpy.array(beechDiameter)
ashDiameter = numpy.array(ashDiameter)
birchDiameter = numpy.array(birchDiameter)
oakDiameter = numpy.array(oakDiameter)
sycamoreDiameter = numpy.array(sycamoreDiameter)
otherDiameter = numpy.array(otherDiameter)

84
85
86
87
88
89
90

beechHeight = numpy.array(beechHeight)
ashHeight = numpy.array(ashHeight)
birchHeight = numpy.array(birchHeight)
oakHeight = numpy.array(oakHeight)
sycamoreHeight = numpy.array(sycamoreHeight)
otherHeight = numpy.array(otherHeight)

91
92
93
94

# Write header line to output file

headerLine = species, meanDiameter, medianDiameter, stDevDiameter\n
outFile.write(headerLine)

95
96
97
98
99
100
101
102
103
104
105
106
107
108

# Calculate statistics for each species and write to file

outLine = Beech, + self.createStatsLine(beechDiameter) + \n
outFile.write(outLine)
outLine = Ash, + self.createStatsLine(ashDiameter) + \n
outFile.write(outLine)
outLine = Birch, + self.createStatsLine(birchDiameter) + \n
outFile.write(outLine)
outLine = Oak, + self.createStatsLine(oakDiameter) + \n
outFile.write(outLine)
outLine = Sycamore, + self.createStatsLine(sycamoreDiameter) + \n
outFile.write(outLine)
outLine = Other, + self.createStatsLine(otherDiameter) + \n
outFile.write(outLine)

109
110

print(Statistics written to: + outFileName)

111
112

# Fit line to each file and save out plot

75

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

76

113
114
115

# Set output directory for plots

outDIR = ./

116
117
118
119

# Plot linear regression for Beech

print(Generating plot:)
self.plotLinearRegression(beechDiameter, beechHeight, outDIR + beech.pdf)

120
121
122
123

def plotLinearRegression(self, inDiameterArray, inHeightArray, outPlotName):

# Perform fit
(aCoeff,bCoeff,rVal,pVal,stdError) = spstats.linregress(numpy.log(inDiameterArray), inHei

124
125
126
127

# Use fits to predict height for a range of diameters

testDiameter = numpy.arange(min(inDiameterArray), max(inDiameterArray), 1)
predictHeight = (aCoeff * numpy.log(testDiameter)) + bCoeff

128
129
130
131
132
133

# Create a string, showing the form of the equation (with fitted coefficients)
# and r squared value
# Coefficients are rounded to two decimal places.
equation = str(round(aCoeff,2)) + log(D) + str(round(bCoeff,2)) + \
(r$^2$ = + str(round(rVal**2,2)) + )

134
135
136
137
138
139
140

# Plot fit against origional data

plt.plot(inDiameterArray, inHeightArray,.)
plt.plot(testDiameter, predictHeight)
plt.xlabel(Diameter (cm))
plt.ylabel(Height (m))
plt.legend([measured data,equation])

141
142
143

# Save plot
plt.savefig(outPlotName, dpi=200, format=PDF)

144
145
146
147
148
149
150

def createStatsLine(self, inArray):

# Calculate statsistics for array and return output line.
meanArray = numpy.mean(inArray)
medianArray = numpy.median(inArray)
stDevArray = numpy.std(inArray)

151
152
153

# Create output line with stats

statsLine = str(meanArray) + , + str(medianArray) + , + str(stDevArray)

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

154

77

return statsLine

155
156
157
158
159
160
161
162
163
164
165

def calcBiomassStatsLine(self, inDiameterArray, inHeightArray, inSpecies):

# Calculates biomass, calculates stats from biomass and returns output line
biomassStatsLine =
try:
# Calculate biomass
biomass = self.calcBiomass(inDiameterArray, inHeightArray, inSpecies)
# Calculate stats from biomass
meanBiomass = numpy.mean(biomass)
medianBiomass = numpy.median(biomass)
stDevBiomass = numpy.std(biomass)

166
167
168
169

# Create output line

biomassStatsLine = str(meanBiomass) + , + str(medianBiomass) + ,\
+ str(stDevBiomass)

170
171
172
173

except Exception:
# Catch exception and write na for all values
biomassStatsLine = na,na,na

174
175

return biomassStatsLine

176
177
178
179
180
181
182
183

def calcBiomass(self, inDiameterArray, inHeightArray, inSpecies):

if inSpecies == BEECH:
a = 0.014306
b = 0.0000748
specificGravity = 0.56
else: # Raise exception is species is not recognised
raise Exception(Species not recognised)

184
185
186
187
188
189
190

# Calcualte volume
volume = a + ((b*(inDiameterArray)**2) * (inHeightArray**0.75))
# Calculate biomass
biomass = volume * specificGravity
# Return biomass
return biomass

191
192
193
194

if __name__ == __main__:
obj = CalculateStatistics()
obj.run()

CHAPTER 6. STATISTICS (SCIPY / NUMPY)

6.4.1

78

Exercise

Produce plots, showing linear regression fits, for the other species.

6.5

Further Reading

SciPy http://www.scipy.org/SciPy
NumPy http://numpy.scipy.org
An Introduction to Python, G. van Rossum, F.L. Drake, Jr. Network Theory
ISBN 0-95-416176-9 (Also available online http://docs.python.org/3/
tutorial/) - Chapter 8.
Python Documentation http://www.python.org/doc/
Matplotlib http://matplotlib.sourceforge.net

Chapter 7
Batch Processing Command Line
Tools
7.1

Introduction

There are many command line tools and utilities available for all platforms (e.g.,
Windows, Linux, Mac OSX), these tools are extremely useful and range from
simple tasks such as renaming a file to more complex tasks such as merging ESRI
shapefiles. One problem with these tools is that if you have a large number of files,
which need to be processed in the same way, it is time consuming and error prone
to manual run the command for each file. Therefore, if we can write scripts to do
this work for us then processing large number of individual files becomes a much
simpler and quicker task.
For this worksheet you will need to have the command line tools which come with
the GDAL/OGR (http://www.gdal.org) open source software library installed
and available with your path. With the installation of python(x,y) the python
libraries for GDAL/OGR have been installed but not the command line utilities
which go along with these libraries. If you do not already have them installed
therefore details on the GDAL website for your respective platform.

79

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

7.2

80

Merging ESRI Shapefiles

The first example illustrates how the ogr2ogr command can be used to merge
shapefiles and a how a python script can be used to turn this command into a
batch process where a whole directory of shapefiles can be merged.
To perform this operation two commands are required. The first makes a copy of
the first shapefile within the list of files into a new file, shown below:
> ogr2ogr <inputfile> <outputfile>

While the second command appends the contents of the inputted shapefile onto
the end of an existing shapefile (i.e., the one just copied).
> ogr2ogr -update -append <inputfile> <outputfile> -nln <outputfilename>

For both these commands the shapefiles all need to be of the same type (point,
polyline or polygon) and contain the same attributes. Therefore, your first exercise
is to understand the use of the ogr2ogr command and try them from the command
line with the data provided. Hint, running ogr2ogr without any options the help
file will be displayed.
The second stage is to develop a python script to call the appropriate commands to perform the required operation, where the following processes will be
required:
1. Get the user inputs.
2. List the contents of the input directory.
3. Iterate through the directory and run the required commands.
But the first step is to create the class structure in which the code will fit, this
will be something similar to that shown below:
1

#! /usr/bin/env python

2
3
4
5
6

#######################################
# MergeSHPfiles.py
# A python script to merge shapefiles
# Author: <YOUR NAME>

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

7
8
9
10

81

# Email: <YOUR EMAIL>

# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12

import os

13
14

class MergeSHPfiles (object):

15
16
17
18
19
20
21

# A function which controls the rest of the script

def run(self):
# Define the input directory
filePath = ./TreeCrowns/
# Define the output file
newSHPfile = Merged_shapefile.shp

22
23
24
25
26
27
28

# The start of the code

if __name__ == __main__:
# Make an instance of the class
obj = MergeSHPfiles()
# Call the function run()
obj.run()

The script will have the input directory and output file hard coded (as shown)
within the run function. Therefore, you need to edit these file paths to the location
you have the files saved. Please note that under Windows you need to insert a
double slash (i.e., \\) within the file path as a single slash is an escape character
(e.g., \n for new line) within strings.
The next step is to check that the input directory exists and is a directory, to do
this edit your run function as below.
1
2
3
4
5
6

# A function which controls the rest of the script

def run(self):
# Define the input directory
filePath = ./TreeCrowns/
# Define the output file
newSHPfile = Merged_shapefile.shp

7
8

# Check input file path exists and is a directory

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

9
10
11
12
13
14
15

82

if not os.path.exists(filePath):
print Filepath does not exist
elif not os.path.isdir(filePath):
print Filepath is not a directory!
else:
# Merge the shapefiles within the filePath
self.mergeSHPfiles(filePath, newSHPfile)

Additionally, you need to add the function mergeSHPFiles, which is where the
shapefiles will be merged.
# A function to control the merging of shapefiles
def mergeSHPfiles(self, filePath, newSHPfile):

To merge the shapefiles the first task is to get a list of all the shapefiles within a
directory. To do this, use the code you developed in Tutorial 4 to list files within a
directory and edit it such that the files are outputted to a list rather than printed
to screen, as shown below.
1
2
3
4
5
6
7
8
9
10
11
12
13

# A function to test the file extension of a file

def checkFileExtension(self, filename, extension):
# Boolean variable to be returned by the function
foundExtension = False;
# Split the filename into two parts (name + ext)
filenamesplit = os.path.splitext(filename)
# Get the file extension into a varaiable
fileExtension = filenamesplit[1].strip()
# Decide whether extensions are equal
if(fileExtension == extension):
foundExtension = True
# Return result
return foundExtension

14
15
16
17
18
19
20
21
22

# A function which iterates through the directory and checks file extensions
def findFilesExt(self, directory, extension):
# Define a list to store output list of files
fileList = list()
# check whether the current directory exits
if os.path.exists(directory):
# check whether the given directory is a directory

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

83

if os.path.isdir(directory):
# list all the files within the directory
dirFileList = os.listdir(directory)
# Loop through the individual files within the directory
for filename in dirFileList:
# Check whether file is directory or file
if(os.path.isdir(os.path.join(directory,filename))):
print os.path.join(directory,filename) + \
is a directory and therefore ignored!
elif(os.path.isfile(os.path.join(directory,filename))):
if(self.checkFileExtension(filename, extension)):
fileList.append(os.path.join(directory,filename))
else:
print filename + is NOT a file or directory!
else:
print directory + is not a directory!
else:
print directory + does not exist!
# Return the list of files
return fileList

Note, that you also need the function to check the file extension.
This can then be added to the mergeSHPfiles function with a list to iterate through
the identified files.
1
2
3
4
5
6
7
8

# A function to control the merging of shapefiles

def mergeSHPfiles(self, filePath, newSHPfile):
# Get the list of files within the directory
# provided with the extension .shp
fileList = self.findFilesExt(filePath, .shp)
# Iterate through the files.
for file in fileList:
print(file)

When iterating through the files the ogr2ogr commands to be executed to merge
the shapefiles need to be built and executed therefore the following code needs
to be added to your script. Once the command is built it is called using the
subprocess modules call command.

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

84

# A function to control the merging of shapefiles

def mergeSHPfiles(self, filePath, newSHPfile):
# Get the list of files within the directory
# provided with the extension .shp
fileList = self.findFilesExt(filePath, .shp)
# Variable used to identify the first file
first = True
# A string for the command to be built
command =
# Iterate through the files.
for file in fileList:
if first:
# If the first file make a copy to create the output file
command = ogr2ogr + newSHPfile + + file
first = False
else:
# Otherwise append the current shapefile to the output file
command = ogr2ogr -update -append + newSHPfile + + \
file + -nln + \
self.removeSHPExtension(self.removeFilePathUNIX(newSHPfile))
# Execute the current command
subprocess.call(command,shell=True)

Note, the use of the option shell=True within the subprocess function call, this
means to treat the command as a string and execute as is. While this is fine for
simple scripts for your own use if poses a security risk when using in production
code as it leaves the program open to shell injection attempts. For example, if
the program tool a user provided file name and used this when building a command a user could potentially pass in a malicious command (e.g., rm -fr *;)
rather than a file name and this would be executed. To avoid these problems, by
default subprocess requires a list of command line arguments and shell is set to
False.
You also require the additional functions to remove the shapefile extension (.shp)
and the windows file path, creating the layer name which are given below.
1
2
3

# A function to remove a .shp extension from a file name

def removeSHPExtension(self, name):
# The output file name

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

4
5
6
7
8
9
10
11
12
13

85

outName = name
# Find how many .shp strings are in the current file
# name
count = name.find(.shp, 0, len(name))
# If there are no instances of .shp then -1 will be returned
if not count == -1:
# Replace all instances of .shp with empty string.
outName = name.replace(.shp, , name.count(.shp))
# Return output file name without .shp
return outName

14
15
16
17
18
19
20
21
22
23
24
25
26
27

# A function to remove the file path a file

# (in this case a windows file path)
def removeFilePathWINS(self, name):
# Remove white space (i.e., spaces, tabs)
name = name.strip()
# Count the number of slashs
# A double slash is required because \ is a
# string escape charater.
count = name.count(\\)
# Split string into a list where slashs occurs
nameSegments = name.split(\\, count)
# Return the last item in the list
return nameSegments[count]

28
29
30
31
32
33
34
35
36
37
38

# A function to remove the file path a file

def removeFilePathUNIX(self, name):
# Remove white space (i.e., spaces, tabs)
name = name.strip()
# Count the number of slashs
count = name.count(/)
# Split string into a list where slashs occurs
nameSegments = name.split(/, count)
# Return the last item in the list
return nameSegments[count]

If you wanted to use this script on UNIX (i.e., Linux or Mac OS X) you would
need to use the removeFilePathUNIX as shown while for windows change the code
to use the removeFilePathWINS function such that the double escaped slashes are
used.

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

86

You script should now be complete so execute it on the data provided, within the
TreeCrowns directory. Take time to understand the lines of code which have been
provided and make sure your script works.
1

#! /usr/bin/env python

2
3
4
5
6
7
8
9
10

#######################################
# MergeSHPfiles.py
# A python script to merge shapefiles
# Author: <YOUR NAME>
# Email: <YOUR EMAIL>
# Date: DD/MM/YYYY
# Version: 1.0
#######################################

11
12
13

import os
import subprocess

14
15

class MergeSHPfiles (object):

16
17
18
19
20
21
22
23
24
25
26
27
28
29

# A function to remove a .shp extension from a file name

def removeSHPExtension(self, name):
# The output file name
outName = name
# Find how many .shp strings are in the current file
# name
count = name.find(.shp, 0, len(name))
# If there are no instances of .shp then -1 will be returned
if not count == -1:
# Replace all instances of .shp with empty string.
outName = name.replace(.shp, , name.count(.shp))
# Return output file name without .shp
return outName

30
31
32
33
34
35
36

# A function to remove the file path a file

# (in this case a windows file path)
def removeFilePathWINS(self, name):
# Remove white space (i.e., spaces, tabs)
name = name.strip()
# Count the number of slashs

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

37
38
39
40
41
42
43

87

# A double slash is required because \ is a

# string escape charater.
count = name.count(\\)
# Split string into a list where slashs occurs
nameSegments = name.split(\\, count)
# Return the last item in the list
return nameSegments[count]

44
45
46
47
48
49
50
51
52
53
54

# A function to remove the file path a file

def removeFilePathUNIX(self, name):
# Remove white space (i.e., spaces, tabs)
name = name.strip()
# Count the number of slashs
count = name.count(/)
# Split string into a list where slashs occurs
nameSegments = name.split(/, count)
# Return the last item in the list
return nameSegments[count]

55
56
57
58
59
60
61
62
63
64
65
66
67
68

# A function to test the file extension of a file

def checkFileExtension(self, filename, extension):
# Boolean variable to be returned by the function
foundExtension = False;
# Split the filename into two parts (name + ext)
filenamesplit = os.path.splitext(filename)
# Get the file extension into a varaiable
fileExtension = filenamesplit[1].strip()
# Decide whether extensions are equal
if(fileExtension == extension):
foundExtension = True
# Return result
return foundExtension

69
70
71
72
73
74
75
76
77

# A function which iterates through the directory and checks file extensions
def findFilesExt(self, directory, extension):
# Define a list to store output list of files
fileList = list()
# check whether the current directory exits
if os.path.exists(directory):
# check whether the given directory is a directory

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97

if os.path.isdir(directory):
# list all the files within the directory
dirFileList = os.listdir(directory)
# Loop through the individual files within the directory
for filename in dirFileList:
# Check whether file is directory or file
if(os.path.isdir(os.path.join(directory,filename))):
print(os.path.join(directory,filename) + \
is a directory and therefore ignored!)
elif(os.path.isfile(os.path.join(directory,filename))):
if(self.checkFileExtension(filename, extension)):
fileList.append(os.path.join(directory,filename))
else:
print(filename + is NOT a file or directory!)
else:
print(directory + is not a directory!)
else:
print(directory + does not exist!)
# Return the list of files
return fileList

98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118

# A function to control the merging of shapefiles

def mergeSHPfiles(self, filePath, newSHPfile):
# Get the list of files within the directory
# provided with the extension .shp
fileList = self.findFilesExt(filePath, .shp)
# Variable used to identify the first file
first = True
# A string for the command to be built
command =
# Iterate through the files.
for file in fileList:
if first:
# If the first file make a copy to create the output file
command = ogr2ogr + newSHPfile + + file
first = False
else:
# Otherwise append the current shapefile to the output file
command = ogr2ogr -update -append + newSHPfile + + \
file + -nln + \
self.removeSHPExtension(self.removeFilePathUNIX(newSHPfile))

88

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

89

# Execute the current command

subprocess.call(command,shell=True)

119
120
121
122
123
124
125
126
127

# A function which controls the rest of the script

def run(self):
# Define the input directory
filePath = ./TreeCrowns/
# Define the output file
newSHPfile = Merged_TreeCrowns.shp

128

# Check input file path exists and is a directory

if not os.path.exists(filePath):
print(Filepath does not exist)
elif not os.path.isdir(filePath):
print(Filepath is not a directory!)
else:
# Merge the shapefiles within the filePath
self.mergeSHPfiles(filePath, newSHPfile)

129
130
131
132
133
134
135
136
137
138
139
140
141
142
143

# The start of the code

if __name__ == __main__:
# Make an instance of the class
obj = MergeSHPfiles()
# Call the function run()
obj.run()

7.3

Convert Images to GeoTIFF using GDAL.

The next example will require you to use the script developed above as the basis
for a new script using the command below to convert a directory of images to
GeoTIFF using the command given:
gdal_translate -of <OutputFormat> <InputFile> <OutputFile>

A useful step is to first run the command from the command line manually to
make sure you understand how this command is working.
The two main things you need to think about are:

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

90

1. What file extension will the input files have? This should be user selectable
alongside the file paths.
2. What output file name should be provided? The script should generate this.
Four test images have been provided in ENVI format within the directory ENVI Images,
you can use these for testing your script. If you are struggling then an example script with a solution to this task has been provided within the code directory.

7.3.1

Passing Inputs from the Command Line into your

script

It is often convenient to provide the inputs the scripts requires (e.g., input and
output file locations) as arguments to the script rather than needing to the edit
the script each time a different set of parameters are required (i.e., changing the
files paths in the scripts above). This is easy within python and just requires
the following changes to your run function (in this case for the merge shapefiles
script).
1
2
3
4
5
6
7
8
9
10
11
12
13

# A function which controls the rest of the script

def run(self):
# Get the number of arguments
numArgs = len(sys.argv)
# Check there are only 2 input argument (i.e., the input file
# and output base).
# Note that argument 0 (i.e., sys.argv[0]) is the name
# of the script currently running.
if numArgs == 3:
# Retrieve the input directory
filePath = sys.argv[1]
# Retrieve the output file
newSHPfile = sys.argv[2]

14
15
16
17

# Check input file path exists and is a directory

if not os.path.exists(filePath):
print("Filepath does not exist")

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

18
19
20
21
22
23
24
25

91

elif not os.path.isdir(filePath):

print("Filepath is not a directory!")
else:
# Merge the shapefiles within the filePath
self.mergeSHPfiles(filePath, newSHPfile)
else:
print("ERROR. Command should have the form:")
print("python MergeSHPfiles_cmd.py <Input File Path> <Output File>")

In addition, to these changes you need to import the system library into your
script to access these arguments.
# Import the sys package from within the
# standard library
import sys

Please note that the list of user provided inputs starts at index 1 and not 0. If you
call sys.argv[0] then the name of the script being executed will be returned. When
retrieving values from the user in this form it is highly advisable to check whether
the inputs provided are valid and that all required inputs have been provided.
Create a copy of the script you created earlier and edit the run function to be as
shown above, making note of the lines which require editing.

7.4

Exercises

1. Using ogr2ogr develop a script that will convert the attribute table of a
shapefile to a CSV file which can be opened within Microsoft Excel. Note,
that the outputted CSV will be put into a separate directory.
2. Create a script which calls the gdal translate command and converts all the
images within a directory to a byte data type (i.e., with a range of 0 to 255).

7.5

Further Reading

GDAL - http://www.gdal.org

CHAPTER 7. BATCH PROCESSING COMMAND LINE TOOLS

92

OGR - http://www.gdal.org/ogr
Python Documentation - http://www.python.org/doc
Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7
Learn UNIX in 10 minutes - http://freeengineer.org/learnUNIXin10minutes.
html
The Linux Command Line. W. E. Shotts. No Starch Press. ISBN 978-159327-389-7 (Available to download from http://linuxcommand.org/tlcl.
php)

Chapter 8
Image Processing using GDAL
and RIOS
8.1

Reading and Updating Header Information

Image files used within spatial data processing (i.e., remote sensing and GIS)
require the addition of a spatial header to the files which provides the origin
(usually from the top left corner of the image), the pixel resolution of the image and
a definition of the coordinate system and projection of the dataset. Additionally,
most formats also allow a rotation to be defined. Using these fields the geographic
position on the Earths surface can be defined for each pixel within the scene.
Images can also contain other information in the header of the file including no
data values, image statistics and band names/descriptions.

8.1.1

Reading Image Headers

The GDAL software library provides a python interface to the C++ library, such
that when the python functions are called is it the C++ implementation which is
executed. These model has significant advantages for operations such as reading
and writing to and from image files as in pure python these operations would be
slow but they as very fast within C++. Although, python is an easier language
93

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

94

for people to learn and use, therefore allows software to be more quickly developed
so combing C++ and python in this way is a very productive way for software to
be developed.

Argparser
Up until this point we have read parameters from the system by just using the
sys.argv list where the user is required to enter the values in a given pre-defined
order. The problem with this is that it is not very helpful to the user as no
help is provided or error messages given if the wrong parameters are entered. For
command line tools it is generally accepted that when providing command line
options they will use switches such as -i or input where the user specifies with a
switch what the input they are providing is.
Fortunately, python provides a library to simplify the implementation of this type
of interface. An example of this is shown below, where first the argparse library
is imported. The parser is then created and the arguments added to the parser
so the parser knows what to expect from the user. Finally, the parser is called to
parse the arguments. Examples will be shown in all the following scripts.
1
2

# Import the python Argument parser

import argparse

3
4
5
6
7
8
9
10
11

# Create the parser

parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str, help="Specify the input image file.")
# Define the argument for specifying the output file.
parser.add_argument("-o", "--output", type=str, help="Specify the output text file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

8.1.2

95

Read image header example.

The follow example demonstrates how to import the GDAL library into python
and to read the image header information and print it to the console - similar to
the functionality within the gdalinfo command. Read the comments within the
code and ensure you understand the steps involved.
1

#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the GDAL python library

import osgeo.gdal as gdal
# Import the python Argument parser
import argparse
# Import the System library
import sys

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# Define a function to read and print the images

# header information.
def printImageHeader(inputFile):
# Open the dataset in Read Only mode
dataset = gdal.Open(inputFile, gdal.GA_ReadOnly)
# Check that the dataset has correctly opened
if not dataset is None:
# Print out the image file path.
print(inputFile)
# Print out the number of image bands.
print("The image has ", dataset.RasterCount, " bands.")
# Loop through all the image bands and print out the band name
for n in range(dataset.RasterCount):
print("\t", n+1, ":\t", dataset.GetRasterBand(n+1).GetDescription(), "\t")

24
25
26

# Print out the image size in pixels

print("Image Size [", dataset.RasterXSize, ",", dataset.RasterYSize, "]")

27
28
29
30
31
32
33

#
#
#
#
#
#

Get the geographic header

geotransform[0] = TL X Coordinate
geotransform[1] = X Pixel Resolution
geotransform[2] = X Rotation
geotransform[3] = TL Y Coordinate
geotransform[4] = Y Rotation

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

# geotransform[5] = Y Pixel Resolution

geotransform = dataset.GetGeoTransform()
# Check that the tranformation has been correctly read.
if not geotransform is None:
# Print out the Origin, Pixel Size and Rotation.
print(Origin = (,geotransform[0], ,,geotransform[3],))
print(Pixel Size = (,geotransform[1], ,,geotransform[5],))
print(Rotation = (,geotransform[2], ,,geotransform[4],))
else:
# Provide an error message is the transform has not been
# correctly read.
print("Could not find a geotransform for image file ", inputFile)
else:
# Provide an error message if the input image file
# could not be opened.
print("Could not open the input image file: ", inputFile)

50
51
52
53
54
55
56
57
58
59
60
61
62

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

63
64
65
66
67
68
69
70

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()
# Otherwise, run the function to print out the image header information.
printImageHeader(args.input)

96

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

97

Running the script

Run the script as you have done others within these worksheets and as shown
below, you need to provide the full path to the image file or copy the image file
into the same directory as your script. This should result in an output like the
one shown below:
> python ReadImageHeader.py -i LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img
The image has 6 bands.
1 :
Band 1
2 :
Band 2
3 :
Band 3
4 :
Band 4
5 :
Band 5
6 :
Band 6
Image Size [ 1776 , 1871 ]
Origin = ( 35.2128071515 , -3.05897460167 )
Pixel Size = ( 0.000271352299023 , -0.000271352299023 )
Rotation = ( 0.0 , 0.0 )

8.1.3

No Data Values

GDAL also allows us to edit the image header values, therefore the following
example provides an example of how to edit the no data value for image band.
Note that when opening the image file the gdal.GA Update option is used rather
than gdal.GA ReadOnly.
A no data value is useful for defining regions of the image which are not valid (i.e.,
outside of the image boundaries) and can be ignored during processing.

Running the script

For the file provided (LSTOA Tanz 2000Wet.img) the no data value for all the
bands should be 0. Therefore, run the following command:

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

> python setnodata.py

Setting No data (0.0)
Setting No data (0.0)
Setting No data (0.0)
Setting No data (0.0)
Setting No data (0.0)
Setting No data (0.0)

98

-i LSTOA_Tanz_2000Wet.img -n 0.0
for band 1
for band 2
for band 3
for band 4
for band 5
for band 6

To check that command successfully edited the input file use the gdalinfo command, as shown below:
gdalinfo -norat LSTOA_Tanz_2000Wet.img

#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the GDAL python library

import osgeo.gdal as gdal
# Import the python Argument parser
import argparse
# Import the System library
import sys

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# A function to set the no data value

# for each image band.
def setNoData(inputFile, noDataVal):
# Open the image file, in update mode
# so that the image can be edited.
dataset = gdal.Open(inputFile, gdal.GA_Update)
# Check that the image has been opened.
if not dataset is None:
# Iterate throught he image bands
# Note. i starts at 0 while the
# band count in GDAL starts at 1.
for i in range(dataset.RasterCount):
# Print information to the user on what is
# being set.
print("Setting No data (" + str(noDataVal) + ") for band " + str(i+1))
# Get the image band
# the i+1 is because GDAL bands

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

# start with 1.
band = dataset.GetRasterBand(i+1)
# Set the no data value.
band.SetNoDataValue(noDataVal)

27
28
29
30
31
32
33
34

99

else:
# Print an error message if the file
# could not be opened.
print("Could not open the input image file: ", inputFile)

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying the no data value.
parser.add_argument("-n", "--nodata", type=float, default=0,
help="Specify the no data value to be set.")
# Call the parser to parse the arguments.
args = parser.parse_args()

50
51
52
53
54
55
56
57
58

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: not input image file provided.")
sys.exit()
# Otherwise, run the function to set the no
# data value.
setNoData(args.input, args.nodata)

8.1.4

Band Name

Band names are useful for a user to understand a data set more easily. Therefore,
naming the image bands, such as Blue, Green, Red, NIR and SWIR, is very
useful. The following example illustrates how to edit the band name description

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

using GDAL.
1

#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the GDAL python library

import osgeo.gdal as gdal
# Import the python Argument parser
import argparse
# Import the System library
import sys

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# A function to set the no data value

# for each image band.
def setBandName(inputFile, band, name):
# Open the image file, in update mode
# so that the image can be edited.
dataset = gdal.Open(inputFile, gdal.GA_Update)
# Check that the image has been opened.
if not dataset is None:
# Get the image band
imgBand = dataset.GetRasterBand(band)
# Check the image band was available.
if not imgBand is None:
# Set the image band name.
imgBand.SetDescription(name)
else:
# Print out an error message.
print("Could not open the image band: ", band)
else:
# Print an error message if the file
# could not be opened.
print("Could not open the input image file: ", inputFile)

31
32
33
34
35
36
37
38
39

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,

100

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

40
41
42
43
44
45
46
47
48

101

help="Specify the input image file.")

# Define the argument for specifying image band.
parser.add_argument("-b", "--band", type=int,
help="Specify image band.")
# Define the argument for specifying band name.
parser.add_argument("-n", "--name", type=str,
help="Specify the band name.")
# Call the parser to parse the arguments.
args = parser.parse_args()

49
50
51
52
53
54

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

55
56
57
58
59
60

# Check that the band parameter has been specified.

if args.band == None:
# Print an error message if not and exit.
print("Error: the band was not specified.")
sys.exit()

61
62
63
64
65
66

# Check that the name parameter has been specified.

if args.name == None:
# Print an error message if not and exit.
print("Error: the band name was not specified.")
sys.exit()

67
68
69
70

# Otherwise, run the function to set the band

# name.
setBandName(args.input, args.band, args.name)

Running the script

The file provided (LSTOA Tanz 2000Wet.img) just has some default band names
defined (i.e., Band 1) but use you script to change them to something more useful.
Therefore, run the following commands:

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

python
python
python
python
python
python

setbandname.py
setbandname.py
setbandname.py
setbandname.py
setbandname.py
setbandname.py

-i
-i
-i
-i
-i
-i

LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img

-b
-b
-b
-b
-b
-b

1
2
3
4
5
6

-n
-n
-n
-n
-n
-n

102

Blue
Green
Red
NIR
SWIR1
SWIR2

Use you script for reading the image header values and printing them to the screen
to find out whether it worked.

8.1.5

GDAL Meta-Data

GDAL supports the concept of meta-data on both the image bands and the whole
image. The meta-data allows any other data to be stored within the image file as
a string.
The following example shows how to read the meta-data values and to list all the
meta-data variables available on both the image bands and the image.
1

#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the GDAL python library

import osgeo.gdal as gdal
# Import the python Argument parser
import argparse
# Import the System library
import sys

9
10
11
12
13
14
15
16
17
18
19

# A function to read a meta-data item

# from a image band
def readBandMetaData(inputFile, band, name):
# Open the dataset in Read Only mode
dataset = gdal.Open(inputFile, gdal.GA_ReadOnly)
# Check that the image has been opened.
if not dataset is None:
# Get the image band
imgBand = dataset.GetRasterBand(band)
# Check the image band was available.

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

if not imgBand is None:

# Get the meta-data value specified.
metaData = imgBand.GetMetadataItem(name)
# Check that it is present
if metaData == None:
# If not present, print error.
print("Could not find \", name, "\ item.")
else:
# Else print out the metaData value.
print(name, " = \", metaData, "\")
else:
# Print out an error message.
print("Could not open the image band: ", band)
else:
# Print an error message if the file
# could not be opened.
print("Could not open the input image file: ", inputFile)

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

# A function to read a meta-data item

# from a image
def readImageMetaData(inputFile, name):
# Open the dataset in Read Only mode
dataset = gdal.Open(inputFile, gdal.GA_ReadOnly)
# Check that the image has been opened.
if not dataset is None:
# Get the meta-data value specified.
metaData = dataset.GetMetadataItem(name)
# Check that it is present
if metaData == None:
# If not present, print error.
print("Could not find \", name, "\ item.")
else:
# Else print out the metaData value.
print(name, " = \", metaData, "\")
else:
# Print an error message if the file
# could not be opened.
print("Could not open the input image file: ", inputFile)

58
59
60

# A function to read a meta-data item

# from a image band

103

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

def listBandMetaData(inputFile, band):

# Open the dataset in Read Only mode
dataset = gdal.Open(inputFile, gdal.GA_ReadOnly)
# Check that the image has been opened.
if not dataset is None:
# Get the image band
imgBand = dataset.GetRasterBand(band)
# Check the image band was available.
if not imgBand is None:
# Get the meta-data dictionary
metaData = imgBand.GetMetadata_Dict()
# Check it has entries.
if len(metaData) == 0:
# If it has no entries return
# error message.
print("There is no image meta-data.")
else:
# Otherwise, print out the
# meta-data.
# Loop through each entry.
for metaItem in metaData:
print(metaItem)
else:
# Print out an error message.
print("Could not open the image band: ", band)
else:
# Print an error message if the file
# could not be opened.
print("Could not open the input image file: ", inputFile)

90
91
92
93
94
95
96
97
98
99
100
101

# A function to read a meta-data item

# from a image
def listImageMetaData(inputFile):
# Open the dataset in Read Only mode
dataset = gdal.Open(inputFile, gdal.GA_ReadOnly)
# Check that the image has been opened.
if not dataset is None:
# Get the meta-data dictionary
metaData = dataset.GetMetadata_Dict()
# Check it has entries.
if len(metaData) == 0:

104

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

102
103
104
105
106
107
108
109
110
111
112
113
114

105

# If it has no entries return

# error message.
print("There is no image meta-data.")
else:
# Otherwise, print out the
# meta-data.
# Loop through each entry.
for metaItem in metaData:
print(metaItem)
else:
# Print an error message if the file
# could not be opened.
print("Could not open the input image file: ", inputFile)

115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying image band.
parser.add_argument("-b", "--band", type=int, default=0,
help="Specify image band.")
# Define the argument for specifying meta-data name.
parser.add_argument("-n", "--name", type=str,
help="Specify the meta-data name.")
# Define the argument for specifying whether the
# meta-data field should be just listed.
parser.add_argument("-l", "--list", action="store_true", default=False,
help="Specify that meta data items should be listed.")
# Call the parser to parse the arguments.
args = parser.parse_args()

137
138
139
140
141
142

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

106

143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170

# Check that the name parameter has been specified.

# If it has been specified then run functions to
# read band meta-data.
if not args.name == None:
# Check whether the image band has been specified.
# the default was set at 0 (to indicate that it)
# hasnt been specified as GDAL band count starts
# at 1. This also means the user cannot type in
# a value of 0 and get an error.
if args.band == 0:
# Run the function to print out the image meta-data value.
readImageMetaData(args.input, args.name)
else:
# Otherwise, run the function to print out the band meta-data value.
readBandMetaData(args.input, args.band, args.name)
elif args.list:
if args.band == 0:
# Run the function to list image meta-data.
listImageMetaData(args.input)
else:
# Otherwise, run the function to list band meta-data.
listBandMetaData(args.input, args.band)
else:
# Print an error message if not and exit.
print("Error: the meta-data name or list option" + \
" need to be specified was not specified.")
sys.exit()

Running the script

This script has a number of options. Have a play with these options on the image
provided, an example shown below.
python
python
python
python
python

ReadGDALMetaData.py
ReadGDALMetaData.py
ReadGDALMetaData.py
ReadGDALMetaData.py
ReadGDALMetaData.py

-h
-i
-i
-i
-i

LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img
LSTOA_Tanz_2000Wet.img

-l
-b 1 -l
-b 1 -n LAYER_TYPE
-b 3 -n STATISTICS_MEAN

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

8.2

107

Raster Input / Output Simplification (RIOS)

Library

The raster input and output (I/O) simplification (RIOS) library is a set of python
modules which makes it easier to write raster processing code in Python. Built
on top of GDAL, it handles the details of opening and closing files, checking
alignment of projections and raster grid, stepping through the raster in small
blocks, etc., allowing the programmer to concentrate on implementing the solution
to the problem rather than on how to access the raster data and detail with the
spatial header.
Also, GDAL provides access to the image data through python RIOS makes it
much more user friendly and easier to use. RIOS is available for as a free download
from https://bitbucket.org/chchrsc/rios/overview

8.2.1

Getting Help Reminder

Python provides a very useful help system through the command line. To get
access to the help run python from the terminal
> python

Then import the library want to get help on

>>> import osgeo.gdal

and then run the help tool on the whole module

>>> import osgeo.gdal
>>> help(osgeo.gdal)

or on individual classes within the module

>>> import osgeo.gdal
>>> help(osgeo.gdal.Dataset)

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

108

To exit the help system just press the q key on the keyboard.

8.2.2

Band Maths

Being able to apply equations to combine image bands, images or scale single bands
is a key tool for remote sensing, for example to calibrate Landsat to radiance. The
following examples demonstrate how to do this within the RIOS framework.

8.2.3

Multiply by a constant

The first example just multiples all the image bands by a constant (provided by
the user). The first part of the code reads the users parameters (input file, output
file and scale factor). To use the applier interface within RIOS you need to first
setup the input and output file associations and then any other options required,
in this case the constant for multiplication. Also, the controls object should be
defined to set any other parameters
All processing within RIOS is undertaken on blocks, by default 200 200 pixels
in size. To process the block a applier function needs to be defined (e.g., mutliplyByValue) where the inputs and outputs are passed to the function (these are
the pixel values) and the other arguments object previously defined. The pixel
values are represented as a numpy array, the dimensions are (n, y, x) where n is
the number of image bands, y is the number of rows and x the number of columns
in the block.
Because numpy will iterate through the array for us to multiply the whole array
by a constant (e.g., 2) then we can just need the syntax shown below, which makes
it very simple.
1

#!/usr/bin/env python

2
3
4
5
6

import sys
# Import the python Argument parser
import argparse
# Import the RIOS applier interface

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

7
8

from rios import applier

from rios import cuiprogress

9
10
11
12
13

# Define the applier function

def mutliplyByValue(info, inputs, outputs, otherargs):
# Multiple the image1 by the scale factor
outputs.outimage = inputs.image1 * otherargs.scale

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying the output file.
parser.add_argument("-o", "--output", type=str,
help="Specify the output image file.")
# Define the argument for multiply the image by.
parser.add_argument("-m", "--multiply", default=1.0, type=float,
help="Multiple the image by.")
# Call the parser to parse the arguments.
args = parser.parse_args()

32
33
34
35
36
37

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

38
39
40
41
42
43

# Check that the output parameter has been specified.

if args.output == None:
# Print an error message if not and exit.
print("Error: No output image file provided.")
sys.exit()

44
45
46
47

# Create input files file names associations

infiles = applier.FilenameAssociations()
# Set image1 to the input image specified

109

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

48
49
50
51
52
53
54
55
56
57
58
59
60

110

infiles.image1 = args.input
# Create output files file names associations
outfiles = applier.FilenameAssociations()
# Set outImage to the output image specified
outfiles.outimage = args.output
# Create other arguments object
otherargs = applier.OtherInputs()
# Define the scale arguments
otherargs.scale = args.multiply
# Create a controls objects
aControls = applier.ApplierControls()
# Set the progress object.
aControls.progress = cuiprogress.CUIProgressBar()

61
62
63
64
65
66
67

# Apply the multiply function.

applier.apply(mutliplyByValue,
infiles,
outfiles,
otherargs,
controls=aControls)

68

Run the Script

Run the script using the following command, the input image is a Landsat scene
and all the pixel values will be multiplied by 2.
python MultiplyRIOSExample.py -i LSTOA_Tanz_2000Wet.img \
-o LSTOA_Tanz_2000Wet_Multiby2.img -m 2

8.2.4

Calculate NDVI

To use the image bands independently to calculate a new value, usually indices
such as the NDVI

NDVI =

NIR RED
NIR + RED

(8.1)

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

111

requires that the bands are referenced independently within the input data. Using
numpy to calculate the index, as shown below, results in a single output block with
the dimensions of the block but does not have the third dimension (i.e., the band)
which is required for RIOS to identify how to create the output image. Therefore,
as you will see in the example below an extra dimension needs to be added before
outputting the data to the file. Within the example given the input pixel values
are converted to floating point values (rather than whatever they were inputted as
from the input) because the output will be a floating point number (i.e., an NDVI
have a range of 1 to 1).
1

#!/usr/bin/env python

2
3
4
5
6
7
8
9

# Import the python Argument parser

import argparse
# Import the RIOS applier interface
from rios import applier
from rios import cuiprogress
#
import numpy

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

# Define the applier function

def mutliplyByValue(info, inputs, outputs, otherargs):
# Convert the input data to Float32
# This is because the output is a float due to the
# divide within the NDVI calculation.
inputs.image1 = inputs.image1.astype (numpy.float32)
# Calculate the NDVI for the block.
# Note. Numpy will deal with the image iterating
#
to all the individual pixels values.
#
within python this is very important
#
as python loops are slow.
out = ((inputs.image1[otherargs.nirband]inputs.image1[otherargs.redband])
/
(inputs.image1[otherargs.nirband]+
inputs.image1[otherargs.redband]))
# Add an extra dimension to the output array.
# The output array needs to have 3 dimensions
# (No Bands, Y Pixels(Rows), X Pixels(Cols)

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

30
31
32

# In this case an extra dimension representing

# the single image band is required.
outputs.outimage = numpy.expand_dims(out, axis=0)

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying the output file.
parser.add_argument("-o", "--output", type=str,
help="Specify the output image file.")
# Define the argument for specifying the red image band
parser.add_argument("-r", "--red", type=int,
help="Specifiy red band.")
# Define the argument for specifying the NIR image band
parser.add_argument("-n", "--nir", type=int,
help="Specifiy NIR band.")
# Call the parser to parse the arguments.
args = parser.parse_args()

54
55
56
57
58
59

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

60
61
62
63
64
65

# Check that the output parameter has been specified.

if args.output == None:
# Print an error message if not and exit.
print("Error: No output image file provided.")
sys.exit()

66
67
68
69
70

# Create input files file names associations

infiles = applier.FilenameAssociations()
# Set image1 to the input image specified
infiles.image1 = args.input

112

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

71
72
73
74
75
76
77
78
79
80
81
82
83
84

113

# Create output files file names associations

outfiles = applier.FilenameAssociations()
# Set outImage to the output image specified
outfiles.outimage = args.output
# Create other arguments object
otherargs = applier.OtherInputs()
# Define the red band argument
otherargs.redband = args.red-1
# Define the NIR band argument
otherargs.nirband = args.nir-1
# Create a controls objects
aControls = applier.ApplierControls()
# Set the progress object.
aControls.progress = cuiprogress.CUIProgressBar()

85
86
87
88
89
90
91

# Apply the multiply function.

applier.apply(mutliplyByValue,
infiles,
outfiles,
otherargs,
controls=aControls)

92

Run the Script

Run the script using the following command, the input image is a Landsat scene
so the red band is therefore band 3 and then NIR band is band 4.
python RIOSExampleNDVI.py -i LSTOA_Tanz_2000Wet.img \
-o LSTOA_Tanz_2000Wet_NDVI.img -r 3 -n 4

8.2.5

Calculate NDVI Using Multiple Images

Where multiple input files are required, in this case the NIR and Red bands are
represented by different image files, the input files need to be specified in the input
files association as image1, image2 etc. and the pixel values within the applier

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

114

function are therefore referenced in the same way. Because, in this example the
images only have a single image band the input images has the same dimensions
as the output so no extra dimensions need to be added.
1

#!/usr/bin/env python

2
3
4
5
6
7
8
9
10
11
12

# Import the system library

import sys
# Import the python Argument parser
import argparse
# Import the RIOS applier interface
from rios import applier
# Import the RIOS progress feedback
from rios import cuiprogress
# Import the numpy library
import numpy

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# Define the applier function

def mutliplyByValue(info, inputs, outputs):
# Convert the input data to Float32
# This is because the output is a float due to the
# divide within the NDVI calculation.
inputs.image1 = inputs.image1.astype (numpy.float32)
inputs.image2 = inputs.image2.astype (numpy.float32)
# Calculate the NDVI for the block.
# Note. Numpy will deal with the image iterating
#
to all the individual pixels values.
#
within python this is very important
#
as python loops are slow.
outputs.outimage = ((inputs.image2-inputs.image1)
/
(inputs.image2+inputs.image1))

29
30
31
32
33
34
35
36

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the output file.

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

37
38
39
40
41
42
43
44
45
46

parser.add_argument("-o", "--output", type=str,

help="Specify the output image
# Define the argument for specifying the red image
parser.add_argument("-r", "--red", type=str,
help="Specifiy red input image
# Define the argument for specifying the NIR image
parser.add_argument("-n", "--nir", type=str,
help="Specifiy NIR input image
# Call the parser to parse the arguments.
args = parser.parse_args()

file.")
band
file.")
band
file.")

47
48
49
50
51
52

# Check that the red input parameter has been specified.

if args.red == None:
# Print an error message if not and exit.
print("Error: No red input image file provided.")
sys.exit()

53
54
55
56
57
58

# Check that the NIR input parameter has been specified.

if args.red == None:
# Print an error message if not and exit.
print("Error: No NIR input image file provided.")
sys.exit()

59
60
61
62
63
64

# Check that the output parameter has been specified.

if args.output == None:
# Print an error message if not and exit.
print("Error: No output image file provided.")
sys.exit()

65
66
67
68
69
70
71
72
73
74
75
76
77

# Create input files file names associations

infiles = applier.FilenameAssociations()
# Set images to the input image specified
infiles.image1 = args.red
infiles.image2 = args.nir
# Create output files file names associations
outfiles = applier.FilenameAssociations()
# Set outImage to the output image specified
outfiles.outimage = args.output
# Create a controls objects
aControls = applier.ApplierControls()
# Set the progress object.

115

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

78

116

aControls.progress = cuiprogress.CUIProgressBar()

79
80
81
82
83
84

# Apply the multiply function.

applier.apply(mutliplyByValue,
infiles,
outfiles,
controls=aControls)

85

Run the Script

Run the script using the following command, the input image is a Landsat scene
so the red band is therefore band 3 and then NIR band is band 4.
python RIOSExampleMultiFileNDVI.py -o LSTOA_Tanz_2000Wet_MultiIn_NDVI.img \
-r LSTOA_Tanz_2000Wet_Red.img -n LSTOA_Tanz_2000Wet_NIR.img

8.3

Filtering Images

To filtering an image is done through a windowing operation where the windows

of pixels, such as a 3 3 or 5 5 (it needs to be an odd number), are selected and
a new value for the centre pixel is calculated using all the pixel values within the
window. In this example a median filter will be used so the middle pixel value will
be replaced with the median value of the window.
Scipy (http://www.scipy.org) is another library of python functions, which is
paired with numpy, and provides many useful functions we can use when processing
the images or other datasets within python. The ndimage module (http://docs.
scipy.org/doc/scipy/reference/tutorial/ndimage.html) provides many useful functions, which can be applied to images in the same way as the median filter
has been used in the example below I strongly recommend you look through
the documentation of scipy to get an idea of the types of functions which are
available.

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

#!/usr/bin/env python

2
3
4
5
6
7
8
9
10
11
12
13

import sys
# Import the python Argument parser
import argparse
# Import the scipy filters.
from scipy import ndimage
#Import the numpy library
import numpy
# Import the RIOS image reader
from rios.imagereader import ImageReader
# Import the RIOS image writer
from rios.imagewriter import ImageWriter

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

# Define the function to iterate through

# the image.
def applyMedianFilter(inputFile, outputFile, fSize):
# Get half the filter size, overlap between blocks
hSize = (fSize-1)/2
# Create the image reader for the input file
# and set the overlap to be half the image
# filter size.
reader = ImageReader(inputFile, overlap=hSize)
# Define the image writer but cannot create
# until within the loop as this need the
# information within the info object.
writer = None
# Loop through all the image blocks within
# the reader.
for (info, block) in reader:
# Create an output block of
# the same size as the input
out = numpy.zeros_like(block)
# Iterate through the image bands
for i in range(len(out)):
# Use scipy to run a median filter
# on the image band data. The image
# bands are filtered in turn
ndimage.median_filter(block[i],size=fSize,output=out[i])
# If it is the first time through the loop
# (i.e., writer has a value of None) then

117

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

42
43
44
45
46
47
48
49
50
51
52
53
54
55

# create the
if writer is
# Create
writer =

loop.
None:
the writer for output image.
ImageWriter(outputFile,
info=info,
firstblock=out,
drivername=HFA)

else:
# If the writer is created write the
# output block to the file.
writer.write(out)
# Close the writer and calculate
# the image statistics.
writer.close(calcStats=True)

56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying the output file.
parser.add_argument("-o", "--output", type=str,
help="Specify the output image file.")
# Define the argument for the size of the image filter.
parser.add_argument("-s", "--size", default=3, type=int,
help="Filter size.")
# Call the parser to parse the arguments.
args = parser.parse_args()

74
75
76
77
78
79

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

80
81
82

# Check that the output parameter has been specified.

if args.output == None:

118

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

119

# Print an error message if not and exit.

print("Error: No output image file provided.")
sys.exit()

83
84
85
86
87
88
89

# Call the function to execute a median filter

# on the input image.
applyMedianFilter(args.input, args.output, args.size)

90

Run the Script

python MedianFilterRIOSExample.py -i LSTOA_Tanz_2000Wet.img \

-o LSTOA_Tanz_2000Wet_median7.img -s 7

After you have run this command open the images in TuiView and flick between
them to observe the change in the image, what do you notice?

8.4

Apply a rule based classification

Another option we have is to use the where function within numpy to select
pixel corresponding to certain criteria (i.e., pixels with an NDVI < 0.2 is not
vegetation) and classify them accordingly where a pixel values are used to indicate
the corresponding class (e.g., 1 = Forest, 2 = Water, 3 = Grass, etc). These images
where pixel values are not continuous but categories are referred to as thematic
images and there is a header value that can be set to indicate this type of image.
Therefore, in the script below there is a function for setting the image band metadata field LAYER TYPE to be thematic. Setting an image as thematic means
that the nearest neighbour algorithm will be used when calculating pyramids and
histograms needs to be binned with single whole values. It also means that a
colour table (See Chapter 9) can also be added.
To build the rule base the output pixel values need to be created, here using the
numpy function zeros (http://docs.scipy.org/doc/numpy/reference/generated/
numpy.zeros.html). The function zeros creates a numpy array of the requested

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

120

shape (in this case the shape is taken from the inputted image) where all the pixels
have a value of zero.
Using the where function (http://docs.scipy.org/doc/numpy/reference/generated/
numpy.where.html) a logic statement can be applied to an array or set of arrays
(which must be of the same size) to select the pixels for which the statement is
true. The where function returns an array of indexes which can be used to address
another array (i.e., the output array) and set a suitable output value (i.e., the
classification code).
1

#!/usr/bin/env python

2
3
4
5
6
7
8
9
10
11
12
13
14

# Import the system library

import sys
# Import the python Argument parser
import argparse
# Import the RIOS applier interface
from rios import applier
# Import the RIOS progress feedback
from rios import cuiprogress
# Import the numpy library
import numpy
# Import the GDAL library
from osgeo import gdal

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# Define the applier function

def rulebaseClassifier(info, inputs, outputs):
# Create an output array with the same dims
# as a single band of the input file.
out = numpy.zeros(inputs.image1[0].shape)
# Use where statements to select the
# pixels to be classified. Give them a
# integer value (i.e., 1, 2, 3, 4) to
# specify the class.
out[numpy.where((inputs.image1[0] > 0.4 )&(inputs.image1[0] < 0.7))] = 1
out[numpy.where(inputs.image1[0] < 0.1 )] = 2
out[numpy.where((inputs.image1[0] > 0.1 )&(inputs.image1[0] < 0.4))] = 3
out[numpy.where(inputs.image1[0] > 0.7 )] = 4
# Expand the output array to include a single
# image band and set as the output dataset.

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

31

outputs.outimage = numpy.expand_dims(out, axis=0)

32
33
34
35
36
37
38
39
40
41
42

# A function to define the image as thematic

def setThematic(imageFile):
# Use GDAL to open the dataset
ds = gdal.Open(imageFile, gdal.GA_Update)
# Iterate through the image bands
for bandnum in range(ds.RasterCount):
# Get the image band
band = ds.GetRasterBand(bandnum + 1)
# Define the meta-data for the LAYER_TYPE
band.SetMetadataItem(LAYER_TYPE, thematic)

43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying the output file.
parser.add_argument("-o", "--output", type=str,
help="Specify the output image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

58
59
60
61
62
63

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

64
65
66
67
68
69

# Check that the output parameter has been specified.

if args.output == None:
# Print an error message if not and exit.
print("Error: No output image file provided.")
sys.exit()

70
71

# Create input files file names associations

121

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

72
73
74
75
76
77
78
79
80
81
82
83
84

122

infiles = applier.FilenameAssociations()
# Set image1 to the input image specified
infiles.image1 = args.input
# Create output files file names associations
outfiles = applier.FilenameAssociations()
# Set outImage to the output image specified
outfiles.outimage = args.output
# Create a controls objects
aControls = applier.ApplierControls()
# Specify that stats shouldnt be calcd
aControls.calcStats = False
# Set the progress object.
aControls.progress = cuiprogress.CUIProgressBar()

85
86
87
88
89
90

# Apply the classifier function.

applier.apply(rulebaseClassifier,
infiles,
outfiles,
controls=aControls)

91
92
93

# Set the output file to be thematic

setThematic(args.output)

Run the Script

Run the script with one of the NDVI layers you previously calculated. To see the
result then it is recommended that a colour table is added (see next worksheet), the
easiest way to do that is to use the gdalcalcstats command, as shown below.
python RuleBaseClassification.py -i LSTOA_Tanz_2000Wet_NDVI.img \
-o LSTOA_Tanz_2000Wet_classification.img
# Run gdalcalcstats to add a random colour table
gdalcalcstats LSTOA_Tanz_2000Wet_classification.img

CHAPTER 8. IMAGE PROCESSING USING GDAL AND RIOS

8.5

123

Exercises

1. Create rule based classification using multiple image bands.

2. Create a rule based classification using image bands from different input
images.
3. Using the previous work sheet as a basis create a script which calls the
gdalwarp command to resample an input image to the same pixel resolution
as another image, where the header is read as shown in this work sheet.

8.6

Further Reading

GDAL - http://www.gdal.org
Python Documentation - http://www.python.org/doc
Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7
Learn UNIX in 10 minutes - http://freeengineer.org/learnUNIXin10minutes.
html
SciPy http://www.scipy.org/SciPy
NumPy http://numpy.scipy.org
RIOS https://bitbucket.org/chchrsc/rios/wiki/Home

Chapter 9
Raster Attribute Tables (RAT)
The RIOS software also allows raster attribute tables to be read and written
through GDAL. Raster attribute tables (RAT) are similar the the attribute tables
which are present on a vector (e.g., shapefile). Each row of the attribute table
refers to a pixel value within the image (e.g., row 0 refers to all pixels with a value
of 0). Therefore, RATs are used within thematic datasets were pixels values are
integers and refer to a category, such as a class from a classification, or a spatial
region, such as a segment from a segmentation. The columns of the RAT therefore
refer to variables, which correspond to information associated with the spatial
region cover by the image pixels of the clump(s) relating to the row within the
attribute table.

9.1

Reading Columns

To access the RAT using RIOS, you need to import the rat module. The RAT
module provides a simple interface for reading and writing columns. When a
column is read it is returned as a numpy array where the size is n 1 (i.e., the
number of rows in the attribute table).
As shown in the example below, a reading a column is just a single function call
specifying the input image file and the column name.

124

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the system library

import sys
# Import the RIOS rat library.
from rios import rat
# Import the python Argument parser
import argparse

9
10
11
12
13
14
15
16

# A function for reading the RAT

def readRatCol(imageFile, colName):
# Use RIOS to read the column name
# The contents of the column are
# printed to the console for the
# user to see.
print(rat.readColumn(imageFile, colName))

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying the column name.
parser.add_argument("-n", "--name", type=str,
help="Specify the column name.")
# Call the parser to parse the arguments.
args = parser.parse_args()

32
33
34
35
36
37
38
39
40
41

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()
# Check that the input parameter has been specified.
if args.name == None:
# Print an error message if not and exit.
print("Error: No RAT column name provided.")

125

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

126

sys.exit()

42
43
44
45
46

# Run the function read and print the

# RAT column.
readRatCol(args.input, args.name)

Run the Script

Run the script as follow, the example below prints the Histogram but use TuiView
to see what other columns are within the attribute table.
python ReadRATColumn.py -i WV2_525N040W_2m_segments.kea -n Histogram

9.2

Writing Columns

Writing a column is also quite straight forward just requiring a n 1 numpy array
with the the data to be written to the output file, the image file path and the
name of the column to be written to.

9.2.1

Calculating New Columns

The first example reads a column from the input image and just multiples it by 2
and writes it to the image file as a new column.
1

#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the system library

import sys
# Import the RIOS rat library.
from rios import rat
# Import the python Argument parser
import argparse

9
10

# The applier function to multiply the input

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

11
12
13
14
15
16
17
18

# column by 2.
def multiplyRATCol(imageFile, inColName, outColName):
# Read the input column
col = rat.readColumn(imageFile, inColName)
# Muliply the column by 2.
col = col * 2
# Write the output column to the file.
rat.writeColumn(imageFile, outColName, col)

19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Define the argument for specifying the input column name.
parser.add_argument("-c", "--inname", type=str,
help="Specify the input column name.")
# Define the argument for specifying the output column name.
parser.add_argument("-o", "--outname", type=str,
help="Specify the output column name.")
# Call the parser to parse the arguments.
args = parser.parse_args()

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()
# Check that the input parameter has been specified.
if args.inname == None:
# Print an error message if not and exit.
print("Error: No input RAT column name provided.")
sys.exit()
# Check that the input parameter has been specified.
if args.outname == None:
# Print an error message if not and exit.
print("Error: No output RAT column name provided.")

127

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

128

sys.exit()

52
53
54
55

# Otherwise, run the function to multiply the input column by 2.

multiplyRATCol(args.input, args.inname, args.outname)

Run the Script

Run the script as follows, in this simple case the histogram will be multiplied by
2 and saved as a new column.
python MultiplyColumn.py -i WV2_525N040W_2m_segments.kea -c Histogram -o HistoMulti2

9.2.2

Add Class Name

A useful column to have within the attribute table, where a classification has been
undertaken, is class names. This allows a user to click on the image and rather
than having to remember which codes correspond to which class they will be shown
a class name.
To add class names to the attribute table a new column needs to be created, where
the data type is set to be ASCII (string). To do this a copy of the histogram column
is made where the new numpy array is empty, of type string and the same length
at the histogram.
The following line using the ... syntax within the array index to specify all elements
of the array, such that they are all set to a value of NA.
Once the new column has been created then the class names can be simply defined
through referencing the appropriate array index.
1

#!/usr/bin/env python

2
3
4
5
6

# Import the system library

import sys
# Import the RIOS rat library.
from rios import rat

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

7
8
9
10

# Import the python Argument parser

import argparse
# Import the numpy library
import numpy

11
12
13
14
15
16
17
18
19
20
21
22
23

# A function to add a colour table.

def addClassNames(imageFile):
histo = rat.readColumn(imageFile, "Histogram")
className = numpy.empty_like(histo, dtype=numpy.dtype(a255))
className[...] = "NA"
className[0] = "Other Vegetation"
className[1] = "Low Woody Vegetation"
className[2] = "Water"
className[3] = "Sparse Vegetation"
className[4] = "Tall Woody Vegetation"
# Write the output column to the file.
rat.writeColumn(imageFile, "ClassNames", className)

24
25
26
27
28
29
30
31
32
33
34
35

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

36
37
38
39
40
41

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

42
43
44

# Run the add class names function

addClassNames(args.input)

129

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

130

Run the Script

Run the script as follows, use the classification you did at the end of worksheet 8.
python AddClassNames.py -i LSTOA_Tanz_2000Wet_classification.img

9.3

Adding a colour table

Another useful tool is being able to add a colour table to an image, such that classes
are displayed in colours appropriate to make interpretation easier. To colour up
the per pixel classification undertake at the end of the previous exercise and given
class names using the previous scripts the following script is used to add a colour
table.
The colour table is represented as an n 5 dimensional array, where n is the
number of colours which are to be present within the colour table.
The 5 values associated with each colour are
1. Image Pixel Value
2. Red (0 255)
3. Green (0 255)
4. Blue (0 255)
5. Opacity (0 255)
Where an opacity of 0 means completely transparent and 255 means solid with
no transparency (opacity is something also referred to as alpha or alpha channel).
1

#!/usr/bin/env python

2
3
4

# Import the system library

import sys

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

5
6
7
8
9
10

# Import the RIOS rat library.

from rios import rat
# Import the python Argument parser
import argparse
# Import the numpy library
import numpy

11
12
13
14
15
16
17
18

# A function to add a colour table.

def addColourTable(imageFile):
# Create a colour table (n,5) where
# n is the number of classes to be
# coloured. The data type must be
# of type integer.
ct = numpy.zeros([5,5], dtype=numpy.int)

19
20
21
22
23
24
25

# Set 0 to
ct[0][0] =
ct[0][1] =
ct[0][2] =
ct[0][3] =
ct[0][4] =

be Dark Mustard Yellow.

0
# Pixel Val
139 # Red
139 # Green
0
# Blue
255 # Opacity

# Set 1 to
ct[1][0] =
ct[1][1] =
ct[1][2] =
ct[1][3] =
ct[1][4] =

be Dark Olive Green.

1
# Pixel Val
162 # Red
205 # Green
90 # Blue
255 # Opacity

# Set 2 to
ct[2][0] =
ct[2][1] =
ct[2][2] =
ct[2][3] =
ct[2][4] =

be Royal Blue.
2
# Pixel Val
72 # Red
118 # Green
255 # Blue
255 # Opacity

# Set 3 to
ct[3][0] =
ct[3][1] =
ct[3][2] =
ct[3][3] =

be Dark Sea Green.

3
# Pixel Val
180 # Red
238 # Green
180 # Blue

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

131

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

46

132

ct[3][4] = 255 # Opacity

47
48
49
50
51
52
53

# Set 4 to
ct[4][0] =
ct[4][1] =
ct[4][2] =
ct[4][3] =
ct[4][4] =

be Forest Green.
4
# Pixel Val
34 # Red
139 # Green
34 # Blue
255 # Opacity

54
55

rat.setColorTable(imageFile, ct)

56
57
58
59
60
61
62
63
64
65
66
67
68
69

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

70
71
72
73
74
75

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

76
77
78

# Run the add colour table function

addColourTable(args.input)

Run the Script

Run the script as follows, use the classification you did at the end of worksheet 8.

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

133

python AddClassNames.py -i LSTOA_Tanz_2000Wet_classification.img

To find the Red, Green and Blue (RGB) values to use with the colour table there
are many websites available only that provide lists of these colours (e.g., http:
//cloford.com/resources/colours/500col.htm).

9.4

Using RATs for rule based classifications.

To use a RAT to undertake a rule based object oriented classification the first
step is to create a set of image clumps (e.g., through segmentation see appendix A
section A.3), then the rows of the attribute table need populating with information
(e.g., see appendix A section A.4). Once these steps have been completed then a
rule base using the numpy where statements can be created and executed, resulting
in a similar process as the eCognition software.

9.4.1

Developing a rule base

This is a similar process to developing a rule based classification within eCognition,

where the clumps are the segments/objects and the columns are the features, such
as mean, standard deviation etc.
Using where statements, similar to those used within the rule based classification
of the image pixels, the clumps can be classified. The example, shown below,
illustrates the classification of the Landcover classification system (LCCS) levels
1 to 3. Where the classes are represented by string names within the class names
column within the attribute table.
The classification is undertaken using three dates of WorldView2 imagery captured over Cors Fochno, in Wales UK. A segmentation has been provided and the
segments have been populated with mean reflectance values from the three WorldView2 images, the DTM minimum and maximum and the CHM height.
1
2

#!/usr/bin/env python

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

3
4
5
6
7
8
9
10
11
12

134

# Import the system library

import sys
# Import the rat modele from RIOS
from rios import rat
# Import the numpy library
import numpy
# Import the gdal library
import osgeo.gdal as gdal
# Import the python Argument parser
import argparse

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# Thresholds which have been globally defined

# so they can easily be changed and therefore
# change everywhere they are used.
URBAN_MASK_THRES = 0.05
CULT_MASK_THRES = 0.05
WBI_PRE_THRES = 1
WBI_PEAK_THRES = 1
WBI_POST_THRES = 1
FDI_PRE_THRES = -100
FDI_PEAK_THRES = -40
FDI_POST_THRES = -115
PSRI_PRE_THRES = -0.2
REP_PEAK_THRES = -0.005
WOODY_PRE_THRES_BG = 0.09
WOODY_PEAK_THRES_CG = 0.195

29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

# A function for classifying the first part of level 1

def classifyLevel1FromImg(urbanMask, wbiPeak, fdiPeak, wbiPost,
fdiPost, wbiPre, fdiPre, psriPre, repPeak):
# Create Output Array
l1P1 = numpy.empty_like(urbanMask, dtype=numpy.dtype(a255))
l1P1[...] = "NA"
# Urban
l1P1 = numpy.where(numpy.logical_and(l1P1 == "NA", urbanMask > URBAN_MASK_THRES),
"Urban", l1P1)
# Water
l1P1 = numpy.where(numpy.logical_and(l1P1 == "NA",
numpy.logical_or(wbiPre >= WBI_PRE_THRES,
wbiPeak >= WBI_PEAK_THRES)),
"Water", l1P1)

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

135

# Photosynthetic Vegetation
l1P1 = numpy.where(numpy.logical_and(l1P1 == "NA",
numpy.logical_or(fdiPeak > FDI_PEAK_THRES,
fdiPost > FDI_POST_THRES)),
"Photosynthetic Vegetated", l1P1)
# Non PhotoSynthetic Vegetation
l1P1 = numpy.where(numpy.logical_and(l1P1 == "NA",
psriPre >= PSRI_PRE_THRES),
"Non Photosynthetic Vegetated", l1P1)
# Non Submerged Aquatic Veg
l1P1 = numpy.where(numpy.logical_and(l1P1 == "NA",
numpy.logical_and(repPeak >= REP_PEAK_THRES,
wbiPost <= WBI_POST_THRES)),
"Non Submerged Aquatic Vegetated", l1P1)
return l1P1

59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76

# A function for classifying the second part of level 1

def classifyLevel1Assign(classLevel1Img):
# Create Output Array
level1 = numpy.empty_like(classLevel1Img, dtype=numpy.dtype(a255))
level1[...] = "NA"
# Non Vegetated
level1 = numpy.where(numpy.logical_or(classLevel1Img == "NA",
numpy.logical_or(classLevel1Img == "Water",
classLevel1Img == "Urban")),
"Non Vegetated", level1)
# Vegetated
level1 = numpy.where(numpy.logical_or(
classLevel1Img == "Photosynthetic Vegetated",
classLevel1Img == "Non Photosynthetic Vegetated",
classLevel1Img == "Non Submerged Aquatic Vegetated"),
"Vegetated", level1)
return level1

77
78
79
80
81
82

# A function for classifying level 2

def classifyLevel2(wbiPre, wbiPeak, wbiPost, classLevel1Img):
# Create Output Array
level2 = numpy.empty_like(classLevel1Img, dtype=numpy.dtype(a255))
level2[...] = "NA"

83
84

# Terrestrial Non Vegetated

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

136

level2 = numpy.where(numpy.logical_or(classLevel1Img == "NA",

classLevel1Img == "Urban"),
"Terrestrial Non Vegetated", level2)
# Aquatic Non Vegetated
level2 = numpy.where(numpy.logical_and(
numpy.logical_not(classLevel1Img == "Urban"),
numpy.logical_or(wbiPre > 1, wbiPeak > 1)),
"Aquatic Non Vegetated", level2)
# Terrestrial Vegetated
level2 = numpy.where(numpy.logical_or(classLevel1Img == "Photosynthetic Vegetated",
classLevel1Img == "Non Photosynthetic Vegetated"),
"Terrestrial Vegetated", level2)
# Aquatic Vegetated
level2 = numpy.where(classLevel1Img == "Non Submerged Aquatic Vegetated",
"Aquatic Vegetated", level2)
return level2

101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125

# A function for classifying level 3

def classifyLevel3(classLevel2, cult, urban):
# Create Output Array
level3 = numpy.empty_like(classLevel2, dtype=numpy.dtype(a255))
level3[...] = "NA"
# Cultivated Terrestrial Vegetated
level3 = numpy.where(numpy.logical_and(
classLevel2 == "Terrestrial Vegetated", cult > CULT_MASK_THRES),
"Cultivated Terrestrial Vegetated", level3)
# Natural Terrestrial Vegetated
level3 = numpy.where(numpy.logical_and(numpy.logical_not
(level3 == "Cultivated Terrestrial Vegetated"),
classLevel2 == "Terrestrial Vegetated"),
"Natural Terrestrial Vegetated", level3)
# Cultivated Aquatic Vegetated
level3 = numpy.where(numpy.logical_and(classLevel2 == "Aquatic Vegetated",
cult > CULT_MASK_THRES), "Cultivated Aquatic Vegetated", level3)
# Natural Aquatic Vegetated
level3 = numpy.where(numpy.logical_and(numpy.logical_not
(level3 == "Cultivated Aquatic Vegetated"),
classLevel2 == "Aquatic Vegetated"),
"Natural Aquatic Vegetated", level3)
# Artificial Surface
level3 = numpy.where(numpy.logical_and(classLevel2 == "Terrestrial Non Vegetated",

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

126
127
128
129
130
131
132
133
134
135

urban > URBAN_MASK_THRES), "Artificial Surface", level3)

# Natural Surface
level3 = numpy.where(numpy.logical_and(numpy.logical_not
(level3 == "Artificial Surface"),
classLevel2 == "Terrestrial Non Vegetated"),
"Natural Surface", level3)
# Natural Water
level3 = numpy.where(classLevel2 == "Aquatic Non Vegetated",
"Natural Water", level3)
return level3

136
137
138
139
140
141
142
143
144
145
146
147
148

def runClassification(fname):
# Open the GDAL Dataset so it is just opened once
# and reused rather than each rios call reopening
# the image file which will large attribute tables
# can be slow.
ratDataset = gdal.Open( fname, gdal.GA_Update )
# Check the image file was openned correctly.
if not ratDataset == None:
# Provide feedback to the user.
print("Import Columns.")
urban = rat.readColumn(ratDataset, "PropUrban")
cult = rat.readColumn(ratDataset, "PropCult")

149
150
151
152
153
154
155

# Read in the RAT columns for the Pre-Flush image

PreCoastal = rat.readColumn(ratDataset, "MarB1")
PreBlue = rat.readColumn(ratDataset, "MarB2")
PreRed = rat.readColumn(ratDataset, "MarB5")
PreRedEdge = rat.readColumn(ratDataset, "MarB6")
PreNIR1 = rat.readColumn(ratDataset, "MarB7")

156
157
158
159
160
161
162
163

# Read in the RAT columns for the Peak-flush image.

PeakCoastal = rat.readColumn(ratDataset, "JulyB1")
PeakBlue = rat.readColumn(ratDataset, "JulyB2")
PeakRed = rat.readColumn(ratDataset, "JulyB5")
PeakRedEdge = rat.readColumn(ratDataset, "JulyB6")
PeakNIR1 = rat.readColumn(ratDataset, "JulyB7")
PeakNIR2 = rat.readColumn(ratDataset, "JulyB8")

164
165
166

137

# Read in the RAT columns for the Post-flush image.

PostCoastal = rat.readColumn(ratDataset, "NovB1")

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

167
168
169

PostBlue = rat.readColumn(ratDataset, "NovB2")

PostRedEdge = rat.readColumn(ratDataset, "NovB6")
PostNIR1 = rat.readColumn(ratDataset, "NovB7")

170
171
172
173
174
175
176
177
178
179

# Provide more feedback to the user.

print("Calculate Indices.")
# As all the columns are numpy arrays then
# we can do numpy arithmatic between the
# arrays to calculate new arrays, such as
# indices.
wbiPre = PreBlue/PreNIR1
fdiPre = PreNIR1 - (PreRedEdge + PreCoastal)
psriPre = (PreRed - PreBlue)/PreRedEdge

180
181
182
183

wbiPeak = PeakBlue/PeakNIR1
fdiPeak = PeakNIR1 - (PeakRedEdge + PeakCoastal)
repPeak = PeakRedEdge - (PeakNIR2 - PeakRed)

184
185
186

wbiPost = PostBlue/PostNIR1
fdiPost = PostNIR1 - (PostRedEdge + PostCoastal)

187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207

# Call the function which classifies the first part

# of the level 1 classification
print("Classifying Level 1")
classLevel1Img = classifyLevel1FromImg(urban,
wbiPeak,
fdiPeak,
wbiPost,
fdiPost,
wbiPre,
fdiPre,
psriPre,
repPeak)
# Write the first part of the level 1 classification
# back into the input file.
rat.writeColumn(ratDataset, "ClassLevel1Part1", classLevel1Img)
# Call function a produce the level 1 classification
classLevel1 = classifyLevel1Assign(classLevel1Img)
# Write the level 1 classification to the image
rat.writeColumn(ratDataset, "ClassLevel1", classLevel1)

138

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

208
209
210
211
212

# Call the function which classifies the level 2 of the classification.

print("Classifying Level 2")
classLevel2 = classifyLevel2(wbiPre, wbiPeak, wbiPost, classLevel1Img)
# Write the level 2 classification to the image.
rat.writeColumn(ratDataset, "ClassLevel2", classLevel2)

213
214
215
216
217
218
219
220

# Call the function which classifies level 3 of the classification

print("Classifying Level 3")
classLevel3 = classifyLevel3(classLevel2, cult, urban)
# Write the level 3 classification to the image.
rat.writeColumn(ratDataset, "ClassLevel3", classLevel3)
else:
print("Image could not be openned")

221
222
223
224
225
226
227
228
229
230
231
232

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

233
234
235
236
237
238

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

239
240
241

139

# Run the classification

runClassification(args.input)

Run the Classification

To run the classification use the following command:

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

140

python LCCS_L13_Classification.py -i WV2_525N040W_2m_segments.kea

Colour the classification

Following the classification, the clusters need to be coloured and the script for this
shown below. The previous example of adding a colour table is not suited to this
case as colours are being applied to the individual segments based on their class
allocation.
1

#!/usr/bin/env python

2
3
4
5
6
7
8
9
10
11
12

# Import the system library

import sys
# import the rat module from rios
from rios import rat
# import numpy
import numpy
# import gdal
import osgeo.gdal as gdal
# Import the python Argument parser
import argparse

13
14
15
16
17
18
19
20
21
22
23
24
25
26

# A function for
def colourLevel3(classLevel3):
# Create the empty output arrays and set them
# so they all have a value of 0 other than
# opacity which is 255 to create solid colours
level3red = numpy.empty_like(classLevel3, dtype=numpy.int)
level3red[...] = 0
level3green = numpy.empty_like(classLevel3, dtype=numpy.int)
level3green[...] = 0
level3blue = numpy.empty_like(classLevel3, dtype=numpy.int)
level3blue[...] = 0
level3alpha = numpy.empty_like(classLevel3, dtype=numpy.int)
level3alpha[...] = 255

27
28
29

# For segmentation of class NA set them to be black

level3red = numpy.where(classLevel3 == "NA", 0, level3red)

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

30
31
32

141

level3green = numpy.where(classLevel3 == "NA", 0, level3green)

level3blue = numpy.where(classLevel3 == "NA", 0, level3blue)
level3alpha = numpy.where(classLevel3 == "NA", 255, level3alpha)

33
34
35
36
37
38
39
40
41
42

# Colour Cultivated Terrestrial Vegetated

level3red = numpy.where(classLevel3 == "Cultivated Terrestrial Vegetated",
192, level3red)
level3green = numpy.where(classLevel3 == "Cultivated Terrestrial Vegetated",
255, level3green)
level3blue = numpy.where(classLevel3 == "Cultivated Terrestrial Vegetated",
0, level3blue)
level3alpha = numpy.where(classLevel3 == "Cultivated Terrestrial Vegetated",
255, level3alpha)

43
44
45
46
47
48
49
50
51
52

# Colour Natural Terrestrial Vegetated

level3red = numpy.where(classLevel3 == "Natural Terrestrial Vegetated",
0, level3red)
level3green = numpy.where(classLevel3 == "Natural Terrestrial Vegetated",
128, level3green)
level3blue = numpy.where(classLevel3 == "Natural Terrestrial Vegetated",
0, level3blue)
level3alpha = numpy.where(classLevel3 == "Natural Terrestrial Vegetated",
255, level3alpha)

53
54
55
56
57
58
59
60
61
62

# Colour Cultivated Aquatic Vegetated

level3red = numpy.where(classLevel3 == "Cultivated Aquatic Vegetated",
0, level3red)
level3green = numpy.where(classLevel3 == "Cultivated Aquatic Vegetated",
255, level3green)
level3blue = numpy.where(classLevel3 == "Cultivated Aquatic Vegetated",
255, level3blue)
level3alpha = numpy.where(classLevel3 == "Cultivated Aquatic Vegetated",
255, level3alpha)

63
64
65
66
67
68
69
70

# Colour Natural Aquatic Vegetated

level3red = numpy.where(classLevel3 == "Natural Aquatic Vegetated",
0, level3red)
level3green = numpy.where(classLevel3 == "Natural Aquatic Vegetated",
192, level3green)
level3blue = numpy.where(classLevel3 == "Natural Aquatic Vegetated",
122, level3blue)

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

71
72

level3alpha = numpy.where(classLevel3 == "Natural Aquatic Vegetated",

255, level3alpha)

73
74
75
76
77
78
79
80
81
82

# Colour Artificial Surface

level3red = numpy.where(classLevel3 == "Artificial Surface",
255, level3red)
level3green = numpy.where(classLevel3 == "Artificial Surface",
0, level3green)
level3blue = numpy.where(classLevel3 == "Artificial Surface",
255, level3blue)
level3alpha = numpy.where(classLevel3 == "Artificial Surface",
255, level3alpha)

83
84
85
86
87
88
89
90
91
92

# Colour Natural Surface

level3red = numpy.where(classLevel3 == "Natural Surface",
255, level3red)
level3green = numpy.where(classLevel3 == "Natural Surface",
192, level3green)
level3blue = numpy.where(classLevel3 == "Natural Surface",
160, level3blue)
level3alpha = numpy.where(classLevel3 == "Natural Surface",
255, level3alpha)

93
94
95
96
97
98
99
100
101
102

# Colour Artificial Water

level3red = numpy.where(classLevel3 == "Artificial Water",
0, level3red)
level3green = numpy.where(classLevel3 == "Artificial Water",
0, level3green)
level3blue = numpy.where(classLevel3 == "Artificial Water",
255, level3blue)
level3alpha = numpy.where(classLevel3 == "Artificial Water",
255, level3alpha)

103
104
105
106
107
108
109
110
111

# Colour Natural Water

level3red = numpy.where(classLevel3 == "Natural Water",
0, level3red)
level3green = numpy.where(classLevel3 == "Natural Water",
0, level3green)
level3blue = numpy.where(classLevel3 == "Natural Water",
255, level3blue)
level3alpha = numpy.where(classLevel3 == "Natural Water",

142

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

112

143

255, level3alpha)

113
114

return level3red, level3green, level3blue, level3alpha

115
116
117
118
119
120
121
122
123
124
125

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str,
help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

126
127
128
129
130
131

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

132
133
134

# Open the input file using GDAL

ratDataset = gdal.Open(args.input, gdal.GA_Update)

135
136
137
138
139
140
141

# Check that is openned correctly

if not ratDataset == None:
# Print some user feedback
print("Import Columns.")
# Read the classification column
level3 = rat.readColumn(ratDataset, "ClassLevel3")

142
143
144
145
146
147
148
149
150
151
152

# Print some user feedback

print("Classifying Level 3")
# Call function to assign colours to arrays
level3red, level3green, level3blue, level3alpha = colourLevel3(level3)
# Write the values to the Output Columns
rat.writeColumn(ratDataset, "Red", level3red)
rat.writeColumn(ratDataset, "Green", level3green)
rat.writeColumn(ratDataset, "Blue", level3blue)
rat.writeColumn(ratDataset, "Alpha", level3alpha)
else:

CHAPTER 9. RASTER ATTRIBUTE TABLES (RAT)

144

# Print an error message to the user if the image

# file could not be openned.
print("Input Image could not be opened")

153
154
155

Run the script using the following command.

python LCCS_L13_ColourClassification.py -i WV2_525N040W_2m_segments.kea

9.5

Exercises

9.6

Further Reading

GDAL - http://www.gdal.org
Python Documentation - http://www.python.org/doc
Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7
Learn UNIX in 10 minutes - http://freeengineer.org/learnUNIXin10minutes.
html
SciPy http://www.scipy.org/SciPy
NumPy http://numpy.scipy.org
RIOS https://bitbucket.org/chchrsc/rios/wiki/Home

Chapter 10
Golden Plover Population
Model
10.1

Introduction

The aim of this work sheet is to develop a populate model for a bird, called the
Golden Plover.

10.2

Model Output

The model is required to output the total population of the birds for each year
and the number of bird, eggs, fledgling and the number of fledglings which are
a year old. Providing an option to export the results as a plot should also be
provided.

10.3

Reading Parameters

To allow a user to parameterise the model a parameter card, such as the one shown
below, needs to be provided.

145

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

1
2
3
4
5
6
7
8
9
10

numOfYears=20
initalAdultPairPop=15
winterSurvivalRate=0.66
averageEggsPerPair=3.64
averageFledgelingsPerPair=3.2
predatorControl=False
numOfFledgelings=14
numOfFledgelingsYearOld=8
fledgelingsSurvivePredatorsCtrl=0.75
fledgelingsSurvivePredatorsNoCtrl=0.18
#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the system library

import sys
# Import the python Argument parser
import argparse
# Import the maths library
import math as math

9
10
11

# A class for the golden plover population model

class GoldenPloverPopModel (object):

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# A function to parse the input parameters file.

def parseParameterFile(self, inputFile):
# A string to store the input parameters to
# be outputted in the output file.
paramsStr = "## Input Parameters to the model.\n"
# Open the input parameter file.
parameterFile = open(inputFile, r)
# Create a dictionary object to store the
# input parameters.
params = dict()
# Loop through each line of the input
# text file.
for line in parameterFile:
# Strip any white space either
# side of the text.
line = line.strip()
# Add the line to the output
# parameters file.

146

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

paramsStr += "# " + line + "\n"

# Split the line on the = symbol
paramVals = line.split("=", 1)
# Find the known parameters and
# convert the input string to the
# correct data type (i.e., float or int).
if paramVals[0] == "numOfYears":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "initalAdultPairPop":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "winterSurvivalRate":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageEggsPerPair":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageFledgelingsPerPair":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "predatorControl":
if paramVals[1].lower() == "false":
params[paramVals[0]] = False
elif paramVals[1].lower() == "true":
params[paramVals[0]] = True
else:
print("predatorControl must be either True or False.")
sys.exit()
elif paramVals[0] == "numOfFledgelings":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "numOfFledgelingsYearOld":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsCtrl":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsNoCtrl":
params[paramVals[0]] = float(paramVals[1])
else:
# If parameter is not known then just store as
# a string.
params[paramVals[0]] = paramVals[1]
# Return the parameters and parameters string
return params, paramsStr

69
70
71

# The run function controlling the overall order

# of when things run.

147

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

72
73
74
75
76
77
78

148

def run(self, inputFile):

# Provide user feedback to the user.
print("Parse Input File.")
# Call the function to parse the input file.
params, paramsStr = self.parseParameterFile(inputFile)
# Print he parameters.
print(params)

79
80
81
82
83
84
85
86
87
88
89
90

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str, help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

91
92
93
94
95
96

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

97
98
99

obj = GoldenPloverPopModel()
obj.run(args.input)

10.4
1

The Model

#!/usr/bin/env python

2
3
4
5
6
7

# Import the system library

import sys
# Import the python Argument parser
import argparse
# Import the maths library

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

import math as math

9
10
11

# A class for the golden plover population model

class GoldenPloverPopModel (object):

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

# A function to parse the input parameters file.

def parseParameterFile(self, inputFile):
# A string to store the input parameters to
# be outputted in the output file.
paramsStr = "## Input Parameters to the model.\n"
# Open the input parameter file.
parameterFile = open(inputFile, r)
# Create a dictionary object to store the
# input parameters.
params = dict()
# Loop through each line of the input
# text file.
for line in parameterFile:
# Strip any white space either
# side of the text.
line = line.strip()
# Add the line to the output
# parameters file.
paramsStr += "# " + line + "\n"
# Split the line on the = symbol
paramVals = line.split("=", 1)
# Find the known parameters and
# convert the input string to the
# correct data type (i.e., float or int).
if paramVals[0] == "numOfYears":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "initalAdultPairPop":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "winterSurvivalRate":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageEggsPerPair":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageFledgelingsPerPair":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "predatorControl":
if paramVals[1].lower() == "false":

149

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

params[paramVals[0]] = False
elif paramVals[1].lower() == "true":
params[paramVals[0]] = True
else:
print("predatorControl must be either True or False.")
sys.exit()
elif paramVals[0] == "numOfFledgelings":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "numOfFledgelingsYearOld":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsCtrl":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsNoCtrl":
params[paramVals[0]] = float(paramVals[1])
else:
# If parameter is not known then just store as
# a string.
params[paramVals[0]] = paramVals[1]
# Return the parameters and parameters string
return params, paramsStr

69
70
71
72
73
74
75
76
77

# A function in which the model is implemented.

def runGPModel(self, params):
# Set up some local variables - to be editted as
# the model runs.
numOfAdultsPairs = params[initalAdultPairPop]
numOfFledgelingsYearOld = params[numOfFledgelingsYearOld]
numOfFledgelings = params[numOfFledgelings]
numOfEggs = 0

78
79
80
81
82
83
84

# Create lists for the information to be outputted.

numOfAdultsPairsOut = list()
numYearOldFledgelingsOut = list()
numOfEggsOut = list()
numOfFledgelingsOut = list()
numOfFledgelingsB4PredOut = list()

85
86
87
88
89

# The main model loop - looping through the years.

for year in range(params[numOfYears]):
# Append the output parameters at the start of the year.
numOfAdultsPairsOut.append(numOfAdultsPairs)

150

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

90
91

151

numYearOldFledgelingsOut.append(numOfFledgelingsYearOld)
numOfFledgelingsOut.append(numOfFledgelings)

92
93
94
95
96
97

# Get the number of pairs (assuming all adults

# are paired.
numOfAdultsPairs += (numOfFledgelingsYearOld/2)
# Set the number of year old fledgelings
numOfFledgelingsYearOld = numOfFledgelings

98
99
100
101
102

# Get the number of adults and fledgelings following winter.

# Based on their winter survival rate.
numOfAdultsPairs=int(numOfAdultsPairs*params[winterSurvivalRate])
numOfFledgelingsYearOld=int(numOfFledgelingsYearOld*params[winterSurvivalRate])

103
104
105
106
107

# Get the numbers of eggs to hatch

numOfEggs = int(numOfAdultsPairs * params[averageEggsPerPair])
# Append to output list.
numOfEggsOut.append(numOfEggs)

108
109
110
111
112

# Get the number of new fledgelings.

numOfFledgelings = int(numOfAdultsPairs * params[averageFledgelingsPerPair])
# Append to output.
numOfFledgelingsB4PredOut.append(numOfFledgelings)

113
114
115
116
117
118
119
120
121

# Apply fledgeling survival rate with an option to

# apply predator control (or not)
if params[predatorControl]:
# With predator control
numOfFledgelings=int(numOfFledgelings*params[fledgelingsSurvivePredatorsCtrl])
else:
# Without predator control
numOfFledgelings=int(numOfFledgelings*params[fledgelingsSurvivePredatorsNoCtrl]

122
123
124

# Once the model has completed return the output variables for analysis.
return numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEggsOut, numOfFledgelingsOut,

125
126
127
128
129
130

# The run function controlling the overall order

# of when things run.
def run(self, inputFile):
# Provide user feedback to the user.
print("Parse Input File.")

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

152

# Call the function to parse the input file.

params, paramsStr = self.parseParameterFile(inputFile)
# Print he parameters.
print(params)
# Provide some progress feedback to the user
print("Run the model")
# Run the model and get the output parameters.
numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEggsOut, numOfFledgelingsOut, numOfFl

131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str, help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

150
151
152
153
154
155

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

156
157
158
159
160

# Create instance of the model class

obj = GoldenPloverPopModel()
# Call the run function to execute the model.
obj.run(args.input)

10.5
1

Exporting Data

#!/usr/bin/env python

2
3
4
5

# Import the system library

import sys
# Import the python Argument parser

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

6
7
8

import argparse
# Import the maths library
import math as math

9
10
11

# A class for the golden plover population model

class GoldenPloverPopModel (object):

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# A function to parse the input parameters file.

def parseParameterFile(self, inputFile):
# A string to store the input parameters to
# be outputted in the output file.
paramsStr = "## Input Parameters to the model.\n"
# Open the input parameter file.
parameterFile = open(inputFile, r)
# Create a dictionary object to store the
# input parameters.
params = dict()
# Loop through each line of the input
# text file.
for line in parameterFile:
# Strip any white space either
# side of the text.
line = line.strip()
# Add the line to the output
# parameters file.
paramsStr += "# " + line + "\n"
# Split the line on the = symbol
paramVals = line.split("=", 1)
# Find the known parameters and
# convert the input string to the
# correct data type (i.e., float or int).
if paramVals[0] == "numOfYears":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "initalAdultPairPop":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "winterSurvivalRate":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageEggsPerPair":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageFledgelingsPerPair":
params[paramVals[0]] = float(paramVals[1])

153

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

elif paramVals[0] == "predatorControl":

if paramVals[1].lower() == "false":
params[paramVals[0]] = False
elif paramVals[1].lower() == "true":
params[paramVals[0]] = True
else:
print("predatorControl must be either True or False.")
sys.exit()
elif paramVals[0] == "numOfFledgelings":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "numOfFledgelingsYearOld":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsCtrl":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsNoCtrl":
params[paramVals[0]] = float(paramVals[1])
else:
# If parameter is not known then just store as
# a string.
params[paramVals[0]] = paramVals[1]
# Return the parameters and parameters string
return params, paramsStr

69
70
71
72
73
74
75
76
77

# A function in which the model is implemented.

def runGPModel(self, params):
# Set up some local variables - to be editted as
# the model runs.
numOfAdultsPairs = params[initalAdultPairPop]
numOfFledgelingsYearOld = params[numOfFledgelingsYearOld]
numOfFledgelings = params[numOfFledgelings]
numOfEggs = 0

78
79
80
81
82
83
84

# Create lists for the information to be outputted.

numOfAdultsPairsOut = list()
numYearOldFledgelingsOut = list()
numOfEggsOut = list()
numOfFledgelingsOut = list()
numOfFledgelingsB4PredOut = list()

85
86
87

# The main model loop - looping through the years.

for year in range(params[numOfYears]):

154

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

88
89
90
91

155

# Append the output parameters at the start of the year.

numOfAdultsPairsOut.append(numOfAdultsPairs)
numYearOldFledgelingsOut.append(numOfFledgelingsYearOld)
numOfFledgelingsOut.append(numOfFledgelings)

92
93
94
95
96
97

# Get the number of pairs (assuming all adults

# are paired.
numOfAdultsPairs += (numOfFledgelingsYearOld/2)
# Set the number of year old fledgelings
numOfFledgelingsYearOld = numOfFledgelings

98
99
100
101
102

# Get the number of adults and fledgelings following winter.

# Based on their winter survival rate.
numOfAdultsPairs=int(numOfAdultsPairs*params[winterSurvivalRate])
numOfFledgelingsYearOld=int(numOfFledgelingsYearOld*params[winterSurvivalRate])

103
104
105
106
107

# Get the numbers of eggs to hatch

numOfEggs = int(numOfAdultsPairs * params[averageEggsPerPair])
# Append to output list.
numOfEggsOut.append(numOfEggs)

108
109
110
111
112

# Get the number of new fledgelings.

numOfFledgelings = int(numOfAdultsPairs * params[averageFledgelingsPerPair])
# Append to output.
numOfFledgelingsB4PredOut.append(numOfFledgelings)

113
114
115
116
117
118
119
120
121

# Apply fledgeling survival rate with an option to

# apply predator control (or not).
if params[predatorControl]:
# With predator control
numOfFledgelings=int(numOfFledgelings*params[fledgelingsSurvivePredatorsCtrl])
else:
# Without predator control
numOfFledgelings=int(numOfFledgelings*params[fledgelingsSurvivePredatorsNoCtrl]

122
123
124

# Once the model has completed return the output variables for analysis.
return numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEggsOut, numOfFledgelingsOut,

125
126
127
128

# A function to write the results to a text file

# for analysis or visualisation within another
# package.

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155

def writeResultsFile(self, outputFile, paramStr, params, numOfAdultsPairsOut, numYearOldFledg

# Open the output file for writing.
outFile = open(outputFile, w)
# Write the input parameters (the string formed
# when we read the input parameters in. This is
# useful as it will allow someone to understand
# where these outputs came from
outFile.write(paramStr)
# Write a header indicating the following is
# the model outputs.
outFile.write("\n\n## Output Results.\n")
# Create a string for each row of the output
# file. Each row presents a parameter.
yearStrs = "Year"
numOfAdultsStrs = "NumberOfAdultsPairs"
numOfYearOldFledgesStrs = "NumberOfYearOldFledgelings"
numOfFledgesStrs = "NumberOfFledgelings"
numOfFledgesB4PredStrs = "NumberOfFledgelingsB4Preds"
numOfEggsStrs = "NumberOfEggs"
# Loop through each year, building the output strings.
for year in range(params[numOfYears]):
yearStrs += "," + str(year)
numOfAdultsStrs += "," + str(numOfAdultsPairsOut[year])
numOfYearOldFledgesStrs += "," + str(numYearOldFledgelingsOut[year])
numOfFledgesStrs += "," + str(numOfFledgelingsOut[year])
numOfFledgesB4PredStrs += "," + str(numOfFledgelingsB4PredOut[year])
numOfEggsStrs += "," + str(numOfEggsOut[year])

156
157
158
159
160
161
162
163

# Add a new line character to the end of each row.

yearStrs += "\n"
numOfAdultsStrs += "\n"
numOfYearOldFledgesStrs += "\n"
numOfFledgesStrs += "\n"
numOfFledgesB4PredStrs += "\n"
numOfEggsStrs += "\n"

164
165
166
167
168
169

156

# Write the rows to the output file.

outFile.write(yearStrs)
outFile.write(numOfAdultsStrs)
outFile.write(numOfFledgesStrs)
outFile.write(numOfFledgesB4PredStrs)

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

170
171

157

outFile.write(numOfFledgesStrs)
outFile.write(numOfEggsStrs)

172
173
174

# Close the output file.

outFile.close()

175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193

# The run function controlling the overall order

# of when things run.
def run(self, inputFile):
# Provide user feedback to the user.
print("Parse Input File.")
# Call the function to parse the input file.
params, paramsStr = self.parseParameterFile(inputFile)
# Print he parameters.
print(params)
# Provide some progress feedback to the user
print("Run the model")
# Run the model and get the output parameters.
numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEggsOut, numOfFledgelingsOut, numOfFl
# Provide some feedback to the user.
print("Write the results to an output file")
# Call the function to write the outputs
# to a text file.
self.writeResultsFile(cmdargs.outputFile, paramsStr, params, numOfAdultsPairsOut, numYear

194
195
196
197
198
199
200
201
202
203
204

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str, help="Specify the input image file.")
# Call the parser to parse the arguments.
args = parser.parse_args()

205
206
207
208
209
210

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

211
212
213
214
215

# Create instance of the model class

obj = GoldenPloverPopModel()
# Call the run function to execute the model.
obj.run(args.input)

10.6
1

Creating Plots

#!/usr/bin/env python

2
3
4
5
6
7
8

# Import the system library

import sys
# Import the python Argument parser
import argparse
# Import the maths library
import math as math

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# A function to test whether a module

# is present.
def module_exists(module_name):
# Using a try block will
# catch the exception thrown
# if the module is not
# available
try:
# Try to import module.
__import__(module_name)
# Catch the Import error.
except ImportError:
# Return false because
# the module could not
# be imported
return False
else:
# The module was successfully
# imported so return true.
return True

158

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

31
32

# A class for the golden plover population model

class GoldenPloverPopModel (object):

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

# A function to parse the input parameters file.

def parseParameterFile(self, inputFile):
# A string to store the input parameters to
# be outputted in the output file.
paramsStr = "## Input Parameters to the model.\n"
# Open the input parameter file.
parameterFile = open(inputFile, r)
# Create a dictionary object to store the
# input parameters.
params = dict()
# Loop through each line of the input
# text file.
for line in parameterFile:
# Strip any white space either
# side of the text.
line = line.strip()
# Add the line to the output
# parameters file.
paramsStr += "# " + line + "\n"
# Split the line on the = symbol
paramVals = line.split("=", 1)
# Find the known parameters and
# convert the input string to the
# correct data type (i.e., float or int).
if paramVals[0] == "numOfYears":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "initalAdultPairPop":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "winterSurvivalRate":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageEggsPerPair":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "averageFledgelingsPerPair":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "predatorControl":
if paramVals[1].lower() == "false":
params[paramVals[0]] = False
elif paramVals[1].lower() == "true":

159

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

params[paramVals[0]] = True
else:
print("predatorControl must be either True or False.")
sys.exit()
elif paramVals[0] == "numOfFledgelings":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "numOfFledgelingsYearOld":
params[paramVals[0]] = int(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsCtrl":
params[paramVals[0]] = float(paramVals[1])
elif paramVals[0] == "fledgelingsSurvivePredatorsNoCtrl":
params[paramVals[0]] = float(paramVals[1])
else:
# If parameter is not known then just store as
# a string.
params[paramVals[0]] = paramVals[1]
# Return the parameters and parameters string
return params, paramsStr

90
91
92
93
94
95
96
97
98

# A function in which the model is implemented.

def runGPModel(self, params):
# Set up some local variables - to be editted as
# the model runs.
numOfAdultsPairs = params[initalAdultPairPop]
numOfFledgelingsYearOld = params[numOfFledgelingsYearOld]
numOfFledgelings = params[numOfFledgelings]
numOfEggs = 0

99
100
101
102
103
104
105

# Create lists for the information to be outputted.

numOfAdultsPairsOut = list()
numYearOldFledgelingsOut = list()
numOfEggsOut = list()
numOfFledgelingsOut = list()
numOfFledgelingsB4PredOut = list()

106
107
108
109
110
111
112

# The main model loop - looping through the years.

for year in range(params[numOfYears]):
# Append the output parameters at the start of the year.
numOfAdultsPairsOut.append(numOfAdultsPairs)
numYearOldFledgelingsOut.append(numOfFledgelingsYearOld)
numOfFledgelingsOut.append(numOfFledgelings)

160

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

161

113
114
115
116
117
118

# Get the number of pairs (assuming all adults

# are paired.
numOfAdultsPairs += (numOfFledgelingsYearOld/2)
# Set the number of year old fledgelings
numOfFledgelingsYearOld = numOfFledgelings

119
120
121
122
123

# Get the number of adults and fledgelings following winter.

# Based on their winter survival rate.
numOfAdultsPairs=int(numOfAdultsPairs*params[winterSurvivalRate])
numOfFledgelingsYearOld=int(numOfFledgelingsYearOld*params[winterSurvivalRate])

124
125
126
127
128

# Get the numbers of eggs to hatch

numOfEggs = int(numOfAdultsPairs * params[averageEggsPerPair])
# Append to output list.
numOfEggsOut.append(numOfEggs)

129
130
131
132
133

# Get the number of new fledgelings.

numOfFledgelings = int(numOfAdultsPairs * params[averageFledgelingsPerPair])
# Append to output.
numOfFledgelingsB4PredOut.append(numOfFledgelings)

134
135
136
137
138
139
140
141
142

# Apply fledgeling survival rate with an option to

# apply predator control or not
if params[predatorControl]:
# With predator control
numOfFledgelings=int(numOfFledgelings*params[fledgelingsSurvivePredatorsCtrl])
else:
# Without predator control
numOfFledgelings=int(numOfFledgelings*params[fledgelingsSurvivePredatorsNoCtrl]

143
144
145

# Once the model has completed return the output variables for analysis.
return numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEggsOut, numOfFledgelingsOut,

146
147
148
149
150
151
152
153

# A function to write the results to a text file

# for analysis or visualisation within another
# package.
def writeResultsFile(self, outputFile, paramStr, params, numOfAdultsPairsOut, numYearOldFledg
# Open the output file for writing.
outFile = open(outputFile, w)
# Write the input parameters (the string formed

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176

# when we read the input parameters in. This is

# useful as it will allow someone to understand
# where these outputs came from
outFile.write(paramStr)
# Write a header indicating the following is
# the model outputs.
outFile.write("\n\n## Output Results.\n")
# Create a string for each row of the output
# file. Each row presents a parameter.
yearStrs = "Year"
numOfAdultsStrs = "NumberOfAdultsPairs"
numOfYearOldFledgesStrs = "NumberOfYearOldFledgelings"
numOfFledgesStrs = "NumberOfFledgelings"
numOfFledgesB4PredStrs = "NumberOfFledgelingsB4Preds"
numOfEggsStrs = "NumberOfEggs"
# Loop through each year, building the output strings.
for year in range(params[numOfYears]):
yearStrs += "," + str(year)
numOfAdultsStrs += "," + str(numOfAdultsPairsOut[year])
numOfYearOldFledgesStrs += "," + str(numYearOldFledgelingsOut[year])
numOfFledgesStrs += "," + str(numOfFledgelingsOut[year])
numOfFledgesB4PredStrs += "," + str(numOfFledgelingsB4PredOut[year])
numOfEggsStrs += "," + str(numOfEggsOut[year])

177
178
179
180
181
182
183
184

# Add a new line character to the end of each row.

yearStrs += "\n"
numOfAdultsStrs += "\n"
numOfYearOldFledgesStrs += "\n"
numOfFledgesStrs += "\n"
numOfFledgesB4PredStrs += "\n"
numOfEggsStrs += "\n"

185
186
187
188
189
190
191
192

# Write the rows to the output file.

outFile.write(yearStrs)
outFile.write(numOfAdultsStrs)
outFile.write(numOfFledgesStrs)
outFile.write(numOfFledgesB4PredStrs)
outFile.write(numOfFledgesStrs)
outFile.write(numOfEggsStrs)

193
194

162

# Close the output file.

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

195

163

outFile.close()

196
197
198
199
200
201
202
203
204
205
206
207
208
209

def plots(self, outputFile, params, numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEggsO

# Test that the matplotlib library is present
# so plots can be created.
if module_exists("matplotlib.pyplot"):
# The matplotlib library exists so
# import it for use in this function.
# Importing a library within a function
# like this means it is only
# available within this function.
import matplotlib.pyplot as plt
# Get the number of years as a list
# of years.
years = range(params[numOfYears])

210
211
212
213
214
215
216
217
218

# Create a simple plot for the number of

# pairs.
fig1 = plt.figure(figsize=(15, 5), dpi=150)
plt.plot(years, numOfAdultsPairsOut)
plt.title("Number of pairs per year predicted by model")
plt.xlabel("Year")
plt.ylabel("Number Of Pairs")
plt.savefig((outputFile+"_adultpairs.pdf"), format=PDF)

219
220
221
222
223
224
225
226
227

# Create a simple plot for the number of

# year old fledgelings
fig2 = plt.figure(figsize=(15, 5), dpi=150)
plt.plot(years, numYearOldFledgelingsOut)
plt.title("Number of year olf fledgelings predicted by model")
plt.xlabel("Year")
plt.ylabel("Number Of Fledglings")
plt.savefig((outputFile+"_numYearOldFledgelings.pdf"), format=PDF)

228
229
230
231
232
233
234
235

# Create a simple plot for the number of

# eggs hatched each year.
fig3 = plt.figure(figsize=(15, 5), dpi=150)
plt.plot(years, numOfEggsOut)
plt.title("Number of eggs per year predicted by model")
plt.xlabel("Year")
plt.ylabel("Number Of Eggs")

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

236

164

plt.savefig((outputFile+"_numOfEggs.pdf"), format=PDF)

237
238
239
240
241
242
243
244
245

# Create a simple plot for the number of

# new born fledgelings
fig4 = plt.figure(figsize=(15, 5), dpi=150)
plt.plot(years, numOfFledgelingsOut)
plt.title("Number of fledgelings per year predicted by model")
plt.xlabel("Year")
plt.ylabel("Number Of Fledgelings")
plt.savefig((outputFile+"_numOfFledgelings.pdf"), format=PDF)

246
247
248
249
250
251
252
253
254

# Create a simple plot for the number of

# fledgelings before that years breeding
fig5 = plt.figure(figsize=(15, 5), dpi=150)
plt.plot(years, numOfFledgelingsB4PredOut)
plt.title("Number of fledgelings before breeding per year predicted by model")
plt.xlabel("Year")
plt.ylabel("Number Of Fledgelings")
plt.savefig((outputFile+"_numOfFledgelingsB4Pred.pdf"), format=PDF)

255
256
257
258
259

else:
# If the matplotlib library is not available
# print out a suitable error message.
print("Matplotlib is not available and therefore the plots cannot be created.")

260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276

# The run function controlling the overall order

# of when things run.
def run(self, inputFile, outputFile, plotsPath):
# Provide user feedback to the user.
print("Parse Input File.")
# Call the function to parse the input file.
params, paramsStr = self.parseParameterFile(inputFile)
# Print he parameters.
print(params)
# Provide some progress feedback to the user
print("Run the model")
# Run the model and get the output parameters.
numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEggsOut, numOfFledgelingsOut, numOfFl
# Provide some feedback to the user.
print("Write the results to an output file")
# Call the function to write the outputs

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

277
278
279
280
281
282
283
284
285
286

165

# to a text file.
self.writeResultsFile(outputFile, paramsStr, params, numOfAdultsPairsOut, numYearOldFledg
# Check whether a path has been provided
# for the plots. If it has then generate
# output plots.
if plotsPath is not None:
# Give the user feedback of whats happenign.
print("Generating plots of the results")
# Call the function to generate plots
self.plots(plotsPath, params, numOfAdultsPairsOut, numYearOldFledgelingsOut, numOfEgg

287
288
289
290
291
292
293
294
295
296
297
298
299
300
301

# This is the first part of the script to

# be executed.
if __name__ == __main__:
# Create the command line options
# parser.
parser = argparse.ArgumentParser()
# Define the argument for specifying the input file.
parser.add_argument("-i", "--input", type=str, help="Specify the input image file.")
# Define the argument for specifying the output file.
parser.add_argument("-o", "--output", type=str, help="Specify the output text file.")
# Define the argument for specifying the output file.
parser.add_argument("-p", "--plot", type=str, help="Specify the output base path for the plot
# Call the parser to parse the arguments.
args = parser.parse_args()

302
303
304
305
306
307

# Check that the input parameter has been specified.

if args.input == None:
# Print an error message if not and exit.
print("Error: No input image file provided.")
sys.exit()

308
309
310
311
312
313

# Check that the input parameter has been specified.

if args.output == None:
# Print an error message if not and exit.
print("Error: No output text file provided.")
sys.exit()

314
315
316
317

# Create instance of the model class

obj = GoldenPloverPopModel()
# Call the run function to execute the model.

CHAPTER 10. GOLDEN PLOVER POPULATION MODEL

318

166

obj.run(args.input, args.output, args.plot)

10.7

Exercises

10.8

Further Reading

Python Documentation - http://www.python.org/doc

Core Python Programming (Second Edition), W.J. Chun. Prentice Hall
ISBN 0-13-226993-7

Appendix A
RSGISLib

A.1

Introduction to RSGISLib

The remote sensing and GIS software library (RSGISLib) was developed at Aberystwyth University by Pete Bunting and Daniel Clewley. Development started in
April 2008 and has been actively maintained and added to ever since. For more
information see http://www.rsgislib.org.

A.2

Using RSGISLib

RSGISLib has an XML interface and Python bindings. The python bindings
allows the RSGISLib functions to be called from python scripts, as with other
functions.

167

APPENDIX A. RSGISLIB

A.3

168

Segmentation

The segmentation algorithm (?) is based on generating spectrally similar units

with a minimum object size.
The algorithm consists of a number of steps
1. Select image bands and stack images
2. Stretch image data
3. Find unique cluster within feature space (KMeans)
4. Assign pixels to clusters
5. Clump the image
6. Eliminate small segments
The KMeans clustering takes just a single image where all the bands are used
as input so if multiple images are required to be inputted then they need to be
stacked and the bands which are to be used selected. As a Euclidean distance is
used within the feature space the image is stretched such that all the pixel values
are within the same range (i.e., 0255).
A clustering algorithm is then used to identify the unique colours within the image,
in this case a KMeans clustering is used but other clustering algorithms could also
be used instead. The image pixels are then assigned to the clusters (classifying the
image) and the image clumped to find the connected regions of the image.
The final step is an iterative elimination of the small segments, starting with the
single pixels and going up to the maximum size of the segments specified by the
user.

APPENDIX A. RSGISLIB

169

Therefore, there are two key parameters within the algorithm:

1. the number of cluster centres identified by the KMeans clustering (-k)
2. the minimum size of the segments (-n)
There are a number of python functions required to perform the segmentation, an
example python script is provided below:
1

#! /usr/bin/env python

2
3

############################################################################

10

11

# Purpose:

12

13

# Author: Pete Bunting

14

# Email: pfb@aber.ac.uk

15

# Date: 24/07/2013

16

# Version: 1.0

17

18

# History:

19

# Version 1.0 - Created.

20

21

############################################################################

RSGISLibSegmentation.py

Copyright 2013 RSGISLib.

RSGISLib: The remote sensing and GIS Software Library

Execute the Shepherd et al segmentation algorithm.

22
23

# Import the python Argument parser

APPENDIX A. RSGISLIB

24

import argparse

25

# Import the rsgislib module

26

import rsgislib

27

# Import the image utilities module from rsgislib

28

import rsgislib.imageutils

29

# Import the image calc module from rsgislib

30

import rsgislib.imagecalc

31

# Import the image segmentation module from rsgislib

32

import rsgislib.segmentation

33

# Import the image rastergis module from rsgislib

34

import rsgislib.rastergis

35

# Import the module from rsgislib

36

import rsgislib

37

import os.path

38

import os

39

# Import the collections module

40

import collections

41

import fnmatch

170

42
43

class RSGISLibSegmentation (object):

44
45

def runShepherdSegmentation(self, inputImg, outputClumps, outputMeanImg, tmpath,

46

gdalFormat, noStats, noStretch, noDelete, numClusters, minPxls, distThres):

47

rsgisUtils = rsgislib.RSGISPyUtils()

48
49

basefile = os.path.basename(inputImg)

50

basename = os.path.splitext(basefile)[0]

51
52

outFileExt = rsgisUtils.getFileExtension(gdalFormat)

53
54

createdDIR = False

APPENDIX A. RSGISLIB

55

171

if not os.path.isdir(tmpath):

56

os.makedirs(tmpath)

57

createdDIR = True

58
59

# Stretch input data if required.

60

segmentFile = inputImg

61

if not noStretch:

62
63
64
65
66
67
68
69

segmentFile = os.path.join(tmpath,basename+
str("_stchd")+outFileExt)
strchFile = os.path.join(tmpath,basename+
str("_stchdonly")+outFileExt)
strchFileOffset = os.path.join(tmpath,basename+
str("_stchdonlyOff")+outFileExt)
strchMaskFile = os.path.join(tmpath,basename+
str("_stchdmaskonly")+outFileExt)

70

print("Stretch Input Image")

71

rsgislib.imageutils.stretchImage(inputImg, strchFile,

72

False, "", True, False, gdalFormat, rsgislib.TYPE_32FLOAT,

73

rsgislib.imageutils.STRETCH_LINEARSTDDEV, 2)

74
75
76
77
78

print("Add 1 to stretched file to ensure there are no all zeros \

(i.e., no data) regions created.")
rsgislib.imagecalc.imageMath(strchFile, strchFileOffset, "b1+1",
gdalFormat, rsgislib.TYPE_32FLOAT)

79
80

print("Create Input Image Mask.")

81

ImgBand = collections.namedtuple(ImgBands,

82

[bandName, fileName, bandIndex])

83

bandMathBands = list()

84

bandMathBands.append(ImgBand(bandName="b1",

85

fileName=inputImg, bandIndex=1))

APPENDIX A. RSGISLIB

86
87

172

rsgislib.imagecalc.bandMath(strchMaskFile, "b1==0?1:0", gdalFormat,

rsgislib.TYPE_32FLOAT, bandMathBands)

88
89

print("Mask stretched Image.")

90

rsgislib.imageutils.maskImage(strchFileOffset, strchMaskFile,

91

segmentFile, gdalFormat, rsgislib.TYPE_32FLOAT, 0, 1)

92
93

if not noDelete:

94

# Deleting extra files

95

rsgisUtils.deleteFileWithBasename(strchFile)

96

rsgisUtils.deleteFileWithBasename(strchFileOffset)

97

rsgisUtils.deleteFileWithBasename(strchMaskFile)

98
99

# Perform KMEANS

100

print("Performing KMeans.")

101

outMatrixFile = os.path.join(tmpath,basename+str("_kmeansclusters"))

102

rsgislib.imagecalc.kMeansClustering(segmentFile, outMatrixFile, numClusters,

103

200, 100, True, 0.0025, rsgislib.imagecalc.INITCLUSTER_DIAGONAL_FULL_ATTACH)

104
105

# Apply KMEANS

106

print("Apply KMeans to image.")

107

kMeansFileZones = os.path.join(tmpath,basename+str("_kmeans")+outFileExt)

108

rsgislib.segmentation.labelPixelsFromClusterCentres(segmentFile,

109

kMeansFileZones, outMatrixFile+str(".gmtxt"), True, gdalFormat)

110
111

# Elimininate Single Pixels

112

print("Eliminate Single Pixels.")

113

kMeansFileZonesNoSgls = os.path.join(tmpath,basename+

114
115
116

str("_kmeans_nosgl")+outFileExt)
kMeansFileZonesNoSglsTmp = os.path.join(tmpath,basename+
str("_kmeans_nosglTMP")+outFileExt)

APPENDIX A. RSGISLIB

117
118

173

rsgislib.segmentation.eliminateSinglePixels(segmentFile, kMeansFileZones,
kMeansFileZonesNoSgls, kMeansFileZonesNoSglsTmp, gdalFormat, False, True)

119
120

# Clump

121

print("Perform clump.")

122

initClumpsFile = os.path.join(tmpath,basename+str("_clumps")+outFileExt)

123

rsgislib.segmentation.clump(kMeansFileZonesNoSgls, initClumpsFile,

124

gdalFormat, False, 0)

125
126

# Elimininate small clumps

127

print("Eliminate small pixels.")

128

elimClumpsFile = os.path.join(tmpath,basename+str("_clumps_elim")+outFileExt)

129

rsgislib.segmentation.RMSmallClumpsStepwise(segmentFile, initClumpsFile,

130

elimClumpsFile, gdalFormat, False, "", False, False, minPxls, distThres)

131
132

# Relabel clumps

133

print("Relabel clumps.")

134

rsgislib.segmentation.relabelClumps(elimClumpsFile, outputClumps,

135

gdalFormat, False)

136
137

# Populate with stats if required.

138

if not noStats:

139

print("Calculate image statistics and build pyramids.")

140

rsgislib.rastergis.populateStats(outputClumps, True, True)

141
142

# Create mean image if required.

143

if not (outputMeanImg == None):

144
145
146
147

rsgislib.segmentation.meanImage(inputImg, outputClumps, outputMeanImg,

gdalFormat, rsgislib.TYPE_32FLOAT)
if not noStats:
rsgislib.imageutils.popImageStats(outputMeanImg, True, 0, True)

APPENDIX A. RSGISLIB

148

if not noDelete:

149
150

# Deleting extra files

151

rsgisUtils.deleteFileWithBasename(kMeansFileZones)

152

rsgisUtils.deleteFileWithBasename(kMeansFileZonesNoSgls)

153

rsgisUtils.deleteFileWithBasename(kMeansFileZonesNoSglsTmp)

154

rsgisUtils.deleteFileWithBasename(initClumpsFile)

155

rsgisUtils.deleteFileWithBasename(elimClumpsFile)

156

if not noStretch:
rsgisUtils.deleteFileWithBasename(segmentFile)

157

if createdDIR:

158

rsgisUtils.deleteDIR(tmpath)

159
160
161

if __name__ == __main__:

162

"""

163

The command line user interface to the RSGISLib Segmentation tool

164

"""

165

parser = argparse.ArgumentParser(prog=rsgislibsegmentation.py,

166

description=Run the Shepherd et al segmentation algorithm.)

167
168

# Define the argument for specifying the input image.

169

parser.add_argument("-i", "--input", type=str,

170

help=Specify the input image file., required=True)

171
172

# Define the argument for specifying the output image.

173

parser.add_argument("-o", "--output", type=str,

174

help=Specify the output clump image file., required=True)

175
176

# Define the argument for specifying the output image.

177

parser.add_argument("-m", "--outmeanimg", type=str,

178

help=Specify the output mean image file.

174

APPENDIX A. RSGISLIB

179

175

If not specified a mean image is not created.)

180
181

# Define the argument for specifying the output image.

182

parser.add_argument("-t", "--tmpath", type=str,

183
184

help=Specify the file path for intermediate

files., required=True)

185
186

# Define the argument for specifying the output image.

187

parser.add_argument("-k", "--numclusters", type=int,

188

help=Specify the number of clusters within

189

the KMeans clustering., required=True)

190
191

# Define the argument for specifying the output image.

192

parser.add_argument("-n", "--minpxls", type=int,

193
194

help=Specify the minimum number pixels within

a segments., required=True)

195
196

# Define the argument for specifying the output image.

197

parser.add_argument("-d", "--distthres", type=float,

198

help=Specify the distance threshold for joining the segments

199

(default is a very large value which turns off this option.).,

200

default=1000000, required=False)

201
202

# Define the argument for specifying the image file format.

203

parser.add_argument("-f", "--format", type=str, default="KEA",

204

help=Specify the image output format (GDAL name).)

205
206

# Define the argument for specifying that statistics

207

# and pyramids should be built for all output images.

208

parser.add_argument("--nostats", action=store_true, default=False,

209

help=Specifies that no image statistics and

APPENDIX A. RSGISLIB

210

176

pyramids should be build for the output images.)

211
212

# Define the argument for specifying that the input image bands

213

# should not be stretched.

214

parser.add_argument("--nostretch", action=store_true, default=False,

215

help=Specifies that the input image

216

bands should not be stretched.)

217
218

# Define the argument for specifying that the input image bands

219

# should not be stretched.

220

parser.add_argument("--nodelete", action=store_true, default=False,

221

help=Specifies that the temporary images

222

created during processing should not be deleted

223

once processing has been completed.)

224
225

# Call the parser to parse the arguments.

226

args = parser.parse_args()

227
228

segsObj = RSGISLibSegmentation()

229
230

segsObj.runShepherdSegmentation(args.input, args.output, args.outmeanimg,

231

args.tmpath, args.format, args.nostats, args.nostretch, args.nodelete,

232

args.numclusters, args.minpxls, args.distthres)

The script can be run as follows:

python rsgissegmentation.py -i L7ETM_530N035W.kea \
-o L7ETM_530N035W_clumps_elim_final.kea \
-m L7ETM_530N035W_clumps_elim_final_mean.kea \
-t ./tmp -k 60 -n 100

The resulting segmentation will have produced 3 output files

APPENDIX A. RSGISLIB

177

1. *clusters.gmtxt Cluster centres.

2. *clumps_elim_final.kea Segment clumps.
3. *clumps_elim_mean.kea Mean colour image using segments.

A.4

Populating Segments

To populate the segments with statistics (i.e., Mean for each spectral band) there
is a function within the rastergis module of RSGISLib. To populate segments with
statistics from two images and a DEM the following script is used:
1

#! /usr/bin/env python

2
3

#######################################

# A python script to attribute

# segments

# Author: <YOUR NAME>

# Email: <YOUR EMAIL>

# Date: DD/MM/YYYY

10

# Version: 1.0

11

#######################################

12
13

# Import the argparse module

14

import argparse

15

# Import the RSGISLib RasterGIS module

16

import rsgislib.rastergis

17
18

class PopWithStats (object):

APPENDIX A. RSGISLIB

178

19
20
21

def __init__(self, inClumps):

self.inClumps = inClumps

22
23

def popWithMayStats(self, image):

24

""" Populate with stats from May image """

25

stats2Calc = list()

26

stats2Calc.append(rsgislib.rastergis.BandAttStats(band=1, \

27
28
29
30
31
32
33
34
35
36
37

meanField="MayBlue", stdDevField="MaySDBlue"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=2, \
meanField="MayGreen", stdDevField="MaySDGreen"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=3, \
meanField="MayRed", stdDevField="MaySDRed"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=4, \
meanField="MayNIR", stdDevField="MaySDNIR"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=5, \
meanField="MaySWIR1", stdDevField="MaySDSWIR1"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=6, \
meanField="MaySWIR2", stdDevField="MaySDSWIR2"))

38
39

rsgislib.rastergis.populateRATWithStats(image, self.inClumps, stats2Calc)

40
41

def popWithJuneStats(self, image):

42

""" Populate with stats from June image """

43

stats2Calc = list()

44

stats2Calc.append(rsgislib.rastergis.BandAttStats(band=1, \

45
46
47
48
49

meanField="JuneBlue", stdDevField="JuneSDBlue"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=2, \
meanField="JuneGreen", stdDevField="JuneSDGreen"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=3, \
meanField="JuneRed", stdDevField="JuneSDRed"))

APPENDIX A. RSGISLIB

50
51
52
53
54
55

179

stats2Calc.append(rsgislib.rastergis.BandAttStats(band=4, \
meanField="JuneNIR", stdDevField="JuneSDNIR"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=5, \
meanField="JuneSWIR1", stdDevField="JuneSDSWIR1"))
stats2Calc.append(rsgislib.rastergis.BandAttStats(band=6, \
meanField="JuneSWIR2", stdDevField="JuneSDSWIR2"))

56
57

rsgislib.rastergis.populateRATWithStats(image, self.inClumps, stats2Calc)

58
59
60

def popWithDEMStats(self, image):

61

""" Populate with DEM stats """

62

stats2Calc = list()

63

stats2Calc.append(rsgislib.rastergis.BandAttStats(band=1, \

64

minField="MinDEM", maxField="MaxDEM", \

65

stdDevField="StdDevDEM", meanField="MeanDEM"))

66
67

rsgislib.rastergis.populateRATWithStats(image, self.inClumps, stats2Calc)

68
69
70

if __name__ == __main__:

71
72

parser = argparse.ArgumentParser()

73
74
75
76
77
78
79
80

parser.add_argument("--inclumps", type=str, \
help=Specify the input clumps file., required=True)
parser.add_argument("--mayimage", type=str, \
help=Specify the input image for May., required=True)
parser.add_argument("--juneimage", type=str, \
help=Specify the input image for June., required=True)
parser.add_argument("--dem", type=str, \

APPENDIX A. RSGISLIB

81

180

help=Specify the input DEM., required=True)

82
83

args = parser.parse_args()

84
85

popStats = PopWithStats(args.inclumps)

86
87

# Pop with may statistics

88

popStats.popWithMayStats(args.mayimage)

89
90

# Pop with june statistics

91

popStats.popWithMayStats(args.juneimage)

92
93

# Pop with DEM statistics

94

popStats.popWithMayStats(args.dem)

95
96
97

If you are going to use indices and other derived information within your classification it is quite often a good idea to set up a python script to calculate those indices
and write them back to the image rather than over complicating your classification
script. An example of this is shown below.
1

#!/usr/bin/env python

2
3

import sys

from rios import rat

import numpy

import osgeo.gdal as gdal

7
8

APPENDIX A. RSGISLIB

#Input file.

10

fname = "L7ETM_530N035W_Classification.kea"

11

ratDataset = gdal.Open( fname, gdal.GA_Update )

12
13

print("Import Columns.")

14

MayBlue = rat.readColumn(ratDataset, "MayBlue")

15

MayGreen = rat.readColumn(ratDataset, "MayGreen")

16

MayRed = rat.readColumn(ratDataset, "MayRed")

17

MayNIR = rat.readColumn(ratDataset, "MayNIR")

18

MaySWIR1 = rat.readColumn(ratDataset, "MaySWIR1")

19

MaySWIR2 = rat.readColumn(ratDataset, "MaySWIR2")

20
21

JuneBlue = rat.readColumn(ratDataset, "JuneBlue")

22

JuneGreen = rat.readColumn(ratDataset, "JuneGreen")

23

JuneRed = rat.readColumn(ratDataset, "JuneRed")

24

JuneNIR = rat.readColumn(ratDataset, "JuneNIR")

25

JuneSWIR1 = rat.readColumn(ratDataset, "JuneSWIR1")

26

JuneSWIR2 = rat.readColumn(ratDataset, "JuneSWIR2")

27
28

MeanDEM = rat.readColumn(ratDataset, "MeanDEM")

29
30

MayNIR.astype(numpy.float32)

31

MayRed.astype(numpy.float32)

32

JuneNIR.astype(numpy.float32)

33

JuneRed.astype(numpy.float32)

34

MayBlue.astype(numpy.float32)

35

JuneBlue.astype(numpy.float32)

36
37

print("Calculate Indices.")

38

MayNDVI = (MayNIR - MayRed) / (MayNIR + MayRed)

39

JuneNDVI = (JuneNIR - JuneRed) / (JuneNIR + JuneRed)

181

APPENDIX A. RSGISLIB

40
41

MayWBI = MayBlue/MayNIR

42

JuneWBI = JuneBlue/JuneNIR

43
44

rat.writeColumn(ratDataset, "MayNDVI", MayNDVI)

45

rat.writeColumn(ratDataset, "JuneNDVI", JuneNDVI)

46

rat.writeColumn(ratDataset, "MayWBI", MayWBI)

47

rat.writeColumn(ratDataset, "JuneWBI", JuneWBI)

182