9117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises A Python Book: Beginning Python, Advanced Python, and Python Exercises Author: Dave Kuhiman Contact: dkuhiman (at) davekuhiman (dot) org Address: http://www. davekuhinan.org Revision: 1.32 Date: October 05, 2014 Copyright: Copyright (c) 2009 Dave Kuhiman. All Rights Reserved. This document is subject to the provisions of the Open Source MIT License http://www. opensource.org/licenses/mit-license.php. Abstract: This document is a self-learning document for a course in Python programming. This course contains (1) a part for beginners, (2) a discussion of several advanced topics that are of interest to Python programmers, and (3) a Python workbook with lots of exercises. Contents: 1_Part 1 -- Beginning Python Introductions Etc 1.1.1 Resources 1.1.2 A general description of Python 1.1.3 Interactive Python Lexical matters Lines ‘Comments Names and tokens Blocks and indentation 5 Doc strings Program structure Also see 9 Code evaluation jon -- preliminaries 14.3_Strings 1.4.3.1 The new string.format method 1.4.3.2 Unicode strings 1.4.4” Dictionaries 1.4.5 Files 1.4.6 Other built-in types 1.4.6.1 The None va ue/tyee Boolean values Sets and frozensets 1.5 Functions and Classes -- A Preview 6 Statements https: hwww-davekuhiman orgfpytnon_book_1.himl 1819117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises Assignment statement ort statement : else: statement statement ment 1.6.8 try: except: statement 1.6.9 rai ement 1.6.10 with: statement 1.6.10.1 Writing a context manager 1.6.10.2 Using the with: statement 1_del 1.6.12 case statement Functions, Modules, Packages, and Debugging Functions ‘The def statement Returning values Parameters ‘Arguments Local variables Other things to know about functions Global variables and the global statement Doc strings for functions Decorators for functions 1.2.3 Iterators and generators 1.2.4 Modules 1.7.4.1 Doc strings for modules 1.7.5 Packages 1.8 Classes A simple class .2 Defining methods ‘The constructor Member variables .5 Calling methods ‘Adding inheritance Class variables Class methods and static methods Properties 0 Interfaces 8.11 New-style classes 2 Doc strings for classes 3 Private members pecial Tasks 1.9.1 Debugging tools 1.9.3.3 Additional unittest feature 1.9.3.4 Guidance on Unit Testing 1 doctest 1.9.5 The Python database API 1.9.6 Installing Python packages 1.10 " More Python Features and Exercises 2_ Part 2 -- Advanced Python https: www-davekuhiman orgfpytnen_book_.himl 22619117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises 2.1 Introduction 2.2 Regular Expressions 2.2.1 Defining regular expressions 2.2.2 Compiling regular expressions 2.2.3 Using regular expres: 2.2.4 Using match objects to extract a value 2.2.5 Extracting multiple items 2.2.6 Replacing multiple items 2.3 Iterator Objects 2.3.1 Example - A generator function 2.3.2 Example - A class containing a generator method 2.3.3 Example - An iterator class Example - An iterator class that uses yield ‘A list comprehension ‘A generator expression 2.4 Unit Tests 2.4.1 Defining unit tests 2.4.1.1 Create a test class. 2.5 Extending and embedding Python Introduction and concepts Extension modules SWIG Pyrex ‘SWIG vs. Pyrex .6 Cython 2.5.2 Extension types 2.5.8 Extension classes 2.6 Parsing 2.6.1 Special purpose parsers 2.6.2 Writing a recursive descent parser by hand 2.6.3 Creating a lexer/tokenizer with Plex 2.6.6 Creating a parser with pyparsing 2.6.6.1 Parsing comma-delimited lines Parsing functors ‘A simple text input dialog box A file selection dialog box 2.8.1 Introduction 2.8.2 Implementing Packages 2.8.3 Using Packages 2.8.4 Distributing and Installing Packages 2.9 _End Matter 2.9.1 Acknowledgements and Thanks 2.9.2 See Also 3_Part 3 -- Python Workbook 3.1_Introduct https: hwww-davekuhiman orgfpytnon_book_1.himl 32619117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises 3.2__Lexical Structures 3. Variables and names 2.2.2 Line structure 3. Indentation and program structure 3_ Execution Model 3.4 Built-in Data Types 34. Literal representations of number: ators for numbers Literal representation of lists Operators on lists Methods on lists 3.4.2.4 List comprehensions Strings 3.1 Characters 3.4.3.2 Operators on strings Methods on stri 3.4 Raw strings 3.4.3.5 Unicode strings Dictionaries 4.1 Literal representation of dictionaries 3.4.4.2 Operators on dictionaries Methods on dictionaries 3.5_ Statements 3.5.1 Assignment statement 3 rint statement : tatement exercises statement exercises while: statement exercises 5.6 break and continue statements Exceptions and the try:except: and raise statements 3.6_Functions 6.1 Optional arguments and default values Passing functions as arguments Extra args and keyword args 3.6.3.1_Order of arguments (positional, extra, and keyword args) 3.6.4 Functions and duck-typing and polymorphism 3.6.5 Recursive functions 3.6.6 Generators and iterators 3.2_Object-oriented programming and classes 3.2Z1_The constructor (eo Jo fos on fos Jus fw fo 3.22 Inheritance -- Implementing a subclass 3.7.3 Classes and polymorphism 3.7.4 Recursive calls to methods 3.7.5 Class variables, class methods, and static methods 5.1 Decorators for classmethod and sti 3.8 Additional and Advanced Topics 3.8.1 Decorators and how to implement them ‘1.1 Decorators with arguments 3.8.1.2 Stacked decorators method https: hwww-davekuhiman orgfpytnon_book_1.himl 4261917124, 9:01 PA [APython Book: Beginning Python, Advanced Python, and Python Exercises 3.8.1.3 More help with decorators 3.8.2_Iterables 3.8.2.1 A few preliminaries on Iterables 3.8.2.2 More help with iterables 3.9 Applications and Recipes 35-1" XML SAX minidom, ElementTree, Lxm! 3.9,2 Relational database access 3.9.3 CSV -- comma separated value files 3.9.4 YAML and PyYAML 3.9.5 Json 4_Part 4 -- Generating Python Bindings for XML 4.1 Introduction 4.2. erating the code 43. 1g the generated code to parse and export an XML document 4.4” Some command line options you might want to know 4.5 The graphical front-end 4.6 Adding application-specific behavior 4.6.3 A combined approach 4.7_Special situations and _uses 4.7.1_ Genet independent processing jenerate the bindings 48-2 Children defined with simple numeric types 4.8.3 The type of an element's character content [table of contents] Preface This book is a collection of materials that I've used when conducting Python training and also materials from my Web site that are intended for self-instruction. You may prefer a machine readable copy of this book. You can find it in various formats here: + HTML -- http://www.davekuhiman.org/python_book O1.html © PDF -- http://www.davekuhIiman.org/python book O1.pdf ‘* ODF/OpenOffice -- htto://www.davekuhlman.org/python book 01.odt And, let me thank the students in my Python classes. Their questions and suggestions were a great help in the preparation of these materials. 1_ Part 1 -- Beginning Python 1,1 Introductions Etc https: hwww-davekuhiman orgfpytnon_book_1.himl 52619117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises Introductions Practical matters: restrooms, breakroom, lunch and break times, etc. Starting the Python interactive interpreter. Also, IPython and Idle. Running scripts Editors -- Choose an editor which you can configure so that it indents with 4 spaces, not tab characters. For a list of editors for Python, see: http://wiki.python.org/moin/PythonEditors. A few possible editors: * SciTE -- http://www.scintilla.org/SciTE.html. «| MS Windows only -- (1) TextPad -- http://www.textpad.com; (2) UltraEdit -- ‘http://www.ultraedit.com/. « Jed -- See http://www.jedsoft.ora/jed/. Emacs -- See http://www.gnu.org/software/emacs/ and http://www.xemacs.org/faq/xemacs-fag.html. jEdit -- Requires a bit of customization for Python -- See http://jedit.org. Vim -- http://www.vim.org/ Geany -- http://www.geany.ora/ And many more. Interactive interpreters: + python + python + Idle IDEs -- Also see http://en.wikipedia.org/wiki/List_of integrated development _environments for Python: PyWin -- MS Windows only. Available at: http://sourceforge.net/projects/pywin32/. + WingIDE -- See http://wingware.com/wingide/. + Eclipse -- http://eclipse.org/, There is a plug-in that supports Python. + Kdevelop -- Linux/KDE -- See htto://www.kdevelop.ora/. * Eric -- Linux KDE? -- See http://eric-ide,python-projects.ora/index.html + Emacs and SciTE will evaluate a Python buffer within the editor, 1.1.1 Resources Where else to get help: Python home page -- http://www. thon.org Python standard documentation -- http://www.python.ora/doc/. You also find links to tutorials there. FAQs -- http://www.python.org/doc/faq/. The Python Wiki - http://wiki.python.oro/ https: www-davekuhiman orgfpytnen_book_.himl 626197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises * The Python Package Index -- Lots of Python packages -- jttps://pypi.python.org/p\ * Special interest groups (SIGs) -- http://www.python.org/sigs/ + Other python related mailing lists and lists for specific applications (for example, Zope, Twisted, etc). Try: http://dir.gmane.org/search.php?match=python, * http://sourceforge.net -- Lots of projects. Search for "python". + USENET -- comp.lang.python. Can also be accessed through Gmane: http://dir.gmane.org/gmane.comp.python.general. * The Python tutor email list Atty //mail.python,org/mailman/listinfo/tutor Local documentation + On MS Windows, the Python documentation is installed with the standard installation. + Install the standard Python documentation on your machine from http://www. python.org/doc/. * pyéoc. Example, on the command line, type: pydoc re + Import 2 module, then view its ._doc_ attribute. * At the interactive prompt, use heip(obj). You might need to import it first. Example: >>> import urllib >>> help(urllib) * In IPython, the question mark operator gives help. Example: In [13]: open? Type: builtin_function_or_method Base Class: String Form: Namespace Python builtin Docstring: open(name[, mode[, buffering]]) -> file object Open a file using the file() type, returns a file object. Constructor Docstring: i «) initializes x; see x._class Yes Calling definition not available.call docstring: XC) doc__ for signatu 1.1.2 A general description of Python hitps:www-davekuhiman.ergpython_book_01 hem! i26197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises Python is a high-level general purpose programming language: + Because code is automatically compiled to byte code and executed, Python is suitable for use as a scripting language, Web application implementation language, etc. + Because Python can be extended in C and C++, Python can provide the speed needed for even compute intensive tasks. + Because of its strong structuring constructs (nested code blocks, functions, classes, modules, and packages) and its consistent use of objects and object- oriented programming, Python enables us to write clear, logical applications for small and large tasks. Important features of Python: * Built-in high level data types: strings, lists, dictionaries, etc. + The usual control structures: if, if-else, if-elif-else, while, plus a powerful collection iterator (for). * Multiple levels of organizational structure: functions, classes, modules, and packages. These assist in organizing code. An excellent and large example is the Python standard library. + Compile on the fly to byte code -- Source code is compiled to byte code without a separate compile step. Source code modules can also be "pre-compiled” to byte code files. * Object-oriented -- Python provides a consistent way to use objects: everything is an object. And, in Python it is easy to implement new object types (called classes in object-oriented programming). + Extensions in C and C++ -- Extension modules and extension types can be written by hand. There are also tools that help with this, for example, SWIG, sip, Pyrex. + Jython is a version of Python that "plays well with” Java. See: The Jython Project -- http://www. jython.org/Project. Some things you will need to know: + Python uses indentation to show block structure. Indent one level to show the beginning of a block, Out-dent one level to show the end of a block. As an example, the following C-style code: if (x) { if (y) { #1) + 20) + in Python would be: if x: if y: #10) 20) hitps:www-davekuhiman.orgpython_book_01 hem! e126‘9117724, 9:01 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises ‘And, the convention is to use four spaces (and no hard tabs) for each level of indentation. Actually, it's more than a convention; it's practically a requirement. Following that "convention" will make it so much easier to merge your Python code with code from other sources. An overview of Python: * A scripting language -- Python is suitable (1) for embedding, (2) for writing small unstructured scripts, (3) for "quick and dirty" programs. * Not a scripting language -- (1) Python scales. (2) Python encourages us to write code that is clear and well-structured. + Interpreted, but also compiled to byte-code. Modules are automatically compiled (to .pyc) when imported, but may also be explicitly compiled * Provides an interactive command line and interpreter shell. In fact, there are several. + Dynamic -- For example: © Types are bound to values, not to variables. © Function and method lookup is done at runtime. © Values are inspect-able. © There is an interactive interpreter, more than one, in fact. © You can list the methods supported by any given object. + Strongly typed at run-time, not compile-time. Objects (values) have a type, but variables do not. + Reasonably high level -- High level built-in data types; high level control structures (for walking lists and iterators, for example). * Object-oriented hiding by agreement, Multiple inheritance. Interfaces by convention. Polymorphism: Almost everything is an object. Simple object definition. Data + Highly structured -- Statements, functions, classes, modules, and packages enable us to write large, well-structured applications. Why structure? Readability, locate-ability, modifiability. + Explicitness * First-class objects © Definition: Can (1) pass to function; (2) return from function; (3) stuff into a data structure. © Operators can be applied to values (not variables). Example: £(x)(3] + Indented block structure -- "Python is pseudo-code that runs.” + Embedding and extending Python -- Python provides a well-documented and supported way (1) to embed the Python interpreter in C/C++ applications and hitps:www-davekuhiman.orgpython_book_01 hem! 9126197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises (2) to extend Python with modules and objects implemented in C/C++ © In some cases, SWIG can generate wrappers for existing C/C++ code automatically. See http://www.swig.ora/ © Cython enables us to generate C code from Python and to "easily" create wrappers for C/C++ functions. See http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/. © To embed and extend Python with Java, there is Jython. See http://www.jython.org/ + Automatic garbage collection. (But, there is a gc module to allow explicit control of garbage collection.) * Comparison with other languages: compiled languages (e.g. C/C++); Java; Perl, Tel, and Ruby. Python excells at: development speed, execution speed, clarity and maintainability. * Varieties of Python: © CPython -- Standard Python 2.x implemented in C. © Jython -- Python for the Java environment =~ htt Lowww.jython.org/ © PyPy -- Python with a JIT compiler and stackless mode -- http://pypy.ora/ © Stackless -- Python with enhanced thread support and microthreads etc. -- http://www.stackless.com/ © IronPython -- Python for .NET and the CLR -- http://ironpython.net/ © Python 3 -- The new, new Python. This is intended as a replacement for Python 2.x, -- http://www.python.org/doc/. A few differences (from Python 2.x): = The print statement changed to the print function. = Strings are unicode by default. = Classes are all "new style" classes. = Changes to syntax for catching exceptions. = Changes to integers -- no long integer; integer division with automatic convert to float. = More pervasive use of iterables (rather than collections). = Ete. For a more information about differences between Python 2.x and Python 3.x, see the description of the various fixes that can be applied with the 2t03 tool: http://docs.python.org/3/library/2to3.html#fixers The migration tool, 2to3, eases the conversion of 2.x code to 3.x. * Also see The Zen of Python -- http://www.python.ora/peps/pep-0020. html. Or, at the Python interactive prompt, type: >>> import this hitps:ww-davekuhiman.ergpython_book_01.hem! 0126197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises 1.1.3 Interactive Python If you execute Python from the command line with no script (no arguments), Python gives you an interactive prompt. This is an excellent facility for learning Python and for trying small snippets of code. Many of the examples that follow were developed using the Python interactive prompt. Start the Python interactive interpreter by typing python with no arguments at the command line. For example: $ python Python 2.6.1 (r261:67515, Jan 11 2009, 15:19:23) [GCC 4.3.2] on Linux2 Type “help”, "copyright", “credits” or “license” for more infornation. >>> print ‘hello’ hello >>> You may also want to consider using IDLE, IDLE is a graphical integrated development environment for Python; it contains a Python shell. It is likely that Idle was installed for you when you installed Python. You will find a script to start up IDLE in the Tools/scripts directory of your Python distribution, IDLE requires Tkinter. In addition, there are tools that will give you a more powerful and fancy Python interactive interpreter. One example is IPython, which is available at http://ipython.scipy.org/. 1.2 Lexical matters 1.2.1 Lines + Python does what you want it to do most of the time so that you only have to add extra characters some of the time. + Statement separator is a semi-colon, but is only needed when there is more than one statement on a line. And, writing more than one statement on the same line is considered bad form. + Continuation lines -- A back-slash as last character of the line makes the following line a continuation of the current line. But, note that an opening ‘context" (parenthesis, square bracket, or curly bracket) makes the back-slash unnecessary. 1.2.2 Comments Everything after "#" on a line is ignored. No block comments, but doc strings are a comment in quotes at the beginning of a module, class, method or function. Also, editors with support for Python often provide the ability to comment out selected blocks of code, usually with "##". 1.2.3__Names and token: hitps:www-davekuhiman.ergpython_book_01 hem! sweet‘9117724, 9:01 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises * Allowed characters: a-z A-Z 0-9 underscore, and must begin with a letter or underscore. + Names and identifiers are case sensitive. + Identifiers can be of unlimited length. * Special names, customizing, etc. -- Usually begin and end in double underscores * Special name classes -- Single and double underscores. © Single leading single underscore -- Suggests a "private" method or variable name. Not imported by "from module import *", © Single trailing underscore -- Use it to avoid conflicts with Python keywords. © Double leading underscores -- Used in a class defi mangling (weak hiding). But, not often used. + Naming conventions -- Not rigid, but © Modules and packages -- all lower case. Globals and constants -- Upper case. Classes -- Bumpy caps with initial upper. Methods and functions -- All lower case with words separated by underscores. © Local variables -- Lower case (with underscore between words) or bumpy caps with initial lower or your choice, © Good advice -- Follow the conventions used in the code on which you are working. + Names/variables in Python do not have a type. Values have types. 1.2.4 Blocks and indentation Python represents block structure and nested block structure with indentation, not with begin and end brackets. ion to cause name The empty block -- Use the pass no-op statement. Benefits of the use of indentation to indicate structure + Reduces the need for a coding standard. Only need to specify that indentation is, 4 spaces and no hard tabs. + Reduces inconsistency. Code from different sources follow the same indentation style. It has to. * Reduces work. Only need to get the indentation correct, not both indentation and brackets. * Reduces clutter, Eliminates all the curly brackets. * If it looks correct, it is correct. Indentation cannot fool the reader. Editor considerations -- The standard is 4 spaces (no hard tabs) for each indentation level. You will need a text editor that helps you respect that. 1.2.5 Doc strings Doc strings are like comments, but they are carried with executing code. Doc strings can be viewed with several tools, ¢.9. help(), obj._doc_, and, in IPython, a question mark (2) after a name will produce help. ‘A doc string is written as a quoted string that is at the top of a module or the first lines after the header line of a function or class. hitps:www-davekuhiman.orgpython_book_01 hem! sa6t97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises We can use triple-quoting to create doc strings that span multiple lines. There are also tools that extract and format doc strings, for example: * pydoc -- Documentation generator and online help system -- http://docs. python.org/lib/module-pydoc.html. + epydoc -- Epydoc: Automatic API Documentation Generation for Python -~ ‘nttp://epydoc.sourceforge.net/index.htm! + Sphinx -- Can also extract documentation from Python doc strings. See ‘http://sphinx-doc.org/index.htm. See the following for suggestions and more information on doc strings: Docstring conventions -- http://www.python.org/dev/peps/pep-0257/. 1.2.6 Program structure * Execution -- def, class, etc are executable statements that add something to the current name-space, Modules can be both executable and import-able. Statements, data structures, functions, classes, modules, packages. Functions Classes Modules correspond to files with a "*.py” extension. Packages correspond to a directory (or folder) in the file system; a package contains a file named "__init__.py". Both modules and packages can be imported (see section import statement). + Packages -- A directory containing a file named "__init__.py". Can provide additional initialization when the package or a module in it is loaded (imported) 1.2.7 Operators + See: http://docs.python.org/ref/operators.html. Python defines the following operators: + - * a uW x « >» a ~ < > < ' ° The comparison operators <> and = are alternate spellings of the same operator. 1s is the preferred spelling; <> is obsolescent. * Logical operators: and or is not in + There are also (1) the dot operator, (2) the subscript operator (1, and the function/method call operator (). * For information on the precedences of operators, see the table at http://docs.python.org/2/reference/expressions.html#operator-precedence, which is reproduced below. hitps:www-davekuhiman.ergpython_book_01 hem! 326197124, 9:01 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises * For information on what the different operators do, the section in the "Python Language Reference" titled "Special method names" may be of help: http://docs. python.org/2/reference/datamodel.html#special-method-names The following table summarizes the operator precedences in Python, from lowest precedence (least binding) to highest precedence (most binding). Operators on the same line have the same precedence. Unless the syntax is explicitly given, operators are binary. Operators on the same line group left to right (except for comparisons, including tests, which all have the same precedence and chain from left to right -- see section 5.9 -- and exponentiation, which groups from right to left) Operator Description Lambda expression Boolean OR Boolean AND Boolean NOT Membership tests Identity tests Comparisons Bitwise OR Bitwise XOR Bitwise AND shifts Addition and subtraction Multiplication, division, remainder Positive, negative Bitwise not Exponentiation x.attribute Attribute reference xLindex] Subscription x[index: index] Slicing (arguments. Function call (expressions Binding or tuple display [expressions List display {key:datum. Dictionary display expressions. String conversion + Note that most operators result in calls to methods with special names, for example _add_, _sub_, _nvl_, etc. See Special method names shttp://docs.python.org/2/reference/datamodel.html:special-method-names Later, we will see how these operators can be emulated in classes that you define yourself, through the use of these special names. 1.2.8 Also see For more on lexical matters and Python styles, see: * Code Like a Pythonista: Idiomatic Python -- ‘nttp://python.net/~goodger/projects/pycon/2007/idiomatic/handout.htm. + Style Guide for Python Code -- http://www. python.org/dew/peps/pep-0008/ hitps:www-davekuhiman.orgpython_book_01 hem! saizet97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises + The flake8 style checking program. See https://pypi.python.org/pypi/flakes. Also see the pylint code checker: https 1.2.9 Code evaluation Understanding the Python execution model -- How Python evaluates and executes your code. Evaluating expressions. Creating names/variables -- Binding -- The following all create names (variables) and bind values (objects) to them: (1) assignment, (2) function definition, (3) class definition, (4) function and method call, (5) importing a module, First class objects -- Almost all objects in Python are first class. Definition: An object is first class if: (1) we can put it in a structured object; (2) we can pass it toa function; (3) we can return it from a function. References + The object(s) satisfy the identity test operator is, + The built-in function :4() returns the same value. + The consequences for mutable objects are different from those for immutable objects + Changing (updating) a mutable object referenced through one variable or container also changes that object referenced through other variables or containers, because it is the same object. * ¢e1() -- The built-in function de1() removes a reference, not (necessarily) the object itself. 1.3. Statements and inspection -- preliminaries print ~~ Example: Objects (or references to them) can be shared. What does this mean? print obj print “one”, "two", ‘three’ -- Example stuff = ['aa", ‘bb', ‘cc'] for item in stuff: print item Learn what the type of an object is -- Example: type(obj) Learn what attributes an object has and what it's capabilities are -- Example: dir (obj) value = "a message” dir(value) hitps:www-davekuhiman.ergpython_book_01 hem! 9126197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises Get help on a class or an object -- Example: help(str) help("") value = “abe” help(value) help(value. upper) In Python (but not standard Python), you can also get help at the interactive prompt by typing "2" and "22" after an object. Example: In [48]: a=" In [49]: a.upper? Type: builtin_function_or_method String Form: Docstring: S.upper() -> string Return a copy of the string S converted to uppercase. 1.4 Built-in data-types For information on built-in data types, see section Built-in Types http://docs.python.org/lib/types.html in the Python standard documentation. 1.4.1 Numeric types The numeric types are: + Plain integers -- Same precision as a C long, usually a 32-bit binary number. + Long integers -- Define with 100. But, plain integers are automatically promoted when needed. + Floats -- Implemented as a C double. Precision depends on your machine. See sys.float_info. + Complex numbers -- Define with, for example, 3j or conplex(3.0, 2.0). See 2.3.4 Numeric Types -- int, float, long, complex -- http://docs. python. org/lib/typesnumeric,htm|. Python does mixed arithmetic. Integer division truncates. This is changed in Python 3. Use float(n) to force coercion to a float. Example: In [8]: a=4 =5 In [9]: b In [10]: a/b out[19]: @ # possibly wrong? In [11]: float(a) / b out[i1]: 2.8 hitps:www-davekuhiman.ergpython_book_01 hem! er26191724, 901 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises Applying the function call operator (parentheses) to a type or class creates an instance of that type or class. Scientific and heavily numeric programming -- High level Python is not very efficient for numerical programming. But, there are libraries that help -- Numpy and SciPy -- See: SciPy: Scientific Tools for Python -- http://scipy.ora/ 1.4.2 _Tuples and lists List -- A list is a dynamic array/sequence. It is ordered and indexable. A list is mutable. List constructors: [], 1ist(). range() and xrange(): + range(n) creates a list of n integers. Optional arguments are the starting integer and a stride. + xrange is like range, except that it creates an iterator that produces the items in the list of integers instead of the list itself. Tuples -- A tuple is a sequence. A tuple is immutable. Tuple constructors: (), but really a comma; also tuple(). Tuples are like lists, but are not mutable. Python lists are (1) heterogeneous (2) indexable, and (3) dynamic. For example, we can add to a list and make it longer. Notes on sequence constructors: * To construct a tuple with a single element, use (x,); a tuple with a single element requires a comma. + You can spread elements across multiple lines (and no need for backslash continuation character "\"). + A comma can follow the last element. The length of a tuple or list (or other container): 1en(nylist) Operators for lists: # Try: listt + 1ista, Listt +n, list += 1ist2, ete. * Comparison operators: <, etc. + Test for membership with the in operator. Example: In [77] In [78]: a out[78]: [11, 22, 33] In [79]: 22 ina out[79]: True a= [11, 22, 33] hitps:ww-davekuhiman.orgpython_book_01 hem! srie6t97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises In [80]: 44 in a out[se]: False Subscription: * Indexing into a sequence + Negative indexes -- Effectively, length of sequence plus (minus) index. * Slicing -- Example: ¢ataf2:s]. Default values: beginning and end of list. * Slicing with strides -- Example: datat::2] Operations on tuples -- No operations that change the tuple, since tuples are immutable. We can do iteration and subscription. We can do "contains" (the in operator) and get the length (the 1en() operator). We can use certain boolean operators. Operations on lists -- Operations similar to tuples plus: + Append -- aylist.append(newiter) + Insert -- aylist.insert(index, newiten). Note on efficiency: The insert method is not as fast as the append method. If you find that you need to do a large number of aylist.insert(a, obi) (that is, inserting at the beginning of the list) consider using a deque instead. See: ttp://docs.python.or9/2/library/collections.html#collections.deque. Or, use append and reverse. + Extend -- mylist.extend(anotherlist). Also can use + and +=. + Remove -- nylist.renove(ites) and nylist.pop(). Note that append) together with pop) implements a stack. * Delete -- dei nylisttindex: * Pop -- Get last (right-most) item and remove from list -~ aylist.pop(). List operators -- +, *, etc. For more operations and operators on sequences, see: http://docs. python.org/2/library/stdtypes. ht ml#sequence-types-str-unicode-list- tuple-bytearray-buffer-xrange. Exercises: + Create an empty list. Append 4 strings to the list. Then pop one item off the end of the list. Solution In [25]: a= [1 In [26]: a.append('aaa’) In [27]: a.append('bbb') In [28]: a.append(‘ecc') In [29]: a.append('ddd") In [30]: print a [‘aaa', ‘bbb’, ‘ccc", ‘ddd'] In [31]: a.pop() out(31}: "ddd" + Use the for statement to print the items in the list. Solution hitps:www-davekuhiman.ergpython_book_01 hem! sar2619117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises In [32]: for item in a: print item aaa bbb cee ‘+ Use the string join operation to concatenate the items in the list. Solution: In [33]: '[|".join(a) out[33]: ‘aaal |bbb| |ccc* «Use lists contai \g three (3) elements to create and show a tree: In [37]: b = ["bb', None, None] In [38]: ¢ = ['ec', None, None] In [39]: root = ['aa', 6, c] In [48]: In [40] In [40]: def show tree(t): seed Ef not tr return print t[@] show_tree(t[1]) show_tree(t[2]) In [41] aa bb ce show_tree(root) Note that we will learn a better way to represent tree structures when we cover implementing classes in Python. 1.4.3 Strings Strings are sequences. They are immutable. They are indexable. They are iterable. For operations on strings, see http://docs.python.org/lib/string-methods.html or use: >>> help(str) Or: >>> din(“abe") String operations (methods). String operators, e.9. +, <, < Constructors/literals: https: www-davekuhiman orgfpytnen_book_.himl 19261‘9117724, 9:01 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises * Quotes: single and double. Escaping quotes and other special characters with a back-slash, * Triple quoting -- Use triple single quotes or double quotes to define multi-line strings, + str) = The constructor and the name of the type/class. # ‘aSeparator’ join(aList) + Many more: Escape characters in strings -- \t, \n, \\, etc. String formatting -- See: http://docs.python.org/2/library/stdtypes.htmli#string- Examples: In [18]: name = ‘dave’ In [19]: size = 25 In [28]: factor = 3.45 In [21]: print ‘Name: ¥s Size: %d Factor: %3.4f' X (name, size, factor, ) Name: dave Size: 25 Factor: 3.4500 In [25]: print ‘Name: %s Size: %d Factor: %8.4f" % (name, size, factor, )| Name: dave Size: 25 Factor: 003.4500 If the right-hand argument to the formatting operator is a dictionary, then you can (actually, must) use the names of keys in the dictionary in your format strings. Examples: In [115]: values = {‘vegetable': ‘chard’, ‘fruit’: ‘nectarine'} In [116]: "I love %(vegetable)s and I love %(fruit)s.' % values Out[116]: ‘I love chard and I love nectarine.' Also consider using the right justify and left justi operations. Examples: nystring.rjust(20), aystring.1just(20, In Python 3, the str. format method is preferred to the string formatting operator. This method is also available in Python 2.7. It has benefits and advantages over the string formatting operator. You can start learning about it here: http://docs.python.org/2/library/stdtypes, html#string-methods Exercises: « Use a literal to create a string containing (1) a single quote, (2) a double quote, (3) both a single and double quote. Solutions: “Some ‘quoted’ text.” "Some "quoted" text." "Some "quoted" \‘extra\’ text." * Write a string literal that spans multiple lines. Solution: hitps:www-davekuhiman.ergpython_book_01 hem! 2ov26t97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises ‘This string spans several lines because it is a little long. * Use the string join operation to create a string that contains a colon as a separator. Solution: >>> content = [] >>> content. append(' finch’) >>> content..append( ' sparrow’) >>> content .append('thrush') >>> content .append( jay") >>> contentstr = ':'.join(content) >>> print contentstr finch: sparrow: thrush: jay * Use string formatting to produce a string containing your last and first names, separated by a comma. Solution: >>> first = ‘Dave’ >>> last = ‘Kuhlman’ >>> full = ‘Ks, Xs’ % (last, first, ) >>> print full Kuhlman, Dave Incrementally building up large strings from lots of small strings -- the old way -- Since strings in Python are immutable, appending to a string requires a re-allocation. So, it is faster to append to a list, then use join. Example: In [25]: strlist = [] In [26]: strlist.append('Line #1") In [27]: strlist.append('Line #2") In [28]: strlist.append('Line #3") In [29]: str = '\n'.join(strlist) In [30]: print str Line #1 Line #2 Line #3 Incrementally building up large strings from lots of small strings -- the new way -- The += operation on strings has been optimized. So, when you do this stri += str2, ‘even many times, it is efficient. The translate method enables us to map the characters in a string, replacing those in one table by those in another. And, the naketrans function in the string module, makes it easy to create the mapping table: import string hitps:www-davekuhiman.ergpython_book_01 hem! irae911724, 9.01 PM [APytnon Book: Begining Python, Advanced Python, and Python Exercises def test(): a = ‘axbycz’ t = string.maketrans(‘abc', '123') print a print a.translate(t) test() 1.4.3.1 The new string.format method The new way to do string formatting (which is standard in Python 3 and perhaps preferred for new code in Python 2) is to use the string.foraat method. See here: ‘+ http://docs.python.org/2/library/stdtypes.html##str. format * http://docs.python,org/2/library/string,htm|#format-string-syntax ‘* http://docs.python.org/2/library/string. htm|#format-specification-mini- language Some examples: In [1]: ‘aaa {1} bbb {@} ccc {1} ddd’ .format('xx', "yy", ) out[1]: ‘aaa yy bbb xx ccc yy ddd" In [2]: ‘number: {@:5d} ok’ .format(25) Out[2]: ‘number: 00025 ok" In [4]: ‘nt: {num1} m2: {num2}".format(num2=25, numi=100) out[4]: ‘nt: 1@@ 2: 25° In [S]: ‘nt: {num1} m2: {num2} again: {num1}".format(num2=25, num1=100) Out[5]: ‘nt: 100 n2: 25 again: 100" In [6]: ‘number: {:@5d} ok’. format (25) out [6]: ‘number: 98025 ok In [7]: values = {‘name': ‘dave’, ‘hobby’: ‘birding’} In [8]: ‘user: {name} activity: {hobby}' format (**values) out[8]: ‘user: dave activity: birding’ 1.4.3.2 Unicode strings Representing unicode: In [96]: a = u'abed" In [97]: a out[(97]: u'abed" In [98]: b = unicode('efgh') In [99]: b out[99]: u'efgh’ Convert to unicode: 2_string.decode(encoding). Examples: In [102]: ‘abcd' .decode('utf-8") out[1@2]: u'abed" In [103]: In [104]: ‘abcd’ .decode(sys.getdefaultencoding()) out[1e4]: u'abed" Convert out of unicode: a_unicode_string.encode(encoding). Examples: https: www-davekuhiman orgfpytnen_book_.himl 22619117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises In [107]: a = u'abed" In [108]: a.encode(‘utf-8") out[108]: ‘abcd" In [109]: a.encode(sys.getdefaultencoding()) out(109]: ‘abed" In [116]: b = u'Sel\xe7uk" In [111]: print b.encode(‘utf-8") Selcuk Test for unicode type -- Example: In [122]: import types In [123]: a = u'abed" In [124]: type(a) is types.UnicodeType out(124]: True In [125]: In [126]: type(a) is type(u'') out [126]: True Or better: In [127]: isinstance(a, unicode) Out[127]: True An example with a character "c" with a hachek: In [135]: name = "Ivan Krsti\xc4\x87" In [136]: name.decode(‘utf-8") out[136]: u'Ivan Krsti\ue1e7" In [137]: In [138]: len(name) out(138]: 12 In [139]: len(name.decode(‘utf-8")) out(139]: 11 You can also create a unicode character by using the unichr() built-in function: In [2]: In [3] out(3] In [6] In [7] Out[7) In [8]: aazbb a= ‘aa’ + unichr(170) + ‘bb! a u'aa\xaabb" b = a.encode('utf-8") b "aa\xc2\xaabb" print b Guidance for use of encodings and unicode -- If you are working with a multibyte character set: 1. Convert/decode from an external encoding to unicode early (ay_string.decode(encoding)). 2. Do your work in unicode. 3. Convert/encode to an external encoding /ate (ay_string-encode(encocing))- titpshndavekuhtman oripython_ ook 01m 2312619117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises For more information, see: * Unicode In Python, Completely Demystified - ‘nttp://farmdev.com/talks/unicode/ + PEP 100: Python Unicode Integration -- http://www. python,ora/dev/peps/pep- 0100/ + In the Python standard library: © codecs -- Codec registry and base classes -- http://docs. python.org/2/library/codecs. html#module-codecs © Standard Encodings http://docs.python.org/2/library/codecs. html#standard-encodings If you are reading and writing multibyte character data from or to a file, then look at the codecs.open() in the codecs module http://docs.python.org/2/library/codecs.html# codecs.open. Handling multi-byte character sets in Python 3 is easier, I think, but different. One hint is to use the encoding keyword parameter to the open built-in function. Here is an ‘example: def test(): infile = open('infile1.txt', ‘r', encoding="utf-8") outfile = open(‘outfilel.txt', ‘w', encoding="utf-8") for line in infile: line = line.upper() outfile.write(line) infile.close() outfile.close() test() 14.4 Dictionaries A dictionary is a collection, whose values are accessible by Key. It is a collection of name-value pairs. The order of elements in a dictionary is undefined. But, we can iterate over (1) the keys, (2) the values, and (3) the items (key-value pairs) in a dictionary. We can set the value of a key and we can get the value associated with a key. Keys must be immutable objects: ints, strings, tuples, ... Literals for constructing dictionari di =<} d2 = {key1: valuei, key2: value2, } Constructor for dictionaries -- aict() can be used to create instances of the class dict: Some examples: dict({*one': 2, ‘two’: 3}) dict({‘one': 2, ‘two’: 3}.items()) https: www-davekuhiman orgfpytnen_book_01.himl 2arzet911724, 9.01 PM [APytnon Book: Begining Python, Advanced Python, and Python Exercises dict({'one': 2, ‘two': 3).iteritens()) dict(zip((‘one’, ‘two'), (2, 3))) dict([['two', 3], ['one’, 2]]) dict(one=2, two=3) dict([(['one’, ‘two' If 21, i) for i in (2, 3)]) For operations on dictionaries, see htt use: //docs.python.org/lib/typesmappin. >>> help({}) Or: >>> dir({}) Indexing -- Access or add items to a dictionary with the indexing operator [ 1. Example: In [102]: dicta = (} In [103]: dict1[‘name'] = ‘dave’ In [104]: dicta['category'] = 38 In [105]: dict1 out [105]: {‘category’: 38, ‘name’: ‘dave') ‘Some of the operations produce the keys, the values, and the items (pairs) in a dictionary. Examples: In [43]: d= {‘aa": 111, ‘bb': 222} In [44]: d.keys() out(4a]: [‘aa", ‘bb'] In [45]: d.values() out(45]: (111, 222] In [46]: d.items() out(46]: [(‘aa", 111), (‘bb', 222)] When iterating over large dictionaries, use methods sterkeys(), itervalues(), and sxerstens(), Example: In [47]: In [47]: d = {("aa’: 111, ‘bb’: 222) In [48]: for key in d.iterkeys(): print key aa bb To test for the existence of a key in a dictionary, use the in operator or the nydict.has_key(k) method. The in operator is preferred. Example: https: hwww-davekuhiman orgfpytnon_book_1.himl 25126197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises >>> d= {"tomato': 101, ‘cucumber’: 102} >>> k = "tomato" >>> k ind True >>> dhas_key(k) True You can often avoid the need for a test by using method get. Example: >>> d= {"tomato': 101, cucumber" >>> d.get('tomato’, -1) 101 >>> d.get(‘chard’, -1) ol >>> if d.get(‘eggplant') is None ses print ‘missing’ 102} missing Dictionary "view" objects provide dynamic (automatically updated) views of the keys or the values or the items in a dictionary. View objects also support set operations. Create views with nycict.viewkeys(), nydict.viewalues(), and nydict.vienitens(). See: http://docs.python.org/2/library/stdtypes.html:#dictionary-view-objects. The dictionary setéefauit method provides a way to get the value associated with a key from a dictionary and to set that value if the key is missing. Example: In [106]: a out[106]: (} In [108]: a.setdefault(‘cc’, 33) out[108]: 33 In [109]: a out[199]: {"cc": 33} In [110]: a.setdefault('cc', 44) out[119]: 33 In [111]: a out(111]: {"ec": 33} Exercises: * Write a literal that defines a dictionary using both string literals and variables containing strings. Solution >>> first = ‘Dave’ >>> last = ‘Kuhlman’ >>> name_dict = (first: last, ‘Elvis’: 'Presley'} >>> print name_dict {'Dave': ‘Kuhlman’, ‘Elvis’: 'Presley'} + Write statements that iterate over (1) the keys, (2) the values, and (3) the items in a dictionary. (Note: Requires introduction of the for statement.) hitps:www-davekuhiman.ergpython_book_01 hem! 26126197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises Solutions: >>> d= (a's 111, ‘bb's 222, ‘cc's: >>> for key in d.keys(): print key 333) aa cc bb >>> for value in d.values(): print value 11 333 222 >>> for item in d.items(): print item (aa‘, 111) (rect, 333) ('bb', 222) >>> for key, value in d.itens(): print key, ‘::', value at: 1. cer: 333 bb :: 222 Additional notes on dictionaries: * You can use iterkeys(), itervalues(), iteritens() to obtain iterators over keys, values, and items. + A dictionary itself is iterable: it iterates over its keys. So, the following two lines are equivalent: for k in mybict: print k for k in myDict.iterkeys(): print k * The in operator tests for a key in a dictionary. Example: In [52]: mydict = {‘peach': ‘sweet’, ‘lemon': ‘tangy'} In [53]: key = ‘peach’ In [54]: if key in mydict: : print mydict[ key] sweet 1.4.5 Files Open a file with the open factory method. Example: hitps:www-davekuhiman.ergpython_book_01 hem! 2rie6t97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises In [28]: f = open(‘mylog.txt', ‘w') In [29]: f.write('message #1\n") In [3@]: f.write('message #2\n') In [31]: f.write('message #3\n‘) In [32]: f.close() In [33]: f = file(*mylog.txt', ‘r') In [34]: for line in f: : print line, message #1 message #2 message #3 In [35]: f.close() Notes: * Use the (built-in) open(path, sode) function to open a file and create a file object. You could also use Fi1e(), but open() is recommended. + A file object that is open for reading a text file supports the iterator protocol and, therefore, can be used in a for statement. It iterates over the ines in the file, This is most likely only useful for text files. * open is a factory method that creates file objects. Use it to open files for reading, writing, and appending. Examples: infile = open(‘myfile.txt', ‘r') # open for reading outfile = open(‘myfile.txt', ‘w') # open for (over-) writing log = open(‘nyfile.txt', ‘a") # open for appending to existing * When you have finished with a file, close it. Examples: infile.close() outfile.close() * You can also use the with: statement to automatically close the file. Example: with open(‘tmpe1.txt', ‘r') as infile: for x in infile: print x, The above works because a file is a context manager: it obeys the context manager protocol. A file has methods _enter__and _exit_, and the _exit_ method automatically closes the file for us. See the section on the with: statement. + In order to open multiple files, you can nest with: statements, or use a single with: statement with multiple "expression as target” clauses. Example hitps:www-davekuhiman.orgpython_book_01 hem! 2arz6t97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises def test(): # # use multiple nested with: statements. with open(‘small_file.txt', ‘r') as infile: with open('tnp_outfile.txt', ‘w') as outfile: for line in infil outfile.write(‘line: %s' % line.upper()) print infile print outfile # # use a single with: statement. with open('small_file.txt', ‘r') as infile, \ open('tmp_outfile.txt', ‘w') as outfile: for line in infile: outfile.write('line: %s' % line.upper()) print infile print outfile test() + fi1e is the file type and can be used as a constructor to create file objects. But, open is preferred. + Lines read from a text file have a newline. Strip it off with something like: Lne.strip('\n") * For binary files you should add the binary mode, for example: ro, wo. For more about modes, see the description of the open() function at Built-in Function: ‘jttp://docs. python. ora/lib/built-in-funcs.htm. + Learn more about file objects and the methods they provide at: 2.3.9 File Objects -- http://docs.python.org/2/library/stdtypes.html#file-objects. You can also append to an existing file. Note the "a" mode in the following example: In [39]: f = open(‘mylog.txt', ‘a') In [40]: f.write('message #4\n") In [41]: f.close() In [42]: f = file(*mylog.txt', ‘r') In [43]: for line in f: : print line, message #1 message #2 message #3 message #4 In [44]: f.close() For binary files, add "b" to the mode. Not strictly necessary on UNIX, but needed on MS Windows. And, you will want to make your code portable across platforms. Example: hitps:www-davekuhiman.ergpython_book_01 hem! 2orzet97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises In [62]: import zipfile In [63]: outfile = open(‘tmpi.zip', ‘wo') In [64]: zfile = zipfile.zipFile(outfile, ‘w', zipfile.ZIP_DEFLATED) In [65]: zfile.writestr(‘entry1', ‘my heroes have always been couboys') In [66]: zfile.writestr(‘entry2', ‘and they still are it seems') In [67]: zfile.writestr(‘entry3", ‘sadly in search of and") In [68]: zfile.writestr(‘entry4', ‘on step in back of") In [69] In [70]: zfile.writestr(‘entry4', ‘one step in back of) In [72]: zfile.close() In [73]: outfile.close() In [75] $ $ unzip -lv tmp1.zip Archive: tmpl.zip Length Method Size Ratio Date RC-32 6674921 entry1 34 95-29-08 27 25-29-08 ledasf3sd entry2 22 05-29-08 3fd17fda_ entry3 18 05-29-08 d55182e6 entry4 19 05-29-08 1ag92acd entry4 37 05-29-08 17:09 €213708c entryS 157 169 -8% 6 files In [71]: zfile.writestr(‘entry5', ‘themselves and their slow moving ways") Exercises: + Read all of the lines of a file into a list. Print the 3rd and Sth lines in the file/list. Solution: In [55]: # = open('tmpi.txt", 'r*) In [56]: lines = f.readlines() In [57]: f.close() In [58]: lines out[58]: [‘the\n', "big\n', ‘brown\n’, ‘dog\n’, *had\n', ‘long\n',, In [59]: print lines[2] brown In [61]: print lines[4] had ‘had More notes: + Strip newlines (and other whitespace) from a string with methods strip), asteip(), and rstrip(). * Get the current position within a file by using yfite.te11(). * Set the current position within a file by using aye: se os.seex_cur and os.seex_exo, For example: © F.seck(2, os.Seek_cur) advances the position by two © F.seck(-3, 05.8eeK END) Sets the position to the third to last. © F.seck(25) sets the position relative to the beginning of the file. hitps:www-davekuhiman.ergpython_book_01 hem! seek(). It may be helpful to 026197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises 1.4.6 Other built-in types Other built-in data types are described in section Built-in Types http://docs.python.org/lib/types.html in the Python standard documentation. 1.4.6.1 The None value/type The unique value None is used to indicate "no value", "nothing", "non-existence", etc There is only one tone value; in other words, it's a singleton. Use is to test for none. Example: >>> flag = None >>> >>> if flag is None: print ‘clear’ clear >>> if flag is not None: see print "hello" >>> 1.4.6.2 Boolean values True and False are the boolean values. The following values also count as false, for example, in an if: statement: False, numeric zero, None, the empty string, an empty list, an empty dictionary, any empty container, etc. All other values, including true, act as true values. 1.4.6.3 Sets and frozensets A set is an unordered collection of immutable objects. A set does not contain duplicates. Sets support several set operations, for example: union, intersection, difference, ... A frozenset is like a set, except that a frozenset is immutable. Therefore, a frozenset is hash-able and can be used as a key in a dictionary, and it can be added to a set. Create a set with the set constructor. Examples: >> a= set() >> a set([]) >>> avadd(*aa") >>> avadd(*bb*) >> a set(['aa', "bb"]) >>> b = set([11, 22]) >> b hitps:www-davekuhiman.ergipython_book_01 hem! aniget97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises set([11, 22]) >>> ¢ = set([22, 33]) >>> b.union(c) set([33, 11, 22]) >>> b.intersection(c) set ([22]) For more information on sets, see: Set Types -- set, frozense! http://docs.python.org/lib/types-set.html 1.5 Functions and Classes -- A Preview Structured code -- Python programs are made up of expressions, statements, functions, classes, modules, and packages. Python objects are first-class objects. Expressions are evaluated. Statements are executed. Functions (1) are objects and (2) are callable. Object-oriented programming in Python. Modeling "real world" objects. (1) Encapsulation; (2) data hiding; (3) inheritance. Polymorphism. Classes -- (1) encapsulation; (2) data hiding; (3) inheritance. An overview of the structure of a typical class: (1) methods; (2) the constructor; (3) class (static) variables; (4) super/subclasses. 1.6 Statements 1.6.1_Assignmeni men Form -- target = expression. Possible targets: * Identifier + Tuple or list -- Can be nested. Left and right sides must have equivalent structure, Example: >>> XY, Z= ML, 22, 33 >>> Du yy Z] = 111, 222, 333 >>> a, (b, ¢) = 11, (22, 33) >>> a, B= 11, (22, 33) ‘This feature can be used to simulate an enum: hitps:www-davekuhiman.ergpython_book_01 hem! aa6t9117124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises In [22]: LITTLE, MEDIUM, LARGE = range(1, 4) In [23]: LITTLE out [23]: 2 In [24]: MEDIUM out (24]: 2 ‘+ Subscription of a sequence, dictionary, etc. Example: oA + At In [10]: a = range(1@) In [a1]: a out(11}: [®, 1, 2, 3, 4, 5, 6, 7, 8 9] In [12]: a[3] = ‘abe’ In [23]: a out(13]: [®, 1, 2, ‘abc’, 4, 5, 6, 7, 8 9] In [14] In [14]: b = {'aa's 11, ‘bb’: 22} In [15]: b Out[1s]: {'aa': 11, "bb": 22} In [16]: b['bb'] = 1000 In [17]: b['cc'] = 2000 In [18]: b out[18]: {"aa": 11, "bb": 2000, ‘cc’: 2000} slice of a sequence Note that the sequence must be mutable. Example: In [1]: a = range(10) In [2]: a out(2]: [8 1, 2, 3, 4, 5, 6, 7, 8 9] In [3]: a[2:5] = (21, 22, 33, 44, 55, 66] In [4]: a out[4]: [@, 1, 11, 22, 33, 44, 55, 66, 5, 6, 7, 8, 9] tribute reference -- Example: >>> class MyClass: pass >>> anObj = MyClass() >>> anObj.desc = ‘pretty >>> print andbj.desc pretty ‘There is also augmented assignment. Examples: >>> >>> >>> >>> >>> >>> index index += 1 index += 5 index += F(x) index -= 1 index *= 3 Things to note: https: hwww-davekuhiman orgfpytnon_book_1.himl 33126197124, 9:01 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises * Assignment to a name creates a new variable (if it does not exist in the namespace) and a binding. Specifically, it binds a value to the new name. Calling a function also does this to the (formal) parameters within the local namespace. + In Python, a language with dynamic typing, the data type is associated with the value, not the variable, as is the case in statically typed languages. + Assignment can also cause sharing of an object. Example: obj obj2 AC) obj Check to determine that the same object is shared with id(obj) or the is operator. Example: In [23]: a = range(10) In [24]: a Out[24]: [@, 1, 2, 3, 4, 5, 6, 7, 8 9] In [25]: b= a In [26]: b out[26]: [8 1, 2, 3, 4) 5, 6 7, 8 9] In [27]: b[3] = 333 In [28]: b out[28]: [®, 1, 2, 333, 4, 5, 6, 7, 8 9] In [29]: a out[29]: [®, 1, 2, 333, 4, 5, 6, 7, 8 9] In [30]: a is b Out [38]: True In [31]: print id(a), id(b) 31037928 31037920 * You can also do multiple assignment in a single statement. Example: In [32]: In [33] out [33] In [34] out [34] In [35] In [35] In [35] In [36] out[36]: a= b= [11, 22] ais b True + You can interchange (swap) the value of two variables using assignment and packing/unpacking: >>> a= dl >>> b = 222 >> a, b=b,a >a hitps:ww-davekuhiman.orgpython_book_01 hem! swiz6t97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises 222 >>> b 11. 1.6.2 import statement Make module (or objects in the module) available. What import does * Evaluate the content of a module. * Likely to create variables in the local (module) namespace. + Evaluation of a specific module only happens once during a given run of the program. Therefore, a module is shared across an application. * Amodule is evaluated from top to bottom. Later statements can replace values created earlier. This is true of functions and classes, as well as (other) variables. + Which statements are evaluated? Assignment, class, def, ... * Use the following idiom to make a module both run-able and import-able: if _name__ == *_main_ # import pdb; pdb.set_trace() main() # or “test()" or some other function defined in modu! Notes: © The above condition will be true only when the module is run as a script and will not be true when the module is imported. © The line containing pab can be copied any place in your program and un- commented, and then the program will drop into the Python debugger when that location is reached Where inport looks for modules: + sys.path shows where it looks. * There are some standard places. Add additional directories by setting the environment variable pytHonparH. You can also add paths by modifying sys.path, for example: import sys sys-path.insert(®, '/path/to/my/nodule’ ) Packages need a file named _init_.py. hitps:www-davekuhiman.ergpython_book_01 hem! asiz6t97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises * Extensions -- To determine what extensions import looks for, do: >>> import imp >>> imp.get_sutfixes() [(.s0", rb", 3), (‘module.so", ‘rb', 3), C'apy's "U's Ds Cupye's * Forms of the inport statement + inport a -- Names in the local (module) namespace are accessible with the dot operator. + Inport @ as 8 -- Import the module A, but bind the module object to the variable B. import a1, #2 -- Not recommended fron & import 8 fron A import 81, 82 fron A import 8 as ¢ fron A inport * -- Not recommended: clutters and mixes name-spaces. fron 4.8 inport ¢ -- (1) Possibly import object c from module # in package or (2) possibly import module c from sub-package # in package * Inport .8.¢ =~ To reference attributes in c, must use fully-qualified name, for example use a.s.c.0 to reference o inside of c More notes on the inport statement: + The import statement and packages -- A file named _init_.py is required in a package. This file is evaluated the first time either the package is imported or a file in the package is imported. Question: What is made available when you do port aPackage? Answer: All variables (names) that are global inside the __init_.py module in that package. But, see notes on the use of _a11_: The import statement -- http://docs.python.org/ref/import. htm! * The use of i¢ _nane_ == "_nain_*: -- Makes a module both import-able and executable. * Using dots in the import statement manual From the Python language reference "Hierarchical module names:when the module names contains one or more dots, the module search path is carried out differently. The sequence of identifiers up to the last dot is used to find a package; the final identifier is then searched inside the package. A package is generally a subdirectory of a directory on sys.path that has a file —init_.py." See: The import statement -- http://docs.python.org/ref/import.htm| Exercises: + Import a module from the standard library, for example re. hitps:www-davekuhiman.orgpython_book_01 hem! 61261‘9117724, 9:01 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises + Import an element from a module from the standard library, for example import conpite from the re module. * Create a simple Python package with a single module in it. Solution 1, Create a directory named sinplepacxage in the current directory. 2. Create an (empty) _init_.py in the new directory. 3. Create an sinple.py in the new directory. 4. Add a simple function name testt in simple.py. 5. Import using any of the following: >>> import simplepackage. simple >>> from simplepackage import simple >>> from simplepackage.simple import test >>> from simplepackage. simple import testi as mytest 1.6 print statement print Sends output to sys.stdout. It adds a newline, unless an extra comma is added. Arguments to print: Multiple items -- Separated by commas. End with comma to suppress carriage return, Use string formatting for more control over output. Also see various "pretty-printing” functions and methods, in particular, pprint- See 3.27 pprint -- Data pretty printer -- http://docs.python.ora/lib/module- pprint.htm. String formatting -- Arguments are a tuple. Reference: 2.3.6.2 String Formatting Operations ro/lib/typesse Can also use sys.stdout. Note that a carriage return is not automatically added. Example: >>> import sys >>> sys. stdout .write( ‘hello\n') Controlling the destination and format of print -- Replace sys.stdout with an instance of any class that implements the method «rite taking one parameter. Example import sys class Writer: def _init_(self, file_name): self.out_file = file(file_name, ‘a') def write(self, msg): self .out_file.write('[[%s]' % msg) hitps:ww-davekuhiman.ergpython_book_01 hem! arizet97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises def close(self): self.out_file.close() def test(): writer = Writer('outputfile.txt') save_stdout = sys.stdout sys.stdout = writer print ‘hello’ print ‘goodbye’ writer.close() # Show the output. tp_file = file('outputfile.txt') sys.stdout = save_stdout content = tmp_file.read() tmp_file.close() print content test() There is an alternative form of the print statement that takes a file-like object, in particular an object that has a write method. For example: In [1]: outfile = open('tmp.log', ‘w') In [2]: print >> outfile, ‘Message #1' In [3]: print >> outfile, ‘Message #2" In [4]: print >> outfile, ‘Message #3" In [5]: outfile.close() In [6] In [6] In [7] infile = open("tmp.log’, ‘r') for line in infile: print ‘Line:', Line.rstrip(*\n') Line: Message #1 Line: Message #2 Line: Message #3 In [8]: infile.close() Future deprecation warning -- There is no print statement in Python 3. There is a print built-in function. 1.6.4 if: elif: else: statement ‘A template for the fr: statement: if conditiont: statements elif condition2 statements elif condition3 statements else: statements The e1i¢ and else clauses are optional. hitps:www-davekuhiman.ergpython_book_01 hem! aei26197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises Conditions -- Expressions -- Anything that returns a value. Compare with evai() and Truth values: * False -- False, none, numeric zero, the empty string, an empty collection (list or tuple or dictionary or ...) * True -- true and everything else. Operators: + ane and or -- Note that both and and or do short circuit evaluation. # not * is and is not -- The identical object. Cf. a is b and id(a) == id’). Useful to test for None, for example: if x is None: if x is not None: + in and not in -- Can be used to test for existence of a key in a dictionary or for the presence of a value in a collection. The in operator tests for equ y, not identity. Example: >>> d= {'aa': 111, bb": 222} >>> taal ind True >>> ta’ not in d False >>> txt ind False * Comparison operators, for example There is an if expression. Example: >>> a= 'aa’ >>> b= ‘bb! >>> x = 'yes' if a == b else ‘no’ >>> x Ree Notes: + The elif: clauses and the else: clause are optional. hitps:www-davekuhiman.ergpython_book_01 hem! 9126191724, 901 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises * The if:, elif:, and else: clauses are all header lines in the sense that they are each followed by an indented block of code and each of these header lines ends with a colon. (To put an empty block after one of these, or any other, statement header line, use the pass statement. It's effectively a no-op.) * Parentheses around the condition in an i: or eii¢: are not required and are considered bad form, unless the condition extends over multiple lines, in which case parentheses are preferred over use of a line continuation character (backslash at the end of the line) Exercises: + Write an if statement with an and operator. * Write an i¢ statement with an or operator. * Write an i¢ statement containing both and and or operators. 1.6.5 for: statement Iterate over a sequence or an “iterable" object. Form: for x in y: block Iterator -- Some notes on what it means to be iterable: * An iterable is something that can be used in an iterator context, for example, in a for: statement, in a list comprehension, and in a generator expression. + Sequences and containers are iterable. Examples: tuples, lists, strings, dictionaries + Instances of classes that obey the iterator protocol are iterable. See ‘http://docs. python.org/lib/typeiter.html. + We can create an iterator object with built-in functions such as iter) and enunerate(). See Built-in Functions -- http://docs.python,org/lib/built-in= funcs.html in the Python standard library reference. * Functions that use the yie1e statement, produce an iterator, although it’s actually called a generator. + An iterable implements the iterator interface and satisfies the iterator protocol. The iterator protocol: _iter_() and next() methods. See 2.3.5 Iterator Types -- (http://docs.python.ora/lib/typeiter.html) Testing for “iterability” + If you can use an object in a for: statement, it's iterable. + If the expresion iter(obj) does not produce a typeereor exception, it's iterable. Some ways to produce iterators: + Ater() and enunerate() -- See: http://docs.python.org/lib/built-in-funcs.html. # sone_dictiterkeys(), sone_dict-itervalues(), sone_dict .iteritens() + Use a sequence in an iterator context, for example in a for statement. Lists, tuples, dictionaries, and strings can be used in an iterator context to produce an iterator, hitps:www-davekuhiman.ergpython_book_01 hem! 40ra619117724, 9:01 PM [APython Book: Beginning Python, Advanced Python, and Python Exercises * Generator expressions -- Latest Python only. Syntactically like list comprehensions, but (1) surrounded by parentheses instead of square brackets and (2) use lazy evaluation. + Aclass that implements the iterator protocol -- Example: class A(object): def init__(self): Self.data = [11,22,33] self.idx = 0 def _iter_(self): return’ self def next(self): if self.idx < len(self.data): x = self.data[self. idx] self.idx t=1 return x else: raise StopIteration def test(): a= A() for x in a: print x test() Note that the iterator protocol changes in Python 3. + A function containing a yield statement. See: © Yield expressions ~~ http://docs. python.org/2/reference/expressions html#yield-expressions © The vield statement -- http://docs.python.or/2/reference/simple_stmts.html#the-yield- statement ‘+ Also see itertools module in the Python standard library for much more help with iterators: itertools — Functions creating iterators for efficient looping -- http://docs.python.org/2/library/itertools.html#module-itertools The for: statement can also do unpacking, Example: In [25]: items = [‘apple', ‘banana‘, ‘cherry’, ‘date‘] In [26]: for idx, item in enumerate(itens): cee print '%d. %s" % (idx, item, ) apple banana cherry date The for statement can also have an optional else: clause. The else: clause is executed if the for statement completes normally, that is if a break statement is not executed. https: www-davekuhiman orgfpytnen_book_.himl 4176197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises Helpful functions with for: + enunerate(iterabie) -- Returns an iterable that produces pairs (tuples) containing count (index) and value. Example: >>> for idx, value in enumerate([11,22,33]): print idx, value * range({start,] stop[, step]) and xrange({start,] stop, step]) List comprehensions -- Since list comprehensions create lists, they are useful in for statements, although, when the number of elements is large, you should consider using a generator expression instead. A list comprehension looks a bit like a for statement, but is inside square brackets, and it is an expression, not a statement. Two forms (among others): © [F(x) for x in iterabie] * [F(%) for x in iterable if t(x)] Generator expressions -- A generator expression looks similar to a list comprehension, except that it is surrounded by parentheses rather than square brackets, Example: In [28]: items = [‘apple', ‘banana', ‘cherry’, ‘date’ ] In [29]: gent = (item.upper() for item in items) In [30]: for x in gen: print "x: APPLE BANANA CHERRY DATE Exercises: + Write a list comprehension that returns all the keys in a dictionary whose associated values are greater than zero. © The dictionary: ¢"aa': 11, "cc's 33, "dd": -95, "bb": 22) © Solution: [x{e] for x in my_dict.iteritems() i x[2] > @] + Write a list comprehension that produces even integers from 0 to 10. Use a for statement to iterate over those values. Solution: for x in [y for y in range(1®) if y % 2 print "x: %s" % x hitps:www-davekuhiman.ergpython_book_01 hem! ant97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises * Write a list comprehension that iterates over two lists and produces all the combinations of items from the lists. Solution: In [19]: a = range(4) In [20]: b = [11,22,33] In [21]: a out[22]: [@, 1, 2, 3] In [22]: b out[22]: [11, 22, 33] In [23]: ¢ = [(x, y) for x in a for y in b] In [24]: print c [(@, 11), (®, 22), (®, 33), (1, 11), (2, 22), (2, 33), (2, 11), (2, 22), (2, 33), (3, 11), (3, 22), (3, 33)] But, note that in the previous exercise, a generator expression would often be better. ‘A generator expression is like a list comprehension, except that, instead of creating the entire list, it produces a generator that can be used to produce each of the elements. The break and continue statements are often useful in a for statement. See continue and break statements The for statement can also have an optional else: clause. The else: clause is executed if the for statement completes normally, that is if a break statement is not executed. Example: for item in data: if item > 100: valuel = item break else: valuel = ‘not found’ print ‘valuet:', valuet When run, it prints: valuel: not found 1.6.6 _whil Form: statement while condition block The while: statement is not often used in Python because the fer: statement is usually more convenient, more idiomatic, and more Pythonic. Exercises: hitps:www-davekuhiman.ergpython_book_01 hem! caret97124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises * Write a white statement that prints integers from zero to 5. Solution: count = @ while count < count += 1 print count The break and continue statements are often useful in a while statement. See continue ni rk ment The while statement can also have an optional else: clause. The else: clause is executed if the wnite statement completes normally, that is if @ break statement is not executed. 1.6.7 continue and break statements The break statement exits from a loop. The continue statement causes execution to immediately continue at the start of the loop. Can be used in for: and while:. When the for: statement or the while: statement has an else: clause, the block in the else: clause is executed only if a break statement is not executed. Exercises: * Using break, write a while statement that prints integers from zero to 5. Solution count = @ while True count += 1 if count > 5: break print count Notes: © A for statement that uses range() oF xrange() would be better than a while statement for this use + Using continue, write a while statement that processes only even integers from 0 to 10. Note: x is the modulo operator. Solution: count = @ while count < 10: count #= 1 if count % 2 continue print count hitps:www-davekuhiman.ergpython_book_01.hem! 4ara6197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises 1.6.8 th Exceptions are a systematic and consistent way of processing errors and “unusual” events in Python. : except: statement Caught and un-caught exceptions -- Uncaught exceptions terminate a program. The try: statement catches an exception. Almost all errors in Python are exceptions. Evaluation (execution model) of the try statement -- When an exception occurs in the try block, even if inside a nested function call, execution of the try block ends and the except clauses are searched for a matching exception. Tracebacks -- Also see the traceback module: http://docs,python.org/lib/module- traceback.htm! Exceptions are classes. Exception classes -- subclassing, args. ‘An exception class in an except: clause catches instances of that exception class and all subclasses, but not superclasses. Built-in exception classes -- See: * Module exceptions. * Built-in exceptions -- http://docs.python.org/lib/module-exceptions,html. User defined exception classes -- subclasses of exception. Example: try raise RuntimeError('this silly error’) except RuntimeError, exp: print "[[[%s]]]" % exp Reference: http://docs.python.org/lib/module-exceptions.html You can also get the arguments passed to the constructor of an exception object. In the above example, these would be: exp. args Why would you define your own exception class? One answer: You want a user of your code to catch your exception and no others. hitps:www-davekuhiman.ergpython_book_01 hem! 45261911724, 9.01 PM [APytnon Book: Begining Python, Advanced Python, and Python Exercises Catching an exception by exception class catches exceptions of that class and all its subclasses. So: except SomeExceptionClass, exp: matches and catches an exception if SomeExceptionClass is the exception class or a base class (superclass) of the exception class, The exception object (usually an instance of some exception class) is bound to exp. A more "modern" syntax is: except SomeExceptionClass as exp: So: class MyE(ValueError): pass try: raise MyE() except Valuetrror: print ‘caught exception’ will print "caught exception", because valuetrrer is a base class of nye. Also see the entries for "EAFP" and "LBYL" in the Python glossary: http://docs.python.org/3/glossary.htm|. Exercises: + Write a very simple, empty exception subclass. Solution: class MyE (Exception): pass ‘+ Write a try:except: statement that raises your exception and also catches it. Solution: try: raise MyE(‘hello there dave") except MyE, e: print e 1.6.9 raise statement Throw or raise an exception Forms: = raise instance raise MyExceptionClass value) -- preferred, https hwww-davekuhiman orgfpytnen_book_01.himl 4526197124, 9:01 PM |APythan Book: Baginning Python, Advanced Python, and Pythen Exercises © raise MyExceptionclass, value The raise statement takes: An (instance of) a built-in exception class An instance of class exception or An instance of a built-in subclass of class exception or An instance of a user-defined subclass of class exception or One of the above classes and (optionally) a value (for example, a string or a tuple). See http://docs.python.org/ref/raise.html. For a list of built-in exceptions, see http://docs.python.org/lib/module- exceptions.html. The following example defines an exception subclass and throws an instance of that subclass. It also shows how to pass and catch multiple arguments to the exception: class NotsobadError (Exception) : pass def test (x) try if x == @: raise NotsobadError('a moderately bad error’, ‘not too bad’) except Notsobadérror, e: print ‘Error args: %s' X (e.args, ) test(a) Which prints out the following: Error args: (‘a moderately bad error’, ‘not too bad") Notes: + In order to pass in multiple arguments with the exception, we use a tuple, or we pass multiple arguments to the constructor. The following example does a small amount of processing of the arguments: lotsobadError(Exception): n exception class. clas: def get_args(self): return ' -join(self.args) def test(x) try if x == @: raise NotsobadError('a moderately bad error’, ‘not too bad’) except Notsobadérror, e: print ‘Error args: {{{%s}}}' % (e.get_args(), ) hitps:www-davekuhiman.ergpython_book_01 hem! arrest