NumPy Cookbook

Table of Contents

NumPy Cookbook

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Winding Along with IPython

Introduction

Installing IPython

How to do it...

How it works...

Using IPython as a shell

How to do it...

How it works...

Reading manual pages

How to do it...

How it works...

Installing Matplotlib

How to do it...

Running a web notebook

Getting ready

How to do it...

How it works...

See also

Exporting a web notebook

How to do it...

Importing a web notebook

How to do it...

Configuring a notebook server

How to do it...

How it works...

Exploring the SymPy profile

Getting ready

How to do it...

2. Advanced Indexing and Array Concepts

Introduction

Installing SciPy

Getting ready

How to do it...

How it works...

Installing PIL

How to do it...

Resizing images

Getting ready

How to do it...

How it works...

See also

Creating views and copies

Getting ready

How to do it...

How it works...

Flipping Lena

How to do it...

See also

Fancy indexing

How to do it...

How it works...

Indexing with a list of locations

How to do it...

Indexing with booleans

How to do it...

How it works...

See also

Stride tricks for Sudoku

How to do it...

How it works...

Broadcasting arrays

How to do it...

3. Get to Grips with Commonly Used Functions

Introduction

Summing Fibonacci numbers

How to do it...

How it works...

See also

Finding prime factors

How to do it...

How it works...

Finding palindromic numbers

How to do it...

How it works...

There's more...

The steady state vector determination

How to do it...

How it works...

See also

Discovering a power law

How to do it...

How it works...

See also

Trading periodically on dips

Getting ready

How to do it...

How it works...

See also

Simulating trading at random

Getting ready

How to do it...

How it works...

See also

Sieving integers with the Sieve of Erasthothenes

How to do it...

4. Connecting NumPy with the Rest of the World

Introduction

Using the buffer protocol

Getting ready

How to do it...

How it works...

See also

Using the array interface

Getting ready

How to do it...

How it works...

See also

Exchanging data with MATLAB and Octave

Getting ready

How to do it...

See also

Installing RPy2

How to do it...

Interfacing with R

Getting ready

How to do it...

See also

Installing JPype

How to do it...

Sending a NumPy array to JPype

How to do it...

How it works...

See also

Installing Google App Engine

How to do it...

Deploying NumPy code in the Google cloud

How to do it...

How it works...

Running NumPy code in a Python Anywhere web console

How to do it...

How it works...

Setting up PiCloud

How to do it...

How it works...

5. Audio and Image Processing

Introduction

Loading images into memory map

Getting ready

How to do it...

How it works...

See also

Combining images

Getting ready

How to do it...

How it works...

See also

Blurring images

How to do it...

How it works...

Repeating audio fragments

How to do it...

How it works...

Generating sounds

How to do it...

How it works...

Designing an audio filter

How to do it...

How it works...

Edge detection with the Sobel filter

How to do it...

How it works...

6. Special Arrays and Universal Functions

Introduction

Creating a universal function

How to do it...

How it works...

Finding Pythagorean triples

How to do it...

How it works...

Performing string operations with chararray

How to do it...

How it works...

Creating a masked array

How to do it...

How it works...

Ignoring negative and extreme values

How to do it...

How it works...

Creating a scores table with recarray

How to do it...

How it works...

7. Profiling and Debugging

Introduction

Profiling with timeit

How to do it...

How it works...

Profiling with IPython

How to do it...

How it works...

Installing line_profiler

Getting ready

How to do it...

See also

Profiling code with line_profiler

How to do it...

How it works...

Profiling code with the cProfile extension

How to do it...

Debugging with IPython

How to do it...

How to do it...

Debugging with pudb

How to do it...

8. Quality Assurance

Introduction

Installing Pyflakes

Getting ready

How to do it...

Performing static analysis with Pyflakes

How to do it...

How it works...

Analyzing code with Pylint

Getting ready

How to do it...

How it works...

See also

Performing static analysis with Pychecker

How to do it...

Testing code with docstrings

How to do it...

How it works...

Writing unit tests

How to do it...

How it works...

Testing code with mocks

How to do it...

How it works...

Testing the BDD way

How to do it…

How it works...

9. Speed Up Code with Cython

Introduction

Installing Cython

How to do it...

Building a Hello World program

How to do it...

How it works...

Using Cython with NumPy

How to do it...

How it works...

Calling C functions

How to do it...

How it works...

Profiling Cython code

How to do it...

How it works...

Approximating factorials with Cython

How to do it...

How it works...

10. Fun with Scikits

Introduction

Installing scikits-learn

Getting ready

How to do it...

Loading an example dataset

How to do it...

Clustering Dow Jones stocks with scikits-learn

How to do it...

How it works...

Installing scikits-statsmodels

How to do it...

Performing a normality test with scikits-statsmodels

How to do it...

How it works...

Installing scikits-image

How to do it...

Detecting corners

Getting ready

How to do it...

How it works...

Detecting edges

How to do it...

Installing Pandas

How to do it...

Estimating stock returns correlation with Pandas

How to do it...

How it works...

Loading data as pandas objects from statsmodels

Getting ready

How to do it...

How it works...

Resampling time series data

How to do it...

How it works...

Index

NumPy Cookbook

NumPy Cookbook

Copyright © 2012 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: October 2012

Production Reference: 1181012

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-849518-92-5

www.packtpub.com

Cover Image by Avishek Roy (<roy007avishek88@gmail.com>)

Credits

Author

Ivan Idris

Reviewers

Alexandre Devert

Ludovico Fischer

Ryan R. Rosario

Acquisition Editor

Usha Iyer

Lead Technical Editor

Ankita Shashi

Technical Editors

Merin Jose

Rohit Rajgor

Farhaan Shaikh

Nitee Shetty

Copy Editor

Insiya Morbiwala

Project Coordinator

Vishal Bodwani

Proofreader

Clyde Jenkins

Indexer

Monica Ajmera Mehta

Production Coordinators

Arvindkumar Gupta

Manu Joseph

Cover Work

Arvindkumar Gupta

Manu Joseph

About the Author

Ivan Idris has an MSc in Experimental Physics. His graduation thesis had a strong emphasis on Applied Computer Science. After graduating, he worked for several companies as a Java Developer, Data Warehouse Developer, and QA Analyst. His main professional interests are business intelligence, big data, and cloud computing. He enjoys writing clean, testable code, and interesting technical articles. He is the author of NumPy 1.5 Beginner's Guide. You can find more information and a blog with a few NumPy examples at ivanidris.net.

I would like to dedicate this book to my family and friends. I would like to take this opportunity to thank the reviewers and the team at Packt for making this book possible. Thanks also goes to my teachers, professors, and colleagues, who taught me about science and programming. Last but not least, I would like to acknowledge my parents, family, and friends for their support.

About the Reviewers

Alexandre Devert is a computer scientist. To put his happy obsessions to good use, he decided to solve optimization problems, in both academic and industrial contexts. This included all kinds of optimization problems, such as civil engineering problems, packing problems, logistics problems, biological engineering problems—you name it. It involved throwing lots of science on the wall and seeing what sticks. To do so, he had to analyze and visualize large amounts of data quickly, for which Python, NumPy, Scipy, and Matplotlib excel. Thus, the latter are among the daily tools he has been using for a couple of years. He also lectures on Data mining at the University of Science and Technology of China, and uses those very same tools for demonstration purposes and to enlighten his students with graphics glittering of anti-aliased awesomeness.

I would like to thank my significant other for her understanding my usually hefty work schedule, and my colleagues, for their patience with my shallow interpretation of concepts such as a deadline.

Ludovico Fischer is a software developer working in the Netherlands. By day, he builds enterprise applications for large multinational companies. By night, he cultivates his academic interests in mathematics and computer science, and plays with mathematical and scientific software.

Ryan R. Rosario is a Doctoral Candidate at the University of California, Los Angeles. He works at Riot Games as a Data Scientist, and he enjoys turning large quantities of massive, messy data into gold. He is heavily involved in the open source community, particularly with R, Python, Hadoop, and Machine Learning, and has also contributed code to various Python and R projects. He maintains a blog dedicated to Data Science and related topics at http://www.bytemining.com. He has also served as a technical reviewer for NumPy 1.5 Beginner's Guide.

www.PacktPub.com

Support files, eBooks, discount offers and more

You might want to visit www.PacktPub.com for support files and downloads related to your book.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

http://PacktLib.PacktPub.com

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books. 

Why Subscribe?

Fully searchable across every book published by Packt

Copy and paste, print and bookmark content

On demand and accessible via web browser

Free Access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access.

Preface

We, NumPy users, live in exciting times. New NumPy-related developments seem to come to our attention every week or maybe even daily. When this book was being written, NumPy Foundation of Open Code for Usable Science was created. The Numba project—NumPy-aware, dynamic Python compiler using LLVM—was announced. Also, Google added support to their Cloud product Google App Engine.

In the future, we can expect improved concurrency support for clusters of GPUs and CPUs. OLAP-like queries will be possible with NumPy arrays.

This is wonderful news, but we have to keep reminding ourselves that NumPy is not alone in the scientific (Python) software ecosystem. There is Scipy, Matplotlib (a very useful Python plotting library),