Learning OpenCV 3 Computer Vision with Python - Second Edition

Table of Contents

Learning OpenCV 3 Computer Vision with Python Second Edition

Credits

About the Authors

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Setting Up OpenCV

Choosing and using the right setup tools

Installation on Windows

Using binary installers (no support for depth cameras)

Using CMake and compilers

Installing on OS X

Using MacPorts with ready-made packages

Using MacPorts with your own custom packages

Using Homebrew with ready-made packages (no support for depth cameras)

Using Homebrew with your own custom packages

Installation on Ubuntu and its derivatives

Using the Ubuntu repository (no support for depth cameras)

Building OpenCV from a source

Installation on other Unix-like systems

Installing the Contrib modules

Running samples

Finding documentation, help, and updates

Summary

2. Handling Files, Cameras, and GUIs

Basic I/O scripts

Reading/writing an image file

Converting between an image and raw bytes

Accessing image data with numpy.array

Reading/writing a video file

Capturing camera frames

Displaying images in a window

Displaying camera frames in a window

Project Cameo (face tracking and image manipulation)

Cameo – an object-oriented design

Abstracting a video stream with managers.CaptureManager

Abstracting a window and keyboard with managers.WindowManager

Applying everything with cameo.Cameo

Summary

3. Processing Images with OpenCV 3

Converting between different color spaces

A quick note on BGR

The Fourier Transform

High pass filter

Low pass filter

Creating modules

Edge detection

Custom kernels – getting convoluted

Modifying the application

Edge detection with Canny

Contour detection

Contours – bounding box, minimum area rectangle, and minimum enclosing circle

Contours – convex contours and the Douglas-Peucker algorithm

Line and circle detection

Line detection

Circle detection

Detecting shapes

Summary

4. Depth Estimation and Segmentation

Creating modules

Capturing frames from a depth camera

Creating a mask from a disparity map

Masking a copy operation

Depth estimation with a normal camera

Object segmentation using the Watershed and GrabCut algorithms

Example of foreground detection with GrabCut

Image segmentation with the Watershed algorithm

Summary

5. Detecting and Recognizing Faces

Conceptualizing Haar cascades

Getting Haar cascade data

Using OpenCV to perform face detection

Performing face detection on a still image

Performing face detection on a video

Performing face recognition

Generating the data for face recognition

Recognizing faces

Preparing the training data

Loading the data and recognizing faces

Performing an Eigenfaces recognition

Performing face recognition with Fisherfaces

Performing face recognition with LBPH

Discarding results with confidence score

Summary

6. Retrieving Images and Searching Using Image Descriptors

Feature detection algorithms

Defining features

Detecting features – corners

Feature extraction and description using DoG and SIFT

Anatomy of a keypoint

Feature extraction and detection using Fast Hessian and SURF

ORB feature detection and feature matching

FAST

BRIEF

Brute-Force matching

Feature matching with ORB

Using K-Nearest Neighbors matching

FLANN-based matching

FLANN matching with homography

A sample application – tattoo forensics

Saving image descriptors to file

Scanning for matches

Summary

7. Detecting and Recognizing Objects

Object detection and recognition techniques

HOG descriptors

The scale issue

The location issue

Image pyramid

Sliding windows

Non-maximum (or non-maxima) suppression

Support vector machines

People detection

Creating and training an object detector

Bag-of-words

BOW in computer vision

The k-means clustering

Detecting cars

What did we just do?

SVM and sliding windows

Example – car detection in a scene

Examining detector.py

Associating training data with classes

Dude, where's my car?

Summary

8. Tracking Objects

Detecting moving objects

Basic motion detection

Background subtractors – KNN, MOG2, and GMG

Meanshift and CAMShift

Color histograms

The calcHist function

The calcBackProject function

In summary

Back to the code

CAMShift

The Kalman filter

Predict and update

An example

A real-life example – tracking pedestrians

The application workflow

A brief digression – functional versus object-oriented programming

The Pedestrian class

The main program

Where do we go from here?

Summary

9. Neural Networks with OpenCV – an Introduction

Artificial neural networks

Neurons and perceptrons

The structure of an ANN

Network layers by example

The input layer

The output layer

The hidden layer

The learning algorithms

ANNs in OpenCV

ANN-imal classification

Training epochs

Handwritten digit recognition with ANNs

MNIST – the handwritten digit database

Customized training data

The initial parameters

The input layer

The hidden layer

The output layer

Training epochs

Other parameters

Mini-libraries

The main file

Possible improvements and potential applications

Improvements

Potential applications

Summary

To boldly go…

Index

Learning OpenCV 3 Computer Vision with Python Second Edition

Learning OpenCV 3 Computer Vision with Python Second Edition

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: September 2015

Production reference: 1240915

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78528-384-0

www.packtpub.com

Credits

Authors

Joe Minichino

Joseph Howse

Reviewers

Nandan Banerjee

Tian Cao

Brandon Castellano

Haojian Jin

Adrian Rosebrock

Commissioning Editor

Akram Hussain

Acquisition Editors

Vivek Anantharaman

Prachi Bisht

Content Development Editor

Ritika Singh

Technical Editors

Novina Kewalramani

Shivani Kiran Mistry

Copy Editor

Sonia Cheema

Project Coordinator

Milton Dsouza

Proofreader

Safis Editing

Indexer

Monica Ajmera Mehta

Graphics

Disha Haria

Production Coordinator

Arvindkumar Gupta

Cover Work

Arvindkumar Gupta

About the Authors

Joe Minichino is a computer vision engineer for Hoolux Medical by day and a developer of the NoSQL database LokiJS by night. On weekends, he is a heavy metal singer/songwriter. He is a passionate programmer who is immensely curious about programming languages and technologies and constantly experiments with them. At Hoolux, Joe leads the development of an Android computer vision-based advertising platform for the medical industry.

Born and raised in Varese, Lombardy, Italy, and coming from a humanistic background in philosophy (at Milan's Università Statale), Joe has spent his last 11 years living in Cork, Ireland, which is where he became a computer science graduate at the Cork Institute of Technology.

I am immensely grateful to my partner, Rowena, for always encouraging me, and also my two little daughters for inspiring me. A big thank you to the collaborators and editors of this book, especially Joe Howse, Adrian Roesbrock, Brandon Castellano, the OpenCV community, and the people at Packt Publishing.

Joseph Howse lives in Canada. During the winters, he grows his beard, while his four cats grow their thick coats of fur. He loves combing his cats every day and sometimes, his cats also pull his beard.

He has been writing for Packt Publishing since 2012. His books include OpenCV for Secret Agents, OpenCV Blueprints, Android Application Programming with OpenCV 3, OpenCV Computer Vision with Python, and Python Game Programming by Example.

When he is not writing books or grooming his cats, he provides consulting, training, and software development services through his company, Nummist Media (http://nummist.com).

About the Reviewers

Nandan Banerjee has a bachelor's degree in computer science and a master's in robotics engineering. He started working with Samsung Electronics right after graduation. He worked for a year at its R&D centre in Bangalore. He also worked in the WPI-CMU team on the Boston Dynamics' robot, Atlas, for the DARPA Robotics Challenge. He is currently working as a robotics software engineer in the technology organization at iRobot Corporation. He is an embedded systems and robotics enthusiast with an inclination toward computer vision and motion planning. He has experience in various languages, including C, C++, Python, Java, and Delphi. He also has a substantial experience in working with ROS, OpenRAVE, OpenCV, PCL, OpenGL, CUDA and the Android SDK.

I would like to thank the author and publisher for coming out with this wonderful book.

Tian Cao is pursuing his PhD in computer science at the University of North Carolina in Chapel Hill, USA, and working on projects related to image analysis, computer vision, and machine learning.

I dedicate this work to my parents and girlfriend.

Brandon Castellano is a student from Canada pursuing an MESc in electrical engineering at the University of Western Ontario, City of London, Canada. He received his BESc in the same subject in 2012. The focus of his research is in parallel processing and GPGPU/FPGA optimization for real-time implementations of image processing algorithms. Brandon also works for Eagle Vision Systems Inc., focusing on the use of real-time image processing for robotics applications.

While he has been using OpenCV and C++ for more than 5 years, he has also been advocating the use of Python frequently in his research, most notably, for its rapid speed of development, allowing low-level interfacing with complex systems. This is evident in his open source projects hosted on GitHub, for example, PySceneDetect, which is mostly written in Python. In addition to image/video processing, he has also worked on implementations of three-dimensional displays as well as the software tools to support the development of such displays.

In addition to posting technical articles and tutorials on his website (http://www.bcastell.com), he participates in a variety of both open and closed source projects and contributes to GitHub under the username Breakthrough (http://www.github.com/Breakthrough). He is an active member of the Super User and Stack Overflow communities (under the name Breakthrough), and can be contacted directly via his website.

I would like to thank all my friends and family for their patience during the past few years (especially my parents, Peter and Lori, and my brother, Mitchell). I could not have accomplished everything without their continued love and support. I can't ever thank everyone enough.

I would also like to extend a special thanks to all of the developers that contribute to open source software libraries, specifically OpenCV, which help bring the development of cutting-edge software technology closer to all the software developers around the world, free of cost. I would also like to thank those people who help write documentation, submit bug reports, and write tutorials/books (especially the author of this book!). Their contributions are vital to the success of any open source project, especially one that is as extensive and complex as OpenCV.

Haojian Jin is a software engineer/researcher at Yahoo! Labs, Sunnyvale, CA. He looks primarily at building new systems of what's possible on commodity mobile devices (or with minimum hardware changes). To create things that don't exist today, he spends large chunks of his time playing with signal processing, computer vision, machine learning, and natural language processing and using them in interesting ways. You can find more about him at http://shift-3.com/

Adrian Rosebrock is an author and blogger at http://www.pyimagesearch.com/. He holds a PhD in computer science from the University of Maryland, Baltimore County, USA, with a focus on computer vision and machine learning.

He has consulted for the National Cancer Institute to develop methods that automatically predict breast cancer risk factors using breast histology images. He has also authored a book, Practical Python and OpenCV (http://pyimg.co/x7ed5), on the utilization of Python and OpenCV to build real-world computer vision applications.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

Preface

OpenCV 3 is a state-of-the-art computer vision library that is used for a variety of image and video processing operations. Some of the more spectacular and futuristic features, such as face recognition or object tracking, are easily achievable with OpenCV 3. Learning the basic concepts behind computer vision algorithms, models, and OpenCV's API will enable the development of all sorts of real-world applications, including security and surveillance tools.

Starting with basic image processing operations, this book will take you through a journey that explores advanced computer vision concepts. Computer vision is a rapidly evolving science whose applications in the real world are exploding, so this book will appeal to computer vision novices as well as experts of the subject who want to learn about the brand new OpenCV 3.0.0.

What this book covers

Chapter 1, Setting Up OpenCV, explains how to set up OpenCV 3 with Python on different platforms. It will also troubleshoot common problems.

Chapter 2, Handling Files, Cameras, and GUIs, introduces OpenCV's I/O functionalities. It will also discuss the concept of a project and the beginnings of an object-oriented design for this project.

Chapter 3, Processing Images with OpenCV 3, presents some techniques required to alter images, such as detecting skin tone in an image, sharpening an image, marking contours of subjects, and detecting crosswalks using a line segment detector.

Chapter 4, Depth Estimation and Segmentation, shows you how to use data from a depth camera to identify foreground and background regions, such that we can limit an effect to only the foreground or background.

Chapter 5, Detecting and Recognizing Faces, introduces some of OpenCV's face detection functionalities, along with the data files that define particular types of trackable objects.

Chapter 6, Retrieving Images and Searching Using Image Descriptors, shows how to detect the features of an image with the help of OpenCV and make use of them to match and search for images.

Chapter 7, Detecting and Recognizing Objects, introduces the concept of detecting and recognizing objects, which is one of the most common challenges in computer vision.

Chapter 8, Tracking Objects, explores the vast topic of object tracking, which is the process of locating a moving object in a movie or video feed with the help of a camera.

Chapter 9, Neural Networks with OpenCV – an Introduction, introduces you to Artificial Neural Networks in OpenCV and illustrates their usage in a real-life application.

What you need for this book

You simply need a relatively recent computer, as the first chapter will guide you through the installation of all the necessary software. A webcam is highly recommended, but not necessary.

Who this book is for

This book is aimed at programmers with working knowledge of Python as well as people who want to explore the topic of computer vision using the OpenCV library. No previous experience of computer vision or OpenCV is required. Programming experience is recommended.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: We can include other contexts through the use of the include directive.

A block of code is set as follows:

import cv2

import numpy as np

img = cv2.imread('images/chess_board.png')

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

gray = np.float32(gray)

dst = cv2.cornerHarris(gray, 2, 23, 0.04)

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

img = cv2.imread('images/chess_board.png')

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

gray = np.float32(gray)

dst =

cv2.cornerHarris(gray, 2, 23, 0.04)

Any command-line input or output is written as follows:

mkdir build && cd build cmake ­D CMAKE_BUILD_TYPE=Release -DOPENCV_EXTRA_MODULES_PATH=/modules  ­D CMAKE_INSTALL_PREFIX=/usr/local .. make

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: On Windows Vista / Windows 7 / Windows 8, click on the Start menu.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <feedback@packtpub.com>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be