Image Processing & Computer Vision with MATLAB

March, 2013

Roy Fahn
Application Engineer royf@systematics.co.il www.linkedin.com/in/royfahn Blog: http://matlabisrael.blogspot.com/

Agenda
Retinal Blood Vessel Extraction Using Kirsch's Templates Circular Objects Detection & Visualization ViolaJones Object Detection Kanade-Lucas-Tomasi Point Tracking Break Image Registration using SURF Speeding-Up Image Processing Applications using the GPU Video Focusing in Real-Time

The Human Eye

Retinal Scan Process

Retinal Scan Output

Applications- Retinal Diseases

Arterial Macroaneurysms

Arterial Macroaneurysms Risk Factors

Often occur in patients with high blood pressure and other forms of vascular disease, such as heart attacks and strokes. Solution: Keep your blood pressure low ! Occur late in life Solution: Stay young ! Tend to occur more commonly in women than men (3:1 ratio) Solution: Undo !

Applications - Retinal Identification

Pros: - Accurate - Fast

Cons: - Cost - Astigmatism - Ergonomics

Sobel Edge Detection

G > th Edge !

Sobel Edge Detection - Drawback

Chapman et al., 2001: Three automated techniques of edge detection to identify the boundaries and corresponding widths of retinal blood vessels Sobel was found to be the most inconsistent. Possibly relates to the program identifying the central light reflex from the blood vessel as an edge.
Chapman,N.,Witt, N.,Gao,X.,Bharath, A.A., Stanton, A.V., Thom,S.A.,Hughes,A.D.,2001. Computer algorithms for the automated measurement of retinal arteriolar diameters. Br. J. Ophthalmol. 85,7479.

Kirsch Operator
A non-linear operator that finds the maximum edge strength in a few predetermined directions. The operator is calculated as follows for directions with 45 difference: where the direction kernels and so on
Kirsch,R.A.,1971. Computer Determination of the Constituent Structure of Biological Images. Computers and Biomedical Research 4,315-328.

Kirsch Implementation
LISP MATLAB help

Kirsch Implementation
MATLAB Central

Sobel Vs. Kirsch

Speeding Up MATLAB
MATLAB Coder Parallel Computing Toolbox

MATLAB Coder

Parallel Computing Toolbox

Solve computationally and data-intensive problems using: multicore processors GPUs computer clusters (requires MDCS)

GUI Creation & MATLAB Compiler

MATLAB Compiler Vs. MATLAB Coder

Coder Supported Functions C/C++ Source Code Generation Numeric Algorithms Speed Up GUI & Graphics Deployment MCR Needed
V V V V V

Compiler
VVV

Wavelet Compression

Eikelboom,R., Yogesan,K., Barry,C., Constable, I., Tay-Kearney, M., Jitskaia,L., House,P., 2000. Methods and limits of digital image compression of retinal images for telemedicine. Invest. Ophthalmol. Vis. Sci. 41,191619 24.

Tools Used
MATLAB Image Processing Toolbox MATLAB Coder Embedded Coder Parallel Computing Toolbox MATLAB Compiler Wavelet Toolbox

Retinal Scan - The Old Fashioned Way

Agenda
Retinal Blood Vessel Extraction Using Kirsch's Templates Circular Objects Detection & Visualization ViolaJones Object Detection Kanade-Lucas-Tomasi Point Tracking Break Image Registration using SURF Speeding-Up Image Processing Applications using the GPU Video Focusing in Real-Time

Circular Objects Detection & Visualization

Image Processing Toolbox (R2012a(

,)viscircles ,imfindcircles( intensity-metric optimization (imfuse, imshowpair ,imregconfig ,imregister(

Agenda
Retinal Blood Vessel Extraction Using Kirsch's Templates Circular Objects Detection & Visualization ViolaJones Object Detection Kanade-Lucas-Tomasi Point Tracking Break Image Registration using SURF Speeding-Up Image Processing Applications using the GPU Video Focusing in Real-Time

System Objects
R2010a- http://www.systematics.co.il/hebrew/News/PDF/2010/MATLAB-VIDEO.PDF System Objects R2011b-

C System Objects R2012a-

( System Toolboxes )R2011a-

MATLAB Simulink- : Fixed-Point IP- MATLABSimulink-

System Toolboxes

Simulink System Toolboxes-

Computer Vision System Toolbox

:

System Objects http://www.systematics.co.il/hebrew/News/PDF/2010/MATLAB-VIDEO.PDF

MATLAB-Simulink- ( )C Simulink

Computer Vision System Toolbox (R2012a)

: DSP System Toolbox - Signal Processing Toolbox - Integral Image CAMShift MSER ' Viola-Jones

trainCascadeObjectDetector (R2013a)

Requires positive and negative samples Trains to reject negative samples as quickly as possible Produces an XML file compatible with OpenCV Training can take from 5 minutes to several days

Agenda
Retinal Blood Vessel Extraction Using Kirsch's Templates Circular Objects Detection & Visualization ViolaJones Object Detection Kanade-Lucas-Tomasi Point Tracking Break Image Registration using SURF Speeding-Up Image Processing Applications using the GPU Video Focusing in Real-Time

Kanade-Lucas-Tomasi (KLT)
Algorithm for tracking points across frames (sparse optical flow) First introduced in 1981 (!) by Lucas and Kanade Workflow:
Specify initial points to track (e. g. detect corners) Use KLT Re-detect points if necessary

Works well for tracking corners

detectMinEigenFeatures detectHarrisFeatures detectSURFFeatures

Agenda
Retinal Blood Vessel Extraction Using Kirsch's Templates Circular Objects Detection & Visualization ViolaJones Object Detection Kanade-Lucas-Tomasi Point Tracking Break Image Registration using SURF Speeding-Up Image Processing Applications using the GPU Video Focusing in Real-Time

Computer Vision System Toolbox

:

System Objects http://www.systematics.co.il/hebrew/News/PDF/2010/MATLAB-VIDEO.PDF

MATLAB-Simulink- ( )C Simulink

MATLAB Functions in CVST (1)

Stereo Vision
Disparity* epipolarLine estimateFundamentalMatrix estimateUncalibratedRectification isEpipoleInImage lineToBorderPoints Disparity map between stereo images Compute epipolar lines for stereo images Estimate fundamental matrix from corresponding points in stereo images Uncalibrated stereo rectification Determine whether image contains epipole Intersection points of lines in image and image border

Filtering
integralFilter* integralImage Isfilterseparable* Integral image filter Compute integral image Determine whether filter coefficients are separable

MATLAB Functions in CVST (2)

Detection, Extraction, Matching & Tracking

assignDetectionsToTrack
configureKalmanFilter detectMSERFeatures detectSURFFeatures extractFeatures insertObjectAnnotation matchFeatures showMatchedFeatures

Assign detections to tracks for multi-object tracking Create Kalman filter for object tracking Detect MSER features Detect SURF features Extract interest point descriptors Insert annotation in image or video stream Find matching features Display corresponding feature points

RANSAC
Random Sample Consensus

RANSAC
)Stereo vision( Fundamental Matrix

Fast Retina Keypoint Extractor (FREAK)

Supported within extractFeatures New descriptor that outperforms SIFT, SURF Typically paired with corner points How it works
Retinal sampling pattern motivated from Human Visual System Binary descriptor
Compare pair of pixel intensities Extract feature in 512 bits

Use Hamming Distance in matchFeatures to match features across images

Not yet supported with MATLAB Coder Reference: http://www.ivpe.com/freak.htm

a b c
G(a,b,c,d) = 0101

Agenda
Retinal Blood Vessel Extraction Using Kirsch's Templates Circular Objects Detection & Visualization ViolaJones Object Detection Kanade-Lucas-Tomasi Point Tracking Break Image Registration using SURF Speeding-Up Image Processing Applications using the GPU Video Focusing in Real-Time

What is a GPU?
Graphical Processing Unit
NVIDIA Quadro NVS 4200M # Cores 48, @810 MHz

Standard Memory Config

Bus Support

1024 MB DDR 3
PCI-E 2.0 x 16

History of GPUs
3D Gaming & CAD Scientific Computing

CPUs vs. GPUs

Core 1 Core 2

Core 3

Core 4

Cache

Device Memory

CPU (Multiple Cores)

GPU (Hundreds of Cores)

System Memory

Problems for Running on the GPU

A selection of problems from the CUDA Community Showcase:
Computational Fluid Dynamics Computational Finance

Weather Modeling

N-Body Simulations

Molecular Modeling

Digital Signal Processing

http://www.nvidia.com/object/cuda_showcase_html.html

The Language of GPUs

Kernel
Device Host CUDA

Code running on the GPU

The card containing the GPU and memory The CPU and system memory Compute Unified Device Architecture Language designed for general purpose use of GPU

Functionality Overview

Parallel Computing Toolbox Support for GPU Computing

Three Levels of Support:

1) 2) 3) Invoking built-in functions on arrays existing on the GPU Translating simple MATLAB functions to run on the GPU Invoking CUDA code from MATLAB

nVIDIA Solutions

Invoking Built-In Functions

GPU Device
Memory

gpuArray(A)

gather(A)

Processing Elements

Built-In Functions that Support gpuArray

Summary: Invoking Built-In Functions

10
1

Matrix Multiplication CPU GPU

10

Backslash CPU GPU

10

10

Time (s)

10

-2

Time (s)
32 64 128 256 512 1024 Matrix Size (n-by-n) 2048 4096

10

-1

10

-1

10

-2

10

-3

10

-3

10

-4

10

-4

32

64

128

256 512 1024 Matrix Size (n-by-n)

2048

4096

10

Array Mutliplication CPU GPU

10

Addition CPU GPU

10

-1

10

-1

Time (s)

10

-3

Time (s)
32 64 128 256 512 1024 Matrix Size (n-by-n) 2048 4096

10

-2

10

-2

10

-3

10

-4

10

-4

10

-5

10

-5

32

64

128

256 512 1024 Matrix Size (n-by-n)

2048

4096

Translating MATLAB Functions

GPU Device Memory arrayfun() bsxfun() Processing Elements

Supported MATLAB Code

Summary: Translating MATLAB Functions

CPU vs. GPU

Accelerate scalar operations on large arrays Combine computations into a single kernel

0.02

0.015

0.01

0.005

CPU

GPU

Invoking a CUDA Kernel using feval

Acircle r
2
2 2

Asquare (2r ) 4r
Acircle Asquare 4

P( x y r )
2 2 2

Generating Parallel Thread Execution Files

Summary: Invoking a CUDA Kernel

Speedup from using a GPU Solution

Pros Call existing CUDA code from MATLAB Highest level of functionality Cons Requires knowledge of CUDA Requires knowledge of Parallel Programming
Speedup

30

25

20

15

10

5 1000

1500

2000

2500 3000 3500 Number of Runs

4000

4500

5000

Summary of Features
1) Invoking built-in functions on arrays existing on the GPU

Greater Functionality

Ease of Use

2)

Translating simple MATLAB functions to run on the GPU

3)

Invoking CUDA code from MATLAB

2011-2012 Upgrades
More GPU-enabled MATLAB functions & features, including capabilities from: - Communications System Toolbox (11) - Signal Processing Toolbox (5) - Phased Array System Toolbox - Neural Network Toolbox MATLAB Compiler generated standalone executables and components now support applications that use the GPU Improved arrayfun function for GPU computing applications in MATLAB Random number generation on GPUs

Image Processing & GPU

GPU-enabled Image Processing Toolbox functions in R2013a: imrotate, imfilter, imdilate, imerode, imopen, imclose, imtophat, imbothat, imshow, padarray and bwlookup User Story: Dr. Alexander Lomes - CTO, ORCASONIX Ltd.

Agenda
Retinal Blood Vessel Extraction Using Kirsch's Templates Circular Objects Detection & Visualization ViolaJones Object Detection Kanade-Lucas-Tomasi Point Tracking Break Image Registration using SURF Speeding-Up Image Processing Applications using the GPU Video Focusing in Real-Time

Computer Vision System Toolbox

:

System Objects http://www.systematics.co.il/hebrew/News/PDF/2010/MATLAB-VIDEO.PDF

MATLAB-Simulink- ( )C Simulink

 ? Simulink
( )

 Simulink
) Application Program Interface (API MATLAB C/HDL

Image Acquisition Toolbox

MATLABSimulink- Frame grabbers- ()GUI ( )Preview

 )R2010b, R2011b(
Interface Analog Camera Link DCAM FireWire Windows Linux Macintosh

GigE Vision
OS Integrated

As of R2011b As of R2010b Pre-R2010b

)R2012a( Image Acquisition Toolbox

imaq.VideoDevice System Object .MATLAB Coder- C .)R2013a- R2012b- (

Video Focusing in Real-Time

More Information
Web: http://www.mathworks.com

Blog: http://matlabisrael.blogspot.com
LinkedIn Group: MATLAB & Simulink users in Israel Seminars: http://www.systematics.co.il/mathworks/Events/events.html Courses: http://www.systematics.co.il/mathworks/Training/Courses.html Webinars: http://www.mathworks.com/events Support & Sales: 03-7660111

Thank you !