Deep Learning with MATLAB

Cagliari (Italy), 19 Sep. 2019

Giuseppe Ridinò
Senior Application Engineer
MathWorks Italy

© 2019 The MathWorks, Inc.

1
MATLAB & Simulink in Research

2
TNG Data Project and Jupyterlab Interface

Project: http://www.tng-project.org/
Jupyterlab: tng-project.org/data/lab/
Scripts: http://www.tng-
project.org/data/docs/scripts/

3
MATLAB Community Tools for Astronomy
https://www.mathworks.com/matlabcentral/fileexchange?sort=downloads_desc&q=astronomy

4
LIGO: Spot 1st-Ever Gravitational Waves with MATLAB

Article
Blog post
5
NASA Build Kepler Pipeline tools with MATLAB
download: github.com/nasa/kepler-pipeline

6
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

7
Artificial Intelligence

▪ Development of computer systems to perform tasks that

normally require human intelligence
Machine Learning

Decision Deep Logistic

Trees Learning Regression

K-means SVM

Nearest Reinforcement Learning

Neighbor Gaussian
Mixture

8
 Machine Learning and Deep Learning

Machine Learning

Unsupervised
Learning
[No Labeled Data]

Clustering

9
 Machine Learning and Deep Learning

Machine Learning

Unsupervised Supervised Learning

Learning [Labeled Data]
[No Labeled Data]

Clustering Classification Regression

10
 Machine Learning and Deep Learning

Machine Learning

Unsupervised Supervised Learning

Learning [Labeled Data]
[No Labeled Data] Supervised learning typically involves
feature extraction
Clustering Classification Regression

Deep Learning
Deep learning typically does not
involve feature extraction

11
What is Deep Learning?

12
Deep Learning

▪ Subset of machine learning with automatic feature extraction

– Learns features and tasks directly from data
– More Data = better model

Deep Learning
Machine
Learning

Deep
Learning

13
Deep Learning uses a neural network architecture

Input Output
Hidden Layers (n)
Layer Layer
14
Deep Learning datatypes

Image Signal

Text
Numeric

15
Example 1: Detection and localization using deep learning

YOLO v2 (You Only Look Once) Semantic Segmentation using SegNet

16
Example 2: Analyzing signal data using deep learning

Signal Classification using LSTMs Speech Recognition using CNNs

17
Machine Learning vs Deep Learning
Deep learning performs end-to-end learning by learning features, representations and tasks directly
from images, text and sound
Deep learning algorithms also scale with data – traditional machine learning saturates
Machine Learning

Deep Learning

18
Deep Learning workflow

PREPARE DATA TRAIN MODEL DEPLOY SYSTEM

Data access and Model design, Multiplatform code

preprocessing Hyperparameter generation (CPU,
tuning GPU)

Ground truth labeling Model exchange Embedded

across frameworks deployment

Hardware-
Simulation-based Enterprise
accelerated
data generation Deployment
training

I Iteration and Refinement

19
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

20
Deep Learning in 6 lines of MATLAB code

21
Using Apps for designing Deep Learning networks

22
Different Neural Networks for Different Data

23
 Types of Data

Numeric Time Series/ Image

Data Text Data Data

Machine Learning or CNN

LSTM LSTM or CNN

24
Convolutional Neural Networks (CNN)

Edges
Shapes
Objects

25
CNN Visualization methods

Layer Activations Class Activations DeepDream Images

26
Another network for signals - LSTM

▪ LSTM = Long Short Term Memory (Networks)

– Signal, text, time-series data
– Use previous data to predict new information
▪ I live in France. I speak ___________.

c0 C1 Ct

27
Combining convolution and recurrent layers
Learn spatial and temporal features simultaneously

Response
CNN LSTM Class 1
class label

CNN LSTM CNN learns features

from each frame
individually
CNN LSTM
LSTM learns
relationships
Video Frames
CNN LSTM between frames

28
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

29
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

30
Two approaches for Deep Learning
1. Train a Deep Neural Network from Scratch

2. Fine-tune a pre-trained model (transfer learning)

31
Semantic Segmentation

CamVid Dataset
1. Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008
2. Semantic Object Classes in Video: A High-Definition Ground Truth Database ,Pattern Recognition Letters
32
Semantic Segmentation Network

Boat
Airplane

Other classes

33
Semantic Segmentation Network

34
Example: Semantic Segmentation for Free Road
Detection

35
1. Use CamVid dataset

▪ Download the dataset (images and labels)

▪ Resize images to fit the input sizes of our neural network

36
2. Balance classes using class weighting

Ideally, all classes would have an

equal number of observations.

However, the classes in CamVid are

imbalanced.

We use the pixel label counts and

calculate the median frequency
class weights.

37
3. Split dataset

Training Validation Test

60% 20% 20%

• Trains the model • Checks accuracy • Tests model accuracy

• Computer “learns” of model during • Not used until validation
from this data training accuracy is good

38
 4. Define network
91x1 Layer array with layers:

1 'inputImage' Image Input 360x480x3 images with 'zerocenter' normalization

2 'conv1_1' Convolution 64 3x3x3 convolutions with stride [1 1] and padding [1 1 1 1]
3 'bn_conv1_1' Batch Normalization Batch normalization
4 'relu1_1' ReLU ReLU
5 'conv1_2' Convolution 64 3x3x64 convolutions with stride [1 1] and padding [1 1 1 1]
6 'bn_conv1_2' Batch Normalization Batch normalization
7 'relu1_2' ReLU ReLU
8 'pool1' Max Pooling 2x2 max pooling with stride [2 2] and padding [0 0 0 0]
9 'conv2_1' Convolution 128 3x3x64 convolutions with stride [1 1] and padding [1 1 1 1]
10 'bn_conv2_1' Batch Normalization Batch normalization
11 'relu2_1' ReLU ReLU
...

• Use segnetLayers to create a SegNet network

initialized from a pre-trained VGG-16.

• Another option: Deeplab v3+ network with weights

initialized from a pre-trained Resnet-18.

Last 9 layers: the last one provides

also the Class Weighting
39
5. Train network

40
 Useful tools for Semantic Segmentation
▪ Automatically create network structures
– Using segnetLayers and fcnLayers

▪ Handle pixel labels

– Using the pixelLabelImageDatastore and
pixelLabelDatastore

▪ Evaluate network performance

– Using evaluateSemanticSegmentation

▪ Examples and tutorials to learn concepts

41
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

42
Common network architectures - signal processing

Convolutional Neural Networks (CNN)

Time-Frequency Transformation
Long Short Term Memory (LSTM) Networks

Feature Engineering

43
Example: Speech Recognition

44
1. Create Datastore

▪ Datastore creates
reference for data

▪ Do not have to load in

all objects into memory

45
2. Compute speech spectrograms

Amplitude
Frequency

Time
46
3. Split datastores

Training Validation Test

70% 15% 15%

• Trains the model • Checks accuracy • Tests model accuracy

• Computer “learns” of model during • Not used until validation
from this data training accuracy is good

47
4. Define architecture and parameters

Model Parameters
Neural Network Architecture

48
5. Train network

49
Training is an iterative process

Parameters adjusted according to performance

50
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

51
Labeling for deep learning is repetitive,
tedious, and time-consuming…

but necessary

52
Ground truth labeling for images and video

“How do I label
my data?”

App for
Ground Truth
Labeling
Label regions
and pixels for
semantic
segmentation

53
Ground truth labeling for signals and audio

“How do I label
my data?”

App for Signal

Analysis and
Labeling
Label signal
waveforms,
attributes, and
samples for
machine
learning

54
MATLAB Apps help you speed up labeling work

For labeling images and videos:

1. Image Labeler (Computer Vision Toolbox)
▪ https://www.mathworks.com/help/vision/ref/imagelabeler-app.html

2. Video Labeler (Computer Vision Toolbox)

▪ https://www.mathworks.com/help/vision/ref/videolabeler-app.html

3. Ground Truth Labeler (Automated Driving Toolbox)

▪ https://www.mathworks.com/help/driving/ref/groundtruthlabeler-app.html

How to choose the correct app for your project?

– https://www.mathworks.com/help/vision/ug/choose-a-labeling-app.html

55
MATLAB Apps help you speed up labeling work

For labeling signals and audio:

1. Audio Labeler
(Audio Toolbox)
https://www.mathworks.com/help/audio/ug/au
dio-labeler-walkthrough.html

2. Signal Labeler
(Signal Processing Toolbox)
https://www.mathworks.com/help/signal/ref/si
gnallabeler.html

56
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

57
Two ways to work with TensorFlow and PyTorch

MATLAB Python

Co-execution (Python and C++)

Model Exchange

58
Three use cases for model exchange

1. Enable teams to use different frameworks and tools

2. Access to code generation and other downstream tools in MATLAB

3. MATLAB users gain access to models in research/DL ecosystem

59
Model exchange with MATLAB

PyTorch Keras-
Tensorflow
Caffe2

MXNet ONNX MATLAB

Core ML
Caffe
CNTK (…)

Open Neural Network Exchange

60
ONNX – Industry Standard for Model Exchange

Source: https://onnx.ai/
61
Transfer Learning with Pretrained Models

Inception-v3 ResNet-101 VGG-16

Inception-
ResNet-v2
ResNet-18 GoogLeNet DenseNet-201
VGG-19
SqueezeNet AlexNet ResNet-50

Import & Export Models Between Frameworks

Keras-Tensorflow Caffe Model ONNX Model
Importer Importer Converter

More comprehensive list here: https://www.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-networks.html

62
Agenda

Deep Learning Overview

Deep Learning in 6 Lines of MATLAB Code

Deep Learning with…

Images Signals

Labeling Ground Truth Data

Working with Other Frameworks

Performance, Deployment, and Next Steps

63
Training Performance and Scalability

64
Deep Learning on CPU, GPU, Multi-GPU and Clusters
HOW TO TA R G E T ?

Single Single CPU

CPU Single GPU

Single CPU, Multiple GPUs

On-prem server with GPUs Cloud GPUs

65
 MATLAB parallel capabilities for cloud are evolving

Option Parallel Computing Toolbox MATLAB Parallel Server (formally called MDCS)
Desktop Custom Cloud User-managed Cloud Pre-configured Cloud Custom Cloud On Premise
Reference Cloud Center Hosting Providers Reference
Architecture Architecture
Description Explicit desktop Explicit desktop Preconfigured clusters in Cloud solutions from Custom infrastructure Scale to clusters in your
scaling scaling Amazon Web Services MathWorks partners for AWS, Azure, and organization
others
When to Prototyping, small Data in cloud, access User-centric workflows, Managed environments with Internally-managed Traditional scaling on
use scaling high-end machines, small studies operating expense rather environments with assets you manage at
especially with GPUs than capital expense operating expense your facilities
rather than capital
expense

Maximum Physical cores in Physical cores in 1024 per cluster Defined by Hosting Provider No limit Physical cores in cluster
workers machine machine

License • Network • Online • Online only • Network • Online • Network

Options • Online • Network (requires • Online • Network (requires • Online
support) support)

http://www.mathworks.com/programs/mdcs-cloud.html 66
Cloud Reference Architecture: MATLAB & Simulink

Virtual Network

Use cases:
Remote • Data analytics on cloud-stored data
Desktop • Access to high-end hardware:
Client − multi-core
− GPUs
− FPGAs
• Prototyping parallel algorithms
• On-server pre-production testing

67
 Reference Architecture: MATLAB Distributed Computing Server

Virtual Network

Head node VM
with MATLAB job scheduler
Use cases:
• Parameter sweeps
• Monte Carlo runs
• Optimization
Client with MATLAB and • Distributed array calculations
Parallel Computing Toolbox

Compute node
VMs
MathWorks Hosted
License Manager

68
NVIDIA NGC & DGX Supports MATLAB for Deep Learning

▪ GPU-accelerated MATLAB Docker container for deep learning

– Leverage multiple GPUs on NVIDIA DGX Systems and in the Cloud
▪ Cloud providers include: AWS, Azure, Google, Oracle, and Alibaba

▪ NVIDIA DGX System / Station

– Interconnects 4/8/16 Volta GPUs in one box

▪ Containers available for R2018a through R2019a

– New Docker container with every major release (a/b)

▪ Download MATLAB container from NGC Registry

– https://ngc.nvidia.com/registry/partners-matlab

69
Inference Performance and Deployment

70
Deployment process

PREPARE DATA TRAIN MODEL DEPLOY SYSTEM

Data access and Model design, Multiplatform code

preprocessing Hyperparameter generation (CPU,
tuning GPU)

Ground truth labeling Model exchange Edge deployment

across frameworks

Hardware-
Simulation-based Enterprise
accelerated
data generation Deployment
training

I Iteration and Refinement

71
Deploying Deep Learning models for inference

Intel
MKL-DNN
Library

NVIDIA
Coder GPU Coder TensorRT &
Products cuDNN
Libraries
Deep Learning
Networks
ARM
Compute
Library

72
Deploying to HW targets

Desktop CPU

NVIDIA
Coder GPU Coder TensorRT &
Products cuDNN
Libraries
Deep Learning
Networks

Raspberry Pi board
73
 With GPU Coder, MATLAB is fast

GPU Coder is faster

than TensorFlow,
MXNet and PyTorch

TensorFlow
MXNet
GPU Coder
PyTorch

Intel® Xeon® CPU 3.6 GHz - NVIDIA libraries: CUDA10 - cuDNN 7 - Frameworks: TensorFlow 1.13.0, MXNet 1.4.0 PyTorch 1.0.0 74
Semantic Segmentation speedup

Running in MATLAB Generated Code from GPU Coder

75
 Blog: Deep Learning

http://blogs.mathworks.com/deep-learning/
76
 Introduction to Deep Learning

http://www.mathworks.com/videos/series/introduction-to-deep-learning.html
77
 Deep Learning with MATLAB

http://www.mathworks.com/videos/series/deep-learning-with-MATLAB.html
78
 Deep Learning Onramp

http://www.mathworks.com/learn/tutorials/deep-learning-onramp.html
79
Thank you!

80