Deep Learning in MATLAB:

A Brief Overview

Brett Shoelson, PhD Principal Application Engineer

© 2018 The MathWorks, Inc.
1
What is can Deep Learning do for us?
(An example)

2
Example 1: Object recognition using deep learning

3
Object recognition using deep learning

Training Millions of images from 1000

(GPU) different categories

Real-time object recognition using

Prediction
a webcam connected to a laptop

4
What
Whatis isMachine Learning?
Deep Learning?

5
Machine Learning vs Deep Learning
We specify the nature
of the features we want
to extract… …and the type of model
we want to build.
Machine Learning

6
Machine Learning vs Deep Learning
We need only specify
the architecture of the
model…
Deep Learning

7
▪ Deep learning is a type of machine learning in which a model learns to
perform tasks like classification – directly from images, texts, or signals.
▪ Deep learning performs end-to-end learning, and is usually
implemented using a neural network architecture.
▪ Deep learning algorithms also scale with data – traditional machine
learning saturates.

8
Why is Deep Learning So Popular Now?

AlexNet

Human
Accuracy

Source: ILSVRC Top-5 Error on ImageNet 9

Two Approaches for Deep Learning
1. Train a Deep Neural Network from Scratch

2. Fine-tune a pre-trained model (transfer learning)

10
Pains In Deep Learning

Expertise Time to Train Data

11
Example: Vehicle recognition using deep transfer learning

Cars
Trucks
5 Category
SUVs
Classifier
Big Trucks
Vans

12
 Import the Latest Models for Transfer Learning
Pretrained Models*
▪ AlexNet Import Models from Frameworks
▪ VGG-16 ▪ Caffe Model Importer

▪ VGG-19 ▪ TensorFlow-Keras Model Importer

▪ GoogLeNet ▪ Onnx - Importer/ Exporter (Coming Soon)

▪ Inception-v3
▪ ResNet50 AlexNet VGG-16 ResNet-50 ResNet-101
PRETRAINED PRETRAINED PRETRAINED MODEL PRETRAINED MODEL
MODEL MODEL
▪ ResNet-101
Caffe TensorFlow-
▪ Inception-resnet-v2 GoogLeNet Inception-v3
IMPORTER PRETRAINED Keras MODELS
MODEL IMPORTER
▪ SqueezeNet
▪ MobileNet(coming soon)

* single line of code to access model 13

Detection and localization using deep learning

Regions with Convolutional Neural Network Features (R-CNN) 14

What is semantic segmentation?

15
Localization using deep learning

Original Image
ROI detection

Pixel classification

16
Semantic Segmentation Network

Boat
Airplane

Other classes

17
Semantic Segmentation Network

18
Semantic Segmentation Demo

CamVid Dataset
1. Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008
2. Semantic Object Classes in Video: A High-Definition Ground Truth Database ,Pattern Recognition Letters
19
Semantic Segmentation

CamVid Dataset
1. Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008
2. Semantic Object Classes in Video: A High-Definition Ground Truth Database ,Pattern Recognition Letters
20
 “I love to label and
preprocess my data”
~ Said no engineer, ever.

21
Ground truth Labeling

“How do I label
my data?”

New App for

Ground Truth
Labeling
Label pixels
and regions for
semantic
segmentation

Data

22
Attributes and Sublabels NEW in

23
 Types of Datasets

Numeric Time Series/ Image

Data Text Data Data

ML or LSTM CNN
LSTM or CNN

24
Analyzing signal data using deep learning

Signal Classification using LSTMs Speech Recognition using CNNs

25
Deep learning features overview

▪ Classification ▪ Data augmentation

▪ Regression ▪ Hyperparameter tuning
▪ Semantic segmentation – Bayesian optimization
▪ Object detection ▪ Python MATLAB interface
▪ Scalability ▪ LSTM networks
– Multiple GPUs – Time series, signals, audio
– Cluster or cloud ▪ Custom labeling
▪ Custom network layers – API for ground-truth labeling
automation
▪ Import models
– Superpixels
– Caffe
– Keras/TensorFlow
▪ Data validation
– Training and testing
26
 Prediction Performance: Fast with GPU Coder
Images/Sec

Why is GPU Coder so fast?

– Analyzes and optimizes
network architecture
– Invested 15 years in code
generation

TensorFlow
MATLAB
MXNet
GPU Coder

Using CUDA v9
AlexNet ResNet-50 VGG-16 and cuDNN v7

27
 Overview of deep learning deployment options

“How do I deploy
my model?”
▪ Create Desktop Apps GPU Coder

▪ Run Enterprise Solution

Introducing:
GPU Coder-
Convert to ▪ Generate C and C++ Code
NVIDIA CUDA
code

Deploy / Share ▪ Target GPUs

▪ Generate C and C++ Code
28
GPU Coder Fills a Gap in Our Deep Learning Solution

Training Inference

Access Data Preprocess Select Network Train Deploy

Image Neural GPU

Image Acq. PCT
Processing Network Coder
Computer
Vision

29
Deploying to CPUs

Intel
MKL-DNN
Library

NVIDIA
GPU TensorRT &
Coder cuDNN
Libraries
Deep Learning
Networks
ARM
Compute
Library

30
MATLAB products for deep learning

Required products Recommended products

▪ Neural Network Toolbox ▪ Statistics and Machine Learning
▪ Parallel Computing Toolbox Toolbox
▪ Image Processing Toolbox ▪ MATLAB Coder
▪ Computer Vision System Toolbox ▪ GPU Coder
▪ Automated Driving System
Toolbox

31
Deep learning features overview

▪ Classification ▪ Data augmentation *

▪ Regression * ▪ Hyperparameter tuning *
▪ Semantic segmentation – Bayesian optimization
▪ Object detection * ▪ Python MATLAB interface *
▪ Scalability * ▪ LSTM networks *
– Multiple GPUs – Time series, signals, audio
– Cluster or cloud ▪ Custom labeling *
▪ Custom network layers * – API for ground-truth labeling
automation
▪ Import models *
– Superpixels
– Caffe
– Keras/TensorFlow
▪ Data validation *
– Training and testing
* We can cover in more detail outside this presentation 32
Thank you!

33
Deep learning in
automated driving…

34
Deep Learning Onramp

▪ Get started using deep learning

methods to perform image recognition.
▪ Free access for everyone
▪ Interactive exercises and short video
demonstrations
▪ Work on real-life image recognition
problems
▪ Topics include:
– Convolutional neural networks
– Working with pre-trained networks
– Transfer learning
– Evaluating network performance
35
Convolutional Neural Networks (CNN)
Edges
Shapes
Objects

36