FDP: IoT and its application on

Instrumentation
DAY 5: Introduction of ML for IOT
 Agenda
• Role of Machine learning in Industrial Domain
• Machine Learning
– Checkout some live observations
– Why and what is ML
• Types of ML
• Steps in Data Modelling and Machine Learning Process
• Type of ML Algorithm
• Python libraries used in Machine Learning
• Data Sets Links
• Digital Transformation with ML
• ML Deployment with Embedded Hardware
• TinyML
• TensorFlow Lite
• Edge Impulse
• Q&A
Machine Learning
Live Observations
Usecase 1: Junk Mail Folder
Usecase 2: Google Translate
Usecase 3: Recommendations
Usecase 4: Object Recognition
Why and what is ML
Aptitude/Logical reasoning
Series completion by finding the missing terms 1, 6, 15, __, 45, 66, 91
Machine Learning
Machine
Learning
 Machine Learning

• “Machine Learning: Field of study that gives

computers the ability to learn without being
explicitly programmed.” -Arthur Samuel
(1959)
Applied ML In NutShell
How Deep Learning is different?
 Types of Learning

• Supervised (inductive) learning

– Given: training data + desired outputs (labels)
• Unsupervised learning
– Given: training data (without desired outputs)
• Reinforcement learning
– Rewards from sequence of actions
Supervised Learning: Regression
Supervised Learning: Classification
Unsupervised Learning
Unsupervised Learning
 Reinforcement Learning

https://www.youtube.com/watch?v=gn4nRCC9TwQ
Framing a Learning Problem
Designing a Learning System
 Set of steps in Machine Learning
• Define: The data that you want to study in order to solve a
problem or meet an objective.
• Collect data
• Filter data
• Analyze data
• Train algorithms
• Test algorithms
• Use algorithms for future predictions
Representing Examples
Learning System
Learning System
DeepBelief Net on Face Images
Training on Multiple Objects
Machine Learning
Usecases
Python libraries
 Python libraries
• NumPy
– NumPy is a well known general-purpose array-processing package.
– very useful for handling linear algebra, Fourier transforms, and random numbers.
– Other libraries like TensorFlow uses NumPy at the backend for manipulating tensors.

• SciPy
– for scientific and analytical computing.

• Scikit-learn
– Scikit-learn was built on top of two Python libraries – NumPy and SciPy and has become the most popular
Python machine learning library for developing machine learning algorithms.

• Theano
– act as an optimizing compiler for evaluating and manipulating mathematical expressions and matrix
calculations.
– Theano can work on Graphics Processing Unit (GPU) and CPU.
– Theano can perform data-intensive computations up to 140x faster on GPU than on a CPU.
 Python libraries
• TensorFlow
– Was developed for Google’s internal use by the Google Brain team.
– Is a popular computational framework for creating machine learning models.
– Supports a variety of different toolkits for constructing models at varying levels of abstraction
– Has a flexible architecture with which it can run on a variety of computational platforms CPUs, GPUs, and
TPUs. TPU stands for Tensor processing unit, a hardware chip built around TensorFlow for machine
learning and artificial intelligence.

• Keras
– used for neural networks and machine learning.
– can run on top of TensorFlow, Theano. Keras also can run efficiently on CPU and GPU.
– works with neural-network building blocks like layers, objectives, activation functions, and optimizers.
Keras also have a bunch of features to work on images and text images that comes handy when writing
Deep Neural Network code.
– Apart from the standard neural network, Keras supports convolutional and recurrent neural networks.

• PyTorch
– has a range of tools and libraries that support dynamic computation graphs, computer vision, machine
learning, and natural language processing.
 Python libraries
• Pandas
– used for data analysis with support for fast, flexible, and expressive data structures
designed to work on both “relational” or “labeled” data.
– Pandas support and perform well with different kinds of data including the below :
• Tabular data with columns of heterogeneous data. For instance, consider the data coming from the SQL
table or Excel spreadsheet.
• Ordered and unordered time series data. The frequency of time series need not be fixed, unlike other
libraries and tools. Pandas is exceptionally robust in handling uneven time-series data
• Arbitrary matrix data with the homogeneous or heterogeneous type of data in the rows and columns
• Any other form of statistical or observational data sets. The data need not be labeled at all. Pandas data
structure can process it even without labeling.

• Matplotlib
– Matplotlib is a data visualization library that is used for 2D plotting to produce publication-
quality image plots and figures in a variety of formats.
– The library helps to generate histograms, plots, error charts, scatter plots, bar charts with
just a few lines of code.
Data Sets Links
 Data Sets Links
• http://archive.ics.uci.edu/ml/datasets.php
• https://lionbridge.ai/datasets/the-50-best-free-datasets-for-machine-learning/
• https://cloud.google.com/solutions/datasets
• https://ml-cheatsheet.readthedocs.io/en/latest/datasets.html
• https://datarade.ai/data-categories/ai-ml-training-data
• https://www.kdnuggets.com/2020/03/20-machine-learning-datasets-project-ideas.html
• https://www.kaggle.com/

• https://www.kaggle.com/competitions
• https://www.kaggle.com/chaitanya99/causes-of-heart-attack-eda-xgboost
• https://www.kaggle.com/learn
Where to run/ How to run ML Algorithms
 Data Sets Links
Anaconda
• It is free and open-source
• It has more than 1500 Python/R data science packages
• Anaconda simplifies package management and deployment
• It has tools to easily collect data from sources using machine learning and AI

Google Colab (https://colab.research.google.com/ )

• Google Colaboratory is a platform built on top of the Jupyter Notebook environment which
runs entirely on Google Cloud Platform (GCP). This platform provides GPU which is free of cost
and supports Python 2 and 3 versions. deep learning libraries such as Keras, TensorFlow,
OpenCV and others.

OpenML (https://www.openml.org/ )
• OpenML is an open, collaborative, and automated machine learning environment which
includes several specific features such as find or add data to analyse, download or create
computational tasks, find or add data analysis flows and much more.
Industry 4.0
Industry 4.0
Usecases and Practical
Implementation
Present Scenario
Instances of failure
EXPECTED SCENARIO
Process Flow of Steel Plant
AFTER INSTALLATION OF VIBRATION
SENSORS
Autonomous Cars
Autonomous Cars Technologies
SMART FARMING
VIBRATION ANALYSIS

51
Other Usecases of Machine Learning?
• Recognizing patterns:
– Facial identities or facial expressions
– Handwritten or spoken words
– Medical images
• Generating patterns:
– Generating images or motion sequences
• Recognizing anomalies:
– Unusual credit card transactions
– Unusual patterns of sensor readings in a nuclear power plant
• Prediction:
– Future stock prices or currency exchange rates
 IOCL
Predictive
maintenance
solution
ML deployment on Embedded HW
Hardware Accelerators
 Hardware Accelerator
Used to accelerate a given specific task to support the processor.

• FPGA
• ASIC
• GPU
• VPU
• TPU
ASIC
 ASIC custom chips
ASIC (application-specific integrated circuit) is an integrated circuit customized for a
particular use, rather than intended for general-purpose use.

There is a lot of movement to ASIC right now:

● Google has Tensor Processing Units (TPU) in the cloud.

● Intel just demonstrated their Nervana processors for training and inference.
● Mobileye (Intel) chips with specially developed ASIC cores are used in BMW, Tesla,
Volvo, etc.
● Movidius (acquired by Intel) Myriad X VPU - a dedicated hardware accelerator for deep
neural network inferences. https://www.movidius.com/myriadx
● Alibaba Hanguang 800
● Huawei Ascend 310, 910
● Bitmain Sophon
● ...
GPU
 GPU
• Graphics processing unit is a specialised chip that can do rapid processing,
primarily for the purpose of rendering images.
• They have become a key part of modern supercomputing.
• Their extremely parallel structure makes them more valuable than general-
purpose central processing units (CPUs) for algorithms that process huge blocks of
data in parallel.
• Multiple GPUs are utilised on supercomputers, on workstations to expedite
processing multiple videos at once and 3D rendering, for VFX and for simulations,
and in AI for training workloads.
• They have been used in growing new hyperscale data centres and have become
accelerators, speeding up all sorts of tasks – from encryption, to networking, to AI.
• GPUs have sparked an AI boom, become a key part of modern supercomputers,
and continue to drive advances in gaming and pro-graphics.
CPU vs GPU
GPU
VPU (vision processing unit)
 VPU
• A vision processing unit (VPU) is a rising class of microprocessor, and a
particular type of AI accelerator intended to quicken machine vision tasks.

• The vision processing unit is reported as more fitting for performing various
kinds of machine vision algorithms.

• These tools may be designed with particular resources for capturing visual data
from cameras, and are built for parallel processing.

• Some of these tools are low power and high performance and may be plugged
into interfaces that enable programmable use.
Movidius (VPU)
TPU
 TPU
• A tensor processing unit (TPU) is a specialised circuit that implements all the
necessary control and arithmetic logic necessary to execute machine learning
algorithms, typically by operating on predictive models such as artificial neural
networks (ANNs) or random forests (RFs).

• The power Google TPUs can be reached in two types, which are cloud TPU and
edge TPU.

• Cloud TPUs may be accessed from Google Colab notebook, which gives users with
TPU pods which sit on Google’s data centres.
 Implementation challenges
Target Hardwares Constraints
• CPU Power Consumption
• ACCELERATORS Memory
• ASIC Speed (MIPS)
• GPU Cost
• TPU Ease of deployment
• VPU
• FPGA
Selection of Hardware:
Type of Applications
Energy Efficiency
Quality, Accuracy and power target
 Key Issues in Machine Learning
• Even if you have the data, you can still run into problems with
its quality, as well as biases hidden within your training sets. To
put it simply, the quality of training data determines the
performance of machine learning systems.

• Today there are over 250 billion embedded and IOT devices active
in the world, with an expected 20 percent yearly growth.
• These devices are gathering large amounts of data and processing
this in the cloud has presented quite a challenge.
Why is ML Moving to the Edge?
Use cases demand more embedded intelligence

Expanding opportunity
Arm for the embedded Sensor fusion Contextual awareness

Cortex-M intelligence market

Voice activation Biometrics

More data More processing More algorithms More compute

7
1
 Size of Cortex-M

•
TinyML
 What is TinyML?
TinyML is a type of machine learning that shrinks deep learning networks to fit on
tiny hardware. It brings together Artificial Intelligence and intelligent devices.

TinyML is a field of study in Machine Learning and Embedded Systems that explores the types of
models you can run on small, low-powered devices like microcontrollers. It enables low-latency,
low power and low bandwidth model inference at edge devices.

While a standard CPUs consume between 65 watts and 85 watts and standard GPU consumes
anywhere between 200 watts to 500 watts,
A typical microcontroller consumes power in the order of milliwatts or microwatts.
 Advantages of TinyML?

Low Latency: Since the model runs on the edge, the data doesn't have to be sent to a server to
run inference. This reduces the latency of the output.

Low Power Consumption: As we discussed before, microcontrollers consume very little power.
This enables them to run without being charged for a really long time.

Low Bandwidth: As the data doesn’t have to be sent to the server constantly, less internet
bandwidth is used.

Privacy: Since the model is running on the edge, your data is not stored in any servers.
 How to use TinyML?

It uses Edge-Impulse as development and deployment framework.

For optimized Deep learning libraries, it used Tensorflow Lite.

TensorFlow Lite
 What is TensorFlow Lite?
TensorFlow Lite is a companion project to TensorFlow, Google’s open-
source project designed to bring machine learning to everyone.

It’s designed for smartphones and Linux-grade devices like the

Raspberry Pi.

It’s now deployed on over two billion mobile devices in production.

TensorFlow Lite is a set of tools that helps developers run TensorFlow machine
learning models on mobile, embedded, and Internet of Things (IoT) devices.
It enables device machine learning with low latency and a smaller binary volume.
 TensorFlow Lite
• It enables on-device machine learning inference with low latency and a small
binary size

• Low latency techniques: optimizing the kernels for mobile apps, pre-fused
activations, and quantized kernels that allow smaller and faster (fixed-point
math) models

• TensorFlow Lite also supports hardware acceleration with the Android Neural
Networks API
 What is TensorFlow Lite for Microcontrollers?
Even a few hundred kilobytes is too large for most embedded systems.
If we want to run on MCUs, we need:
• Low tens of kilobytes code footprint.
• No operating system dependencies.
• No dynamic memory allocation.

We managed to create a version of TensorFlow Lite that meets these constraints,

sharing most of the same code, but with only a 20 kilobyte footprint.
tf-lite and android NN in Google I/O

• New TensorFlow runtime

• Optimized for mobile and

embedded apps

• Runs TensorFlow models on

device

• Leverage Android NN API

• Soon to be open sourced

from Google I/O 2017 video

Edge Impulse
Edge Impulse
IMAGE CLASSIFICATION USING EDGE
IMPULSE

84
PREDICTIVE ANALYSIS USING EDGE
IMPULSE

85
Future Trend for Machine Learning
Building Intelligent systems in the Cloud and at the
Edge
Summary
• ML is moving to the IoT Edge and Cortex-M plays a key role in enabling embedded
intelligence.
• Case study: keyword spotting on Cortex-M CPU
• Hardware-constrained neural network architecture exploration
• Deployment flow of model on Cortex-M CPUs
• CMSIS-NN: optimized neural network functions
 THANKS
kaushal@uniconvergetech.in
UniConverge Technologies Pvt. Ltd.
(www.uniconvergetech.in)
&
The IoT Academy
(www.theiotacademy.co)
C56/11, Sector-62, Noida