DEEP LEARNING IN

COMPUTER VISION
TECHNOLOGY
БАЙКАЛЬСКИЙ ИНСТИТУТ БРИКС
EDGAR SOLIS ROMEU
13TH LECTURE
 • Optical character recognition technology.
• Applying computer vision and deep learning to embedded
applications
• Raspberry Pi
• NVIDIA
TODAY’S CLASS • Jetson Nano
• Coral Platform
 APPLYING COMPUTER
VISION AND DEEP LEARNING
TO EMBEDDED
APPLICATIONS
WHAT ARE EMBEDDED SYSTEMS?

Embedded systems are computer hardware systems

designed for performing dedicated functions in a
combination with a larger system. They include and are
used in many everyday items from mobile
phones to household appliances.
An embedded system is a combination of computer hardware and software designed for a specific function.
Embedded systems may also function within a larger system. The systems can be programmable or have a fixed
functionality. Industrial machines, consumer electronics, agricultural and processing industry devices, automobiles,
medical equipment, cameras, digital watches, household appliances, airplanes, vending machines and toys, as well
as mobile devices, are possible locations for an embedded system.
While embedded systems are computing systems, they can range from having no user interface (UI) -- for
example, on devices designed to perform a single task -- to complex graphical user interfaces (GUIs), such
as in mobile devices. User interfaces can include buttons, LEDs (light-emitting diodes) and touchscreen
sensing. Some systems use remote user interfaces as well.
EXAMPLES OF EMBEDDED SYSTEMS

• Automobiles. Modern cars commonly consist of many computers (sometimes as many as 100), or embedded
systems, designed to perform different tasks within the vehicle. Some of these systems perform basic utility
functions and others provide entertainment or user-facing functions. Some embedded systems in consumer
vehicles include cruise control, backup sensors, suspension control, navigation systems and airbag systems.
• Mobile phones. These consist of many embedded systems, including GUI software and hardware, operating
systems (OSes), cameras, microphones, and USB (Universal Serial Bus) I/O (input/output) modules.
• Industrial machines. They can contain embedded systems, like sensors, and can be embedded systems
themselves. Industrial machines often have embedded automation systems that perform specific monitoring
and control functions.
• Medical equipment. These may contain embedded systems like sensors and control mechanisms. Medical
equipment, such as industrial machines, also must be very user-friendly so that human health isn't
jeopardized by preventable machine mistakes. This means they'll often include a more complex OS and GUI
designed for an appropriate UI.
Embedded systems always function as part of a complete device -- that's what's meant by the term
embedded. They are low-cost, low-power-consuming, small computers that are embedded in other
mechanical or electrical systems. Generally, they comprise a processor, power supply, and memory and
communication ports. Embedded systems use the communication ports to transmit data between the
processor and peripheral devices -- often, other embedded systems -- using a communication protocol.
The processor interprets this data with the help of minimal software stored on the memory. The software
is usually highly specific to the function that the embedded system serves.
• The processor may be a microprocessor or microcontroller. Microcontrollers are simply microprocessors
with peripheral interfaces and integrated memory included. Microprocessors use separate integrated
circuits for memory and peripherals instead of including them on the chip. Both can be used, but
microprocessors typically require more support circuitry than microcontrollers because there is less
integrated into the microprocessor. The term system on a chip (SoC) is often used. SoCs include multiple
processors and interfaces on a single chip. They are often used for high-volume embedded systems.
Some example SoC types are the application-specific integrated circuit (ASIC) and the field-
programmable gate array (FPGA).
Often, embedded systems are used in real-time operating environments and use a real-time operating system (RTOS)
to communicate with the hardware. Near-real-time approaches are suitable at higher levels of chip capability,
defined by designers who have increasingly decided the systems are generally fast enough and the tasks tolerant of
slight variations in reaction. In these instances, stripped-down versions of the Linux operating system are commonly
deployed, although other OSes have been pared down to run on embedded systems, including Embedded Java and
Windows IoT (formerly Windows Embedded).
CHARACTERISTICS OF EMBEDDED SYSTEMS
The main characteristic of embedded systems is that they are task-specific.
Additionally, embedded systems can include the following characteristics:
• Typically, consist of hardware, software and firmware.
• Can be embedded in a larger system to perform a specific function, as they are built for specialized tasks
within the system, not various tasks.
• Can be either microprocessor-based or microcontroller-based -- both are integrated circuits that give the
system compute power.
• Are often used for sensing and real-time computing in internet of things (IoT) devices, which are devices that
are internet-connected and do not require a user to operate.
• Can vary in complexity and in function, which affects the type of software, firmware and hardware they use.
• Are often required to perform their function under a time constraint to keep the larger system functioning
properly.
STRUCTURE OF EMBEDDED SYSTEMS

• Hardware. The hardware of embedded systems is based around microprocessors and microcontrollers.
Microprocessors are very similar to microcontrollers and, typically, refer to a CPU (central processing
unit) that is integrated with other basic computing components such as memory chips and digital signal
processors (DSPs). Microcontrollers have those components built into one chip.
• Software and firmware. Software for embedded systems can vary in complexity. However, industrial-
grade microcontrollers and embedded IoT systems usually run very simple software that requires little
memory.
• Real-time operating system. These are not always included in embedded systems, especially smaller-
scale systems. RTOSes define how the system works by supervising the software and setting rules during
program execution.
In terms of hardware, a basic embedded system would consist of the following elements:
• Sensors convert physical sense data into an electrical signal.
• Analog-to-digital (A-D) converters change an analog electrical signal into a digital one.
• Processors process digital signals and store them in memory.
• Digital-to-analog (D-A) converters change the digital data from the processor into analog data.
• Actuators compare actual output to memory-stored output and choose the correct one.
TYPES OF EMBEDDED SYSTEMS
There are a few basic embedded system types, which differ in their functional requirements. They are:
Mobile embedded systems are small-sized systems that are designed to be portable. Digital cameras are an example of
this.
Networked embedded systems are connected to a network to provide output to other systems. Examples include home
security systems and point of sale (POS) systems.
Standalone embedded systems are not reliant on a host system. Like any embedded system, they perform a specialized
task. However, they do not necessarily belong to a host system, unlike other embedded systems. A calculator or MP3
player is an example of this.
Real-time embedded systems give the required output in a defined time interval. They are often used in medical,
industrial and military sectors because they are responsible for time-critical tasks. A traffic control system is an example of
this.
• Embedded systems can also be categorized by their performance requirements:
• Small-scale embedded systems often use no more than an 8-bit microcontroller.
• Medium-scale embedded systems use a larger microcontroller (16-32 bit) and often link
microcontrollers together.
• Sophisticated-scale embedded systems often use several algorithms that result in software and
hardware complexities and may require more complex software, a configurable processor and/or a
programmable logic array.
 EMBEDDED SYSTEMS AND COMPUTER VISION
While the variety of the embedded computer devices produced and used is quite wide, most academic
research conducted on embedded machine learning is focused on using Raspberry Pi and NVIDIA Jetson
devices. Some other devices used include the ASUS Tinker board series, Google’s Coral TPU dev series,
ODROID-XU4 Boards, and the Banana Pi board series.
 WHICH EMBEDDED SYSTEM SHOULD I USE?

When choosing an embedded computing device for specific applications, many different parameters need to be kept in
mind. The parameters include, but are not limited to, system processing speed, affected by the integrated CPU and GPU
of a system, system memory affected by the RAM, system storage space, system bus and drivers, the overall power
consumption of a system, and its cost of purchase.
 PROCESSOR UNITS
Processing units are the integrated electrical circuits responsible for performing the fundamental algorithmic and
arithmetic logic processes for running a computer device. There are different categories of processors, with the
most common ones in embedded computer systems being CPUs and GPUs. Central Processing Units, or CPUs, are
the processors present in most electrical devices and are responsible for the execution of programs and
applications, they are usually composed of multiple cores and have their performance measured in gigahertz.
Graphical Processing Units, or GPUs, are dedicated processors used for graphical rendering, allowing devices to
allocate graphically intensive tasks, such as real-time object recognition, to them.
 MEMORY UNITS
System memory generally refers to a computing system’s Random Access Memory or RAM, which is responsible for
storing application data for quick access. The larger a system’s RAM, the quicker the system can run simultaneous
applications, making RAM proportional to the overall performance of a system. Embedded computing devices
are packaged with their own memory component, with most embedded systems having between 1 GB, 2 GB, and 4
GB of RAM, while the most recent NVIDIA kits have between 8 GB and 16 GB. Memory Bandwidth is another
important parameter of system memory, indicating the rate at which data can be accessed and edited, with
the bandwidth of the system included ranging from 128-bit to 256-bit.
STORAGE UNITS

Computer storage refers to the component of a computing device responsible for retaining longtime application and
computation data. While access and alteration to storage data by the CPU are much slower than its access to RAM data, it
consumes far less power and processing capability. Storage systems come in many varieties such as flash drives, hard
drives, solid state drives, SD cards, and embedded MultiMediaCard memory or eMMC. Hard drives have been the most
common form of storage up until recently, with their advantage over other alternatives being their overall size and their
downside being their relatively slow data access speed. Solid state drives or SSDs have provided far faster data access at
the cost of storage size, however, in recent years, SSDs have made leaps in storage capacity and are now comparable in
overall storage size to hard drives. Flash drives are quick and easy to connect or disconnect from different computing
devices while having very small storage space, they are very similar to SSDs in terms of performance. Secure digital cards
or SD cards are also similar to flash storage but have both much smaller sizes and storage capacities. eMMCs are
architecturally similar to flash storage and are generally used in small laptops and embedded computing systems. Most
development kit embedded computing systems contain eMMCs, this being very much the case in NVIDIA Jetson, Coral
Edge, and ASUS Tinker board devices, and others, such as ODROID-XU4 boards, do not have their own integrated storage
devices but instead have flash storage interface. Raspberry Pi boards have interfaces for both SD cards and Flash drives.
 OPERATING SYSTEMS
Operating systems are responsible for managing and running all of the applications on a computing device, allowing
applications to make requests for services through a defined application program interface (API). . Operating systems
rely on a library of device drivers to their services to specific hardware environments, so while every application makes
a common call to a storage device, it is the OS that receives that call and uses the corresponding driver to translate the
call into commands needed for the underlying hardware. Hardware capabilities are divided into three sections:
providing UI through a CLI or GUI, launching and managing application execution, and identifying and exposing system
hardware resources to the applications. Most embedded computing systems either utilize operating systems based on
or very similar to general-purpose computer operating systems. For example, Nvidia Jetson boards have Linux for Tegra
included in their development software kits.
BUS AND DRIVERS

• Computer buses are communication systems responsible for transferring data between the various
components of a computing system. While most home computer systems have 32-bit to 64-bit buses,
embedded devices have far smaller bit rates between 4-bit and 8-bit.
• Drivers refer to the systems responsible for communicating the software of a computer device to its
hardware component. They generally run at a high privilege level in the OS run time environment, and
in many cases are directly linked to the OS kernel, which is a portion of an OS such as Windows, Linux,
or Mac OS, which remains memory-resident and handles execution for all other code. Drivers are what
defines the messages from the OS to a specific device that facilitate the devices’ fulfillment of the OS’s
request. The device drivers used in each embedded computing system are related to the operating
systems of each device.
EMBEDDED TOOLS FOR
COMPUTER VISION
NVIDIA JETSON

• Jetson is the name of a series of machine

learning embedded systems by NVIDIA used for
autonomous devices and various embedded
applications. they are generally very reliable for
implementing machine learning tasks—this is
especially true for more graphically intensive
applications.
• The downside to this is that NVIDIA Jetson
boards also tend to be more costly than
market alternatives.
GOOGLE CORAL

• Google Coral Dev Board is a single-board

computer by Coral that can be used to perform
fast machine learning (ML) inferencing in a small
form factor; it is mainly used for prototyping
custom embedded systems, but it can also be
used for embedded machine learning on its
own. It has an Edge TPU coprocessor that is
capable of performing 4 trillion operations per
second, as well as being compatible with
TensorFlow Lite.
RASPBERRY PI

• Raspberry Pi is a series of extremely popular

embedded computers developed by the
Raspberry Pi Foundation in the United Kingdom.
The uses for these systems are extremely wide,
including machine learning. Like the Jetson
series, Raspberry Pi products are very commonly
used in embedded machine-learning
implementation projects.
ODROID XU4

• The ODROID XU4 is an energy-efficient single-

board embedded computing system
by Hardkernel Co. located in South Korea. It is
compatible with open-source software and can
use different versions of Linux, such as Ubuntu,
as its operating system.
BANANA PI

• Banana Pi is an open-source hardware platform

by Shenzhen SINOVOIP Co. Located In China.
Like other embedded systems, it has a wide
range of applications, amongst them, embedded
machine learning implementation.
ASUS TINKER BOARD

• The ASUS Tinker Board S is a powerful SBC board

with a wide range of functions such as computer
vision, gesture recognition, image stabilization,
and processing, as well as computational
photography.
• The ASUS Tinker Edge R is specifically developed
for AI applications, containing an integrated
Machine Learning (ML) accelerator that speeds
up processing efficiency, lowers power
demands, and makes it easier to build
connected devices and intelligent applications.
COMPUTER VISION IN FACE
RECOGNITION TASKS
Computer
Vision in
Medical
Applications
COMPUTER VISION
IN AUTONOMOUS
VEHICLES
OPTICAL CHARACTER
RECOGNITION
WHAT IS OPTICAL CHARACTER RECOGNITION?

• OCR is a technology that analyzes the text of a

page and turns the letters into code that may be
used to process information. OCR is a technique
for detecting printed or handwritten text
characters inside digital images of paper files,
such as scanning paper records (optical
character recognition). OCR systems are
hardware and software systems that turn
physical documents into machine-readable text.
• Text is copied or read using technology such as
an optical scanner or dedicated circuit board,
while the software handles further analysis. The
essential application of OCR is to convert hard
copy legal or historical documents into PDFs.
Users may modify, style, and analyze the paper
as if generated with a word processor after it is
saved in pdf format.
HOW DOES IT WORK?

• An OCR system is made up of both hardware

and software. The service aims to analyze a
physical document's content and convert the
elements into a script that can subsequently be
utilized for processing data.
 Image Pre-Processing AI Character Recognition Post-Processing

Artificial Intelligence analyses the image's dark

portions to recognize characters and numerals.
• In the first stage, the technology converts Typically, AI uses one of the following
the document's physical shape into a approaches to target one letter, phrase, or
picture, such as a record picture. The paragraph at a time:
purpose of this stage is for the machine's Pattern Recognition: Technologies use a range
representation to be precise while also of language, text formats, and handwriting to AI corrects flaws in the final file during Post-
removing any undesired aberrations. train the AI system. The program compares the Processing. One approach is to teach the AI a
• The concept is subsequently transformed letters on the detected letter picture to the glossary of terms that will appear in the paper.
to a black and white rendition, evaluated notes it has already learned to find matches. Then, limit the AI's output to those
for bright vs. dark regions (characters). Feature Recognition: The algorithm uses rules words/formats to verify that no interpretations
• The image is then segmented into based on specific character properties to are beyond the vocabulary.
individual pieces, such as spreadsheets, recognize new characters. The amount of
text, or inset graphics, using an OCR angled, crossing, or curved lines in a letter is
system. one example of a feature.
To identify original characters, the algorithm
employs rules based on particular character
attributes.
WHAT MAKES OCR POSSIBLE?
PDF Scanner: Document
Scan+ OCR Online OCR Office Lens

'PDF Scanner:Document Scan+ OCR' is one of Office Lens is a mobile-based OCR that
the most famous OCR tools, and it tends to This OCR is likewise very basic and Microsoft developed. Its primary function is
garner positive feedback for its user-friendly straightforward to use and may be accessed to convert notes written on whiteboards to
features. The program, compatible with online. In addition, the 'Free Online OCR' is digital format. It can also edit digital versions
Android users, allows you to add your beneficial because it supports 46 languages, of printed papers, letterheads, and billboards.
signature to papers by importing photos and including Italian, Portuguese, Spanish, Its appeal originates from its capacity to
Pdfs. Japanese, and Chinese. improve and optimize photos taken,
dynamically resizing them to scale.
WHAT ARE THE BENEFITS?

• The key benefits of OCR technology are time

savings, reduced mistakes, and reduced effort.
Compressing into ZIP files, emphasizing phrases,
integrating into a webpage, and forwarding to
an email are options that aren't available with
hard copies.
• While photographing papers allows them to be
digitally stored, OCR adds the ability to alter and
search those documents.
OCR APPLICATIONS

• Word Processing • Legal Documentation

Word processing is perhaps one of the first and most Crucial approved legal papers, such as loan
popular applications of OCR. Print files may be documentation, can be scanned and stored in an
scanned and turned into modifiable and accessible electronic database for convenient retrieval. The
versions—AI assists in ensuring that these papers are documents may also be viewed and shared by many
transformed as accurately as feasible. people.

• Banking
You may snap a front and back photo of a cheque you want to
deposit with your phone. The check may be automatically
reviewed by AI-powered OCR technology to ensure it is
legitimate and verifies the cash you wish to deposit.
• Converting physical writing to digital required human labor; each page would have to be retyped, a
time-consuming and error-prone job. The conversion takes less time using the OCR system and is more
accurate than the original material. Users can modify, style, and search a page once OCR turns it into
pdf format. They can also quickly share it through email, embed it in a webpage, and save it as zip files.
• This document interpretation capacity enables firms to study many documents without having to use
human labor. Thus, reducing time-consuming admin duties is essential for increasing work engagement
and lowering attrition.
OPTIONS
• Amazon Textract is a machine learning (ML) service that
uses OCR to automatically extract text, handwriting, and
data from scanned documents such as PDFs. It can read
thousands of different documents in multiple layouts and
formats at high speed. When it extracts information from
documents, Amazon Textract returns a confidence score
for everything it identifies so that you can make informed
decisions about how you want to use the results.
• Amazon Rekognition can analyze millions of images and
videos within minutes and augment human visual review
tasks with artificial intelligence. You can use Amazon
Rekognition APIs to extract text from both images and
videos. You can extract skewed and distorted text from
images and videos of street signs, social media posts, and
product packaging.
- OCR (Optical Character Recognition) with world-
class Google Cloud AI
https://cloud.google.com/use-cases/ocr?hl=en
- Microsoft OCR Engine
https://learn.microsoft.com/en-us/azure/ai-
services/computer-vision/overview-ocr
- Adobe OCR
https://www.adobe.com/acrobat/guides/what-is-
ocr.html
REFERENCES

• file:///C:/Users/Admin/Downloads/MLConference_Jelena_RG.pdf
• file:///C:/Users/Admin/Downloads/sensors-23-02131.pdf
• https://www.techtarget.com/iotagenda/definition/embedded-system
THANK YOU FOR YOUR ATTENTION