1

VISVESVARAYA TECHNOLOGICAL UNIVERSITY

BELAGAVI

Project Phase1 Work Report on

“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”
Submitted in partial fulfillment for curriculum prescribed for sixth
semesterBachelor’s degree in
Electrical & Electronics Engineering
during the year
2022-23
PROJECT ASSOCIATES

1. G VAMSHI KRISHNA [ 3PG19EE005]

2. GOPI KRISHNA.S [ 3PG19EE006]
3. GAGAN DEEP.K [ 3PG20EE410]
4. T. GOUTHAM [ 3PG20EE427]

Under the Guidance of

Prof. Firdose P.M. Tech

Assistant Professor

PROUDHADEVARAYA INSTITUTE OF TECHNOLOGY,

DEPARTMENT OF ELECTRICAL & ELECTRONICS ENGINEERING
T.B. DAM, HOSAPETE-583225

EEE Dept
 V.V.SANGHA’S

PROUDHADEVARAYA INSTITUTE OF TECHNOLOGY,

DEPARTMENT OF ELECTRICAL & ELECTRONICS ENGINEERING
T.B. DAM, HOSAPETE-583225.
(AFFILIATED TO VISVESVARAYA TECHNOLOGICAL UNIVERSITY, BELAGAVI & APPROVED BY
AICTE, NEW DELHI.)

CERTIFICATE

This is to certify that the Project Phase 1 work entitled “BIND ASSIST SYSTEM
USING ML AND IMAGE PROCESSING” is a bonafide work carried out by
“ G VAMSHI KRISHNA [ 3PG19EE005] , GOPI KRISHNA.S [3PG19EE006] , GAGAN
DEEP.K [ 3PG20EE410], T. GOUTHAM [ 3PG20EE427] ” in partial fulfillment for
curriculum prescribed for sixth semester Bachelor of Engineering in “Electrical &
Electronics” of the Visvesvaraya Technological University, Belagavi, during the year
2020-2021. It is certified that all corrections/suggestions indicated for Internal
Assessment have been incorporated in the report deposited in the department library.
The mini project report has been approved as it satisfies the academic requirements in
respect of Mini project work prescribed for said Degree.

.................................. ......................................
Signature of the Guide Signature of the HOD
Prof. FIRDOSH PM.Tech Prof. PRAKASHA .S M.TECH. (PhD)
......................................................
Signature of the Principal
Dr. S.M. SHASHIDHAR ME.PhD

External Viva

Name of the Examiner Signature with date

1. ...........................................… .........................................

2................................................ ............................
……….
 3

ACKNOWLEDGMENT

I express deep sense of gratitude to my guide Prof. FIRDOSH P, Assistant

Professor, Department of Electrical and Electronics Engineering, PDIT, Hosapete, for his
invaluable guidance and suggestions imparted at various stages in carrying out this project
Phase 1 work successfully.
I would like to thank, Prof. PRAKHASH SIR, Main project work Coordinators,
Department of Electrical and Electronics Engineering, PDIT, Hosapete, for their extended
encouragement and support during this project phase 1 work.
I would like to add a few words to express my thanks to Prof. PRAKASHA.S, HOD
of Electrical & Electronics Engineering, of PDIT, Hosapete, for their valuable and
unstintedadvice from time to time.
I am grateful to Dr. SM SHASHIDHAR, Principal, PDIT, Hosapete, for having
provided us an excellent academic environment which has nurtured our practical skills and
for kindly obliging the requests and providing timely assistance.
I express my sincere thanks to all the faculty and staff members of Department of
Electrical and Electronics Engineering, PDIT, Hosapete, for their kind and constant support
throughout the project phase1 work.
I would like to thank the Almighty, who has always guided me to work on the right
path of the life. My greatest thanks are to my parents who bestowed ability and strength in me
to complete this work.
I am also thankful to the authors whose work has been consulted, utilized and cited in
this report.
Words are often less to reveal one’s deep regards. An understanding of work like this
is never the outcome of the efforts of a single person. I take this opportunity to express my
profound sense of gratitude and respect to all those who helped me directly and indirectly
throughout the duration of this project phase.
PROJECT ASSOCIATES
1. G VAMSHI KRISHNA
2. GOPI KRISHNA.S
3. GAGAN DEEP.K
4. T.GOUTHAM

EEE Dept
 4

UNDERTAKING
We,arehereby undertake that the project phase work entitled “ BIND ASSIST SYSTEM
USING ML AND IMAGE PROCESSING ” is carried out by us independently under the
guidance of Prof. FIRDOSE, Department of Electrical & Electronics Engineering, PDIT,
Hosapete, in partial fulfillment for curriculum prescribed for sixth semester Bachelor of
Engineering in “Electrical & Electronics” of the Visvesvaraya Technological University,
Belagavi, during the year 2022-2023. The Mini project has been my original work and has
not formed the basis for the award of any degree, associateship, fellowship or any other
similar titles.

PROJECT ASSOCIATES

1. G VAMSHI KRISHNA
2. GOPI KRISHNA.S
3. GAGAN DEEP.K
4. T.GOUTHAM

4
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

ABSTRACT

There are lots of strategies or ways that have been adopted by visually impaired people to
address this hassle of theirs. A traditional approach that has been used for years by the
visually impaired humans is using dogs that could help so as to navigate through their paths
or using walking canes to keep themselves away from any obstacles. Both of them are
inexpensive or reachable but aren't error prone. Being error prone is what is wanted for blind
people as even the slightest of blunders can motive a large damage. Another manner to cope
with this problem is to provide blind people with clever rehabilitative shoes alongside the
spectacles. Each such shoe is surmounted with ultrasonic transducers to detect objects at
unique level of heights and spectacles have a pair of ultrasonic transducers mounted centrally
stored above the bridge and with a buzzer at one of the ends. A major drawback of this sensor
primarily based approach is that it is just beneficial to detect items in place of recognizing
them and hence image processing gives a promising answer to address such situations. One
more traditional methods use stick to find obstacle ,these sticks are used to find the
obstaclein front. User gets to know the obstacleInfront of him when the stick touches the
obstacle.

5
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CONTENTS
Page.no
Certificatei
Acknowledgement ii
Undertaking iii
Abstract iv
Contents v

Contents
Chapter 1
1.1 Introduction………………………………………………………………………7
Chapter 2
2.1 History……………………………………………………………………………8
Chapter 3
3.1 Block diagram……………………………………………………………………10
Chapter 4
4.1 Parts of parts……………………………………………………………………..11
4.2 Raspberry Pi 4 Model B…………………………………………………………11
4.3 RASPI CAM ……………………………………………………………………………………..12

4.4 SanDisk Micro SD/SDHC 32GB Class 10 Memory Card.....................................13

Chapter 5
5.1 Design methodology……….……………………………………………………..14
Chapter 6
6.1 Applications………………………………………………………………………15
6.2 Hardware used…………………………………………............................……….15
6.3 Software use………………………………………………………………………15
Chapter 7
7.1 LITERATURE SURVEY……………………………...........................................16
7.2 FUTURE OF SCOPE…………………………………………………………….19
7.3 CONCLUSION……………………………………………………………………19

6
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CHAPTER 1

1.1 INTRODUCTION

Visual impairment is one of the issues that several millions of people suffer from. They go
through a lot of difficulties even to complete the basic chores. Even in their own home or
office the struggle to navigate from one place to another without being dependent on
anybody. As per the data from WHO(world health organization) there are around 250+
million people with visual disablement out of which nearly 35+ million are totally blind
which constitute a huge part of the population.
Visually impaired people or in other words especially abled people are the ones who face a
lot of difficulties even to accomplish their daily routine chores. Most of them even though
they don’t want, have to rely on other people for some kind of help. There are thousands of
technologies being developed or have been developed for the assistance of these people.
Computer vision being one of these technologies is providing the most promising solution
Blind people find hard time navigating around the street.Due to their inability to see
world ,they are often in danger of getting hit by obstical and vechicle.
It is challenging for them to zebra cross since they are not able to the pedestrian traffic light.

Various technologies are used in order to build this project

Opencv is one of the open source library this is used for image processing

7
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CHAPTER 2

2.1 HISTORY

OPEN CV :

OpenCV (Open Source Computer Vision Library) is an open source computer vision and
machine learning software library. OpenCV was built to provide a common infrastructure for
computer vision applications and to accelerate the use of machine perception in the
commercial products. Being a BSD-licensed product, OpenCV makes it easy for businesses
to utilize and modify the code.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of
both classic and state-of-the-art computer vision and machine learning algorithms. These
algorithms can be used to detect and recognize faces, identify objects, classify human actions
in videos, track camera movements, track moving objects, extract 3D models of objects,
produce 3D point clouds from stereo cameras, stitch images together to produce a high
resolution image of an entire scene, find similar images from an image database, remove red
eyes from images taken using flash, follow eye movements, recognize scenery and establish
markers to overlay it with augmented reality, etc. OpenCV has more than 47 thousand people
of user community and estimated number of downloads exceeding 18 million. The library is
used extensively in companies, research groups and by governmental bodies.

Along with well-established companies like Google, Yahoo, Microsoft, Intel, IBM, Sony,
Honda, Toyota that employ the library, there are many startups such as Applied Minds,
VideoSurf, and Zeitera, that make extensive use of OpenCV. OpenCV’s deployed uses span
the range from stitching streetview images together, detecting intrusions in surveillance video
in Israel, monitoring mine equipment in China, helping robots navigate and pick up objects at
Willow Garage, detection of swimming pool drowning accidents in Europe, running
interactive art in Spain and New York, checking runways for debris in Turkey, inspecting
labels on products in factories around the world on to rapid face detection in Japan.

8
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

NUMPY

NumPy is a general-purpose array-processing package. It provides a high-performance

multidimensional array object, and tools for working with these arrays.

It is the fundamental package for scientific computing with Python. It contains various
features including these important ones:

● A powerful N-dimensional array object

● Sophisticated (broadcasting) functions
● Tools for integrating C/C++ and Fortran code
● Useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional
container of generic data.

Arbitrary data-types can be defined using Numpy which allows NumPy to seamlessly and
speedily integrate with a wide variety of databases.

Objective

● 1:To build a device which is able to identify obstacle ,and traffic sign board and
should be able to give voice alert.
● 2: to build a device which is able to read the book for blind people
● 3:The device should be small and compact
● 4:The device should be easy to manufacture and affordable to everyone

9
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CHAPTER 3

3.1 BLOCK DIAGRAM

Fig:1.1 block diagram

The brain of our system is raspberry pi ,all the processes are executed by raspberry pi.
The system consists of a camera which is used to take live video capture.The video is then
processed by raspberry .This camera is connected to raspberry pi
And SD card is connected to raspberry pi, in this SD card operating system of raspberry pi is
stored ( Raspbian OS ).
An ear phone is connected to raspberry pi to audio jack through which audio is played
A optional monitor is connected through which we can see real time video

10
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CHAPTER 4
4.1 PARTS
Hardware description

Raspberry Pi 4 Model B

Details:
Processor: Broadcom BCM2711, quad-core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz
Memory: 1GB, 2GB, 4GB and 8GB LPDDR4 (depending on model)
Connectivity: 2.4 GHz and 5.0 GHz IEEE 802.11b/g/n/ac wireless LAN, Bluetooth 5.0, BLE
Gigabit Ethernet 2 × USB 3.0 ports 2 × USB 2.0 ports.
GPIO: Standard 40-pin GPIO header (fully backwards-compatible with previous boards)
Video & sound: 2 × micro HDMI ports (up to 4Kp60 supported) 2-lane MIPI DSI display
port 2-lane MIPI CSI camera port 4-pole stereo audio and composite video port
Multimedia: H.265 (4Kp60 decode); H.264 (1080p60 decode, 1080p30 encode); OpenGL
ES, 3.0 graphics
SD card support: Micro SD card slot for loading operating system and data storage
Input power: 5V DC via USB-C connector (minimum 3A1 ) 5V DC via GPIO header
(minimum 3A1 ) Power over Ethernet (PoE)–enabled (requires separate PoE HAT)
Environment: Operating temperature 0–50ºC
Raspberry Pi 4 Model B is the latest product in the popular Raspberry Pi range of computers.
It offers ground-breaking increases in processor speed, multimedia performance, memory.

11
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

RASPI CAM

The 8MP Raspberry Pi Official Camera Module v2 can be used to take high-definition video,
as well as stills photographs. It uses high quality 8 megapixel Sony IMX219 image sensor
custom designed add-on board for Raspberry Pi, featuring a fixed focus lens. It is capable of
3280 x 2464 pixel static images, and also supports 1080p30, 720p60 and 640x480p60/90
video.

Raspberry Pi Camera V2 attaches to Pi by way of one of the small sockets on the board
upper surface and uses the dedicated CSI interface, designed especially for interfacing to
cameras. It is suitable for mobile or other applications where size and weight are important.

It connects to Raspberry Pi by way of a short ribbon cable. It’s easy to use for beginners. The
camera works with all models of Raspberry Pi 1, 2, and 3. It can be accessed through the
MMAL and V4L APIs, and there are numerous third-party libraries built for it, including the
Picamera Python library.

The camera module is very popular in home security applications, and in wildlife camera
traps.

Applications

CCTV security camera

Motion Detection

Time Lapse Photography etc.

12
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

SanDisk Micro SD/SDHC 32GB Class 10 Memory Card

Sandisk Micro SD/SDHC 32GB Memory Card is Multipurpose Class 10 SD Card comes
which can convert this SD card to SDHC or SDXC for multipurpose use like PC Storage,
DSLR camera etc.

Give the power to your project with Sandisk Micro SD/SDHC 32GB Class 10 Memory Card.
A1 series memory card, new and made for APP or operating system to load and run more
faster and smoother ever before. With speed up to 98 MB/s Shoot, save and share more with
high speed. Built to perform in harsh conditions, SanDisk Ultra microSDHC UHS-I cards are
Moistureproof, temperature proof, shockproof, and X-ray proof.

The A1 microSD card is rated Class 10 which is known for the high speed of
Read/Write operation which helps to load applications and operating system faster in
development board like the raspberry pi.
Durable Design for Use in Harsh Environments SanDisk Ultra microSD UHS-I Cards are
shockproof, temperature-proof, waterproof, and X-ray-proof so you can enjoy your
adventures without worrying about the durability of your memory card. Premium Transfer
Speeds of up to 98MB/s.Your content moves amazingly fast. Now you can transfer up to
1200 photos in just one minute.

13
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CHAPTER 5

Design methodology

Our project consists of two steps to achieve blind assist using image processing and machine
learning
The steps are :

1:Designing and training the neural network to identify objects

A convolution neural network is designed with multiple dense layer, with several activation
functions such as relu and sigmoid.This neural network is trained with 1000’s of images of
objects which are used in day to day life.The images used to train are collected using open
source data sets available on the internet.The trained neural network is saved and used to
identify object.

2: Object recognition and audio conversion

Live video acquisition using camera is the first step.video will be converted into frames of
images which will be used to detect object. The image is then pre processed to achieve
required resolution using open cv library.Pre-processes image will be sent to Convolution
Neural Network to classify and detect object.The detected object is then announced using
speaker

14
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CHAPTER 6
6.1 APPLICATION

Application
1: can be used to assist blind people
2: can be used to assist old age people
3: can be used to assist people with slow reaction speed due to head injury
4: can be implemented in automobiles to increase safety
Hardware used
Raspberry pi
Speaker
Battery
Camera
Software used
Python
Opencv

15
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

CHAPTER 7
7.1 LITERATURE SURVEY
1.Assisting Blind People Using Object Detection with Vocal Feedback
Publisher: IEEE
Heba Najm; KhirallahElferjani; AlhaamAlariyibi

Abstract:
For visually impaired people, it is highly difficult to make independent movement and safely
move in both indoors and outdoors environment. Furthermore, these physically and visually
challenges prevent them from in day-to- day live activities. Similarly, they have problem
perceiving objects of surrounding environment that may pose a risk to them. The proposed
approach suggests detection of objects in real-time video by using a web camera, for the
object identification, process
Published in: 2022 IEEE 2nd International Maghreb Meeting of the Conference on
Sciences and Techniques of Automatic Control and Computer Engineering (MI-STA)
https://ieeexplore.ieee.org/document/9837737

2. Reader and Object Detector for Blind

Publisher: IEEE

M. Murali; Shreya Sharma; Neel Nagansure

Abstract:
This work aims to assist the visually impaired people for reading a text material and detect
objects in their surroundings. The input is taken in the form of an image captured from the
web camera. This image is then processed either for the purpose of text reading or for object
detection based on user choice. The Raspberry Pi acts as the microcontroller for processing of
the entire process. The text reading is supported by software named OCR. The read text is
changed into an audio output using the TTS Synthesis.
Published in: 2020 International Conference on Communication and Signal Processing
(ICCSP)

https://ieeexplore.ieee.org/document/9182201

16
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

3.Robot Eye: Automatic Object Detection And Recognition Using Deep Attention
Network to Assist Blind People
Publisher: IEEE

Ervin Yohannes; Paul Lin; Chih-Yang Lin; Timothy K. Shih

Abstract:
Detection and Recognition is a well-known topic in computer vision that still faces many
unresolved issues. One of the main contributions of this research is a method to guide blind
people around an outdoor environment with the assistance of a ZED stereo camera, a camera
that can calculate depth information. In this paper, we propose a deep attention network to
automatically detect and recognize objects. Published in: 2020 International Conference on
Pervasive Artificial Intelligence (ICPAI)
https://ieeexplore.ieee.org/document/9302726

4.Detection and Identification of Things for Blind People Using Raspberry PI

Publisher: IEEE

S. Gayathri; K. Jeyapiriya; K. JeyaPrakash; K. Lalitha

Abstract:
This work assists visually impaired persons with grocery shopping. The robot module
(trolley) contains a Raspberry Pi, as well as an RFID reader, a headset, and motors. The
person's speech (part where the person wants to go) is used as input, and this information is
sent to the Raspberry Pi through Bluetooth. According to the specifications, the Raspberry
will send a command to the driver IC, which will then drive the motors in the desired
direction. All of the objects in the section will be RFID- tagged. The RFID scanner will
detect (read) the tag whenever an item is picked and dropped into the cart. This will be
relayed to Raspberry Pi, which will then send audio output (item name and price) to the
person via headset. In addition, all things in the trolley are logged in the IOT and printed at
the bill section.

17
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”
Published in: 2021 4th International Conference on Computing and Communications
Technologies (ICCCT)

https://ieeexplore.ieee.org/document/9711820

5.Visual Assistance for Blind Using Image Processing

Publisher: IEEE

B Deepthi Jain; Shwetha M Thakur; K V Suresh

Abstract:
Visually impaired people face lot of difficulties in their daily life. Many a times they rely on
others for help. Several technologies for assistance of visually impaired people have been
developed. Among the various technologies being utilized to assist the blind, Computer
Vision based solutions are emerging as one of the most promising options due to their
affordability and accessibility. Published in: 2018 International Conference on
Communication and Signal Processing (ICCSP)
https://ieeexplore.ieee.org/document/8524251

6.Smart Assistive System for Visually Impaired People Obstruction Avoidance Through
Object Detection and Classification
Publisher: IEEE
Usman Masud; Tareq Saeed; Hunida M. Malaikah; FezanUl Islam; Ghulam Abbas

Abstract:
Recent progress in innovation is making the life prosper, simpler and easier for common
individual. The World Health Organization (WHO) statistics indicate that a large amount of
people experience visual losses, because of which they encounter many difficulties in
everyday jobs.
Published in: IEEE Access( Volume: 10)
https://ieeexplore.ieee.org/document/9691323

7.Design and Implementation of Obstacle Detection and Warning System for Visually
Impaired People
Publisher: IEEE
18
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

Yusuf Sahabi Lolo; Kelechi Lawrence Ohammah; Amina Nna Alfa; Sadiq Abubakar
Mohammed;

Abstract:
Environmental information assists human beings to learn about the source that surround
them, most visually impaired people make extensive use of the auditory environment not just
to determine the presence of an obstacle, but also to successfully maneuver around it. This
paper discusses various methods for improving blind people’s navigation by utilizing readily
available technologies. The system includes the ability to detect obstacles for collision
avoidance, as well as the ability to detect objects in up, down, and front directions using
ultrasonic sensor. Published in: 2022 IEEE Nigeria 4th International Conference on
Disruptive Technologies for Sustainable Development (NIGERCON)

https://ieeexplore.ieee.org/document/9803138

7.2 FUTURE OF TESLA COIL: -

The device can be made much smaller by the use of SMD components and a micro PCB
board and low power, by reducing the size of the components placement of the camera can be
even more perfect and detection of the object will be even more clear which will be very
helpful for the blind person to carry the product

7.3 CONCLUSION: -

In this project we have trying to help the blind person by identifying the object in front to the
blind person and convert it to audio and it can played in the earphone of the blind person, and
in this project we have book reading features which will help the blind person to come out
from the conventional book reading to accurate book reading using image processing and text
reorganization .

19
DEPT OF EEE
“ BIND ASSIST SYSTEM USING ML AND IMAGE PROCESSING ”

REFERENCES

[1] Vision impairment and blindness: Fact sheet Available:

http://www.who.int/mediacentre/factsheets/fs282/en/.
[2] Ziad O. Abu-Faraj, Paul Ibrahim, Elie Jabbour and Anthony Ghaoui, “Design and
Development of a Prototype Rehabilitative Shoes and Spectacles for the Blind”, IEEE Int.
Conf. BioMedical Engineering and Informatics, 2012, pp. 795-799.
[3] Giva Andriana Mutiara, Gita Indah Hapsari and RamantaRijalul, “Smart Guide Extension
for Blind Cane", IEEE Int. Conf. Information and Communication Technologies, 2016.
[4] G. Balakrishnan, G. Sainarayanan, R. Nagarajan, and Sazali Yaacob, “A Stereo Image
Processing System for Visually Impaired”, Int. Journal of Computer, Electrical, Automation,
Control and Information Engineering, vol.2, No.8, 2008, pp. 2794-2803.
[5] Rui Jiang, Qian Lin Li, “Let Blind People See: Real-Time Visual Recognition with
Results Converted to 3D Audio”, Proc. International Conference on Computer Vision, 2015.
[6] M. Hiromoto, H. Sugano, and R. Miyamoto,“Partially Parallel Architecture for AdaBoost-
Based Detection With Haar-Like Features”, IEEE Trans. Circuits and Systems for Video
Technology, vol. 19, Jan 2009, pp. 41-52.
[7] Pritpal Singh, B.B.V.L. Deepak, Tanjot Sethi and Meta Dev Prasad Murthy ,“Real Time
Object Detection and Tracking Using Color Feature and Motion”, IEEE Int. Conf.
Communication and Signal Processing, 2015.
[8] Rainer Lienhart and Jochen Mandt, “An Extended Set of Haar-like Features for Rapid
Objection Detection”, IEEE Int. Conf. Image Procesing, 2002.

20
DEPT OF EEE