Proceedings of the Second International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2018)

IEEE Xplore Part Number:CFP18OZV-ART; ISBN:978-1-5386-1442-6

A Novel Design of Autonomous Cars using

IoT and Visual Features
B Padmaja P V Narasimha Rao M Madhu Bala E Krishna Rao Patro
Department of CSE Department of CSE Department of CSE Department of CSE
Institute of Aeronautical Institute of Aeronautical Institute of Aeronautical Institute of Aeronautical
Engineering College Engineering College Engineering College Engineering College
Hyderabad, Telangana, Hyderabad, Telangana, Hyderabad, Telangana, Hyderabad, Telangana,
India India India India
b.padmaja@gmail.com Pvnrao222@gmail.com baladandamudi@gmail.c patrosir@gmail.com
om

Abstract— Autonomous car is a ground vehicle that is capable initial development of computer program for Advanced
of driving without user interference. Traffic congestion and Driver Help Systems (ADAS).
number of collisions are major issues in road traffic control due
to rapid increase day-by-day. Autonomous cars provide a TSR works in two phases such as detection and
solution to this problem in an efficient and economical way. recognition phase. The sole aim of detection phase is to
The proposed system utilizes mathematical models like neural focus on the signs in the optical flow. But the core challenge
networks and image processing techniques to sense the in this phase is how to eliminate all the non-sign objects
environment. This is implemented as three major components: from the image. Especially, the problems related to partial
curved road detection (steering), road sign and signal detection (blocking objects/clogging objects), rotation, distortion,
and obstacle detection (collision avoidance). Back Propagation shape (twist/bend/change the shape), overlapping signs and
is used for steering control with detection of curved roads; to the dangerous (related to surrounding conditions or the
Haar features are used for road signal, sign detection and a health of the Earth) conditions. In this stage, the camera
distance sensor for collision avoidance. Data collected from the captures the image of the road and detects where the sign is
sensors is sent to a server for processing. Based on the result, a found with the help maximally stable external regions
command is sent to the car. A GPS module attached to the car (MSER) technique. For recognition and extraction of the
identifies the location of the car and with the help of a 3rd traffic signs MSER method with shape data is used. The
party location service, route to destination is identified and steps used in detection phase are depicted in figure 1. The
directions are sent to the car. Wireless networks are used to MSER area identifier is used to find maximally stable
transmit data between sensors and the server. Python scripts districts, for example, street traffic signs. This technique is
are used to control and integrate all the units together. The
connected to two monochromatic plans S and V. These
designed system can attain high accuracy with real – time
constraints.
procedures work in parallel route as shown in figure 2. In
every iteration, locales of intrigue are identified. Therefore,
the combination of the consolidated outcomes from every
Keywords—Autonomous Car; Neural Networks; Image iteration is processed. At that point, the maximally stable
Processing; GPS; Distance Sensor; Back Propagation associated parts are separated and known as MSER locales.

I. INTRODUCTION
Automatic traffic sign recognition plays an important
role in our day to day life and because of increasing concern
about traffic safety. It avoids sudden unplanned bad
events/crashes by warning the driver about the dangers /
risks ahead. Sometimes the drivers may not see or aware of
presence road signs because of tiredness or bad surrounding
conditions, so the automatic recognition system works as an
alert system to the drivers. Over the last few years, the self-
propelling vehicles are more in industry and this has
promoted many large-scale industrial projects, researches to
develop self-ruling and smart vehicles. In self-controlled
cars, safety systems are carefully thought about/believed as
an essential means to improve road safety and security. In
these days, evolution of embedded systems and software’s
has enhanced the value of the vehicles over 30-40% [Fig. 1 Flowchart of Detection Process]
worldwide and still this percentage is expected to increase.
Traffic Sign Recognition (TSR) is considered as one of the

978-1-5386-1442-6/18/$31.00 ©2018 IEEE 18

 Proceedings of the Second International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2018)
IEEE Xplore Part Number:CFP18OZV-ART; ISBN:978-1-5386-1442-6

In the second step, (more than two, but not a lot of) and then followed by various sensors like motion sensors,
machine-learning ways of doing things are applied to cameras, and different computer-aided driving technologies
recognize the (pulled out or taken from something else) [1, 2]. One of the popular area of research in autonomous
features.The target of the recognition procedure is to navigation system is driver assistance systems development
recognize the importance of the removed image as indicated (DASD) and many embedded systems used in recent
by a database. At that point, the size standardization of the vehicles are intended to assist the driver to avoid accidents
removed articles is completed keeping in mind the end goal and unsafe situations by providing comfort and energy
to acquire a grid of 60 X 60 pixels measurement. This step efficiency. Present systems contain fully electronic driving
extracts the highlights, and then it does highlights
support, like the anti-lock braking system (ABS), the system
coordinating after examining a database. Extraction
trajectory control, automatic parking cameras and mirrors.
highlights is completed on account of the arranged quick and
turned short ORB (Oriented FAST and Rotated BRIEF)
descriptors which is shown in figure 3. This method follows Safat B Wali et al. (2015) designed a system called
a quick paired descriptor that consolidates the FAST key automatic Traffic Sign Detection and Recognition (TSDR),
point identifier and creates a BRIEF descriptor. Put away which is an emerging area of research in the field of ADAS.
ORB highlights from signs in the database are contrasted and In this paper, author has designed a traffic signaling system
those from obtained pictures. Highlight coordinating permits which provides significant visual information to assist the
choosing, whether the protest is a sign or not. Figure 3 drivers such as how to drive on proper lanes, controlling
represents a case of the coordinating element process. speed limit, avoiding obstacles, lanes for pedestrians’, and
current traffic condition etc. If the driver fails to notice these
traffic signs and then it may lead directly / indirectly to
accidents. This kind of system provides extensive support to
drivers by providing information about the traffics and its
surroundings by detecting and recognizing those traffic signs
[3]. This system is also useful in identifying and classifying
traffic signs within images that are taken by cameras /
imaging sensors [4].

Rongqiang Qian et al. (2015) developed a robust Chinese

traffic sign detection and recognition (TSDR) system using
deep convolutional neural network (CNN). Collection of
[Fig. 2 Flowchart of Recognition Module] traffic sign information from various sources is important in
many applications, such as autonomous driving, mapping,
navigation and intelligent transportation systems. Normally,
a TSDR system involve two interrelated issues: traffic sign
detection (TSD) and traffic sign classification (TSC). The
aim of TSD is to correctly localize the traffic signs contained
in an image, while the aim of TSC is to recognize the labels
of detected object into specific categories/subcategories.
This topic has gained a lot of interest among researchers in
the area of computer vision for many years [5], but it is
[Fig. 3 Matching Results]
considered as a challenging task due to numerous
This paper is organized into the following subsequent complexities like diversified backgrounds of traffic sign
sections. Section 2 gives a summary of the related work done images. This Chinese TSDR system is a complex one
in this field. Section 3 gives an overview of the proposed compared to western countries due to increase in number of
system and its advantages compared to the existing system. Chinese characters. Many feature description algorithms like
It also covers different systems which formed the base for circle detector [6], histogram of gradients, scale-invariant
this proposed system. Section 4 explains the overview of the feature transform (SIFT) and HAAR-wavelet [7] are used in
architecture of the system, algorithms used different modules traffic sign recognition. However, these features are
that are present in the proposed system and how these
collected manually and divided using the classifier, which
modules work together. The last section discusses the results
indicates there is no joint-optimization for the two modules.
obtained from the system.
III. PROPOSED SYSTEM
II. RELATED WORK The proposed system refers to a system which is an
Rihab Hmide et al. (2016) proposed a system where the improvement for existing system. This system also addresses
safety travelers are considered as the primary concern in issues of existing system. It consists of three units automatic
automobile industries due to increase in number of traffic steering (self-control), sign detection and collision
accidents and loss of human life. It started with basic airbags avoidance. Here with the help of camera module connected
with raspberry pi which is used to detect the obstacle. Once

978-1-5386-1442-6/18/$31.00 ©2018 IEEE 19

 Proceedings of the Second International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2018)
IEEE Xplore Part Number:CFP18OZV-ART; ISBN:978-1-5386-1442-6

the obstacle is detected it captures the sequence of frames in like edge detection, blurring, sharpening, etc.Usually 3
order to detect the sign. Each frame is to be analyzed with kinds of Haar features are used.
the sign and once the sign is matched. The car has automatic
steering control with the help of Arduino software and GPS
tracking control system.GPS used to track the map in order
to avoid collision.
Artificial neural networks are used to identify road
curves and trigger the car to move in that direction. Image
processing is used to find the signals on roads and to analyze
the traffic signs. With the help of Arduino system steering
controls directly just like remote control with the help of
GPS system.This system uses image processing technique to
identify the signs in the image and neural networks to [Fig. 5 Haar Features]
identify road curves in order to move in that direction with
the help GPS tracking system. Unlike the existing system, b. BACK PROPAGATION ALGORITHM: The
this system performs more after the sign matches. The Backpropagation calculation searches for the base
Arduino controls the system based on the sign with the help estimation of the blunder work in weight space
of GPS tracking system. The software requirements for this utilization method called the delta lead or angle
system is Linux OS, Raspbian OS, Open CV and GPIO and plunge. The weights that limit the blunder work is
the hardware requirements are Raspberry Pi, camera module,
then thought to be an answer for the learning
RC car, Arduino, Router, GPS, traffic sign – stop and speed
issue.The backpropagation calculation prepares a
limit and a personal computer.
given sustain forward multilayer neural system for a
given arrangement of information designs with
IV. SYSTEM DESIGN AND IMPLEMENTATION known groupings. At the point when every passage
The system is implemented in a single tier architecture of the example set is displayed to the system, then it
which contains three different modules with their respective looks at its yield reaction for the example input
functionalities. The architecture consists of design. The yield reaction is then contrasted with
the known and wanted yield the blunder esteem are
1. Computer module
ascertained. In light of the mistake, the association
2. Raspberry Pi module weights are balanced. The backpropagation
3. GPS module calculation depends on Widrow-Hoff delta learning
principle in which the weight change is done
through mean square blunder of the yield reaction
to the example input. Backpropagation neural
systems utilize a standout amongst the most well-
known neural system learning calculations, called
the Backpropagation (BP) calculation. It has been
utilized effectively for wide assortment of uses, for
example, discourse or voice acknowledgment,
picture design acknowledgment, restorative
conclusion, and programmed controls.

The screenshots of our implemented application are

taken and are shown as follows:
[Fig. 4 Overview of the System Architecture]

This paper uses two algorithms: HAAR cascade and Back

propagation algorithm.

a. HAAR CASCADE ALGORITHM: This algorithm is

used for identifying objects in images. This algorithm
uses the classifier generated during training. Haar
Cascades algorithm uses Haar features in the image to
identify the patterns. Haar features are similar to
convolutional kernel. A convolutional kernel is a small
matrix that is used for performing operations on images
[Fig. 6 Detecting a Left Turn]

978-1-5386-1442-6/18/$31.00 ©2018 IEEE 20

 Proceedings of the Second International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2018)
IEEE Xplore Part Number:CFP18OZV-ART; ISBN:978-1-5386-1442-6

Figure 6 shows the home screen of the application, which obstructions (mostly static) on the way and note their
has two interfaces - the command line and the live feed.The trademark highlights.
command line indicates the direction left which has to be
taken by the car as seen through its live feed camera. Acknowledgment
We would like to acknowledge Akhil for helping out in
experimentation and Neha for her assistance in editing and
proofreading the paper.

References
[1] C. K. Chandini, V. V. Sajith Variyar, K. Guruvayurappan, "Vision based
closed loop PID controller design and implementation for autonomous
cars", 2017 International Conference on Advances in Computing,
Communications and Informatics (ICACCI), Udupi, India, 2017,
pp.1928 – 1933, 2017.
[2] Nikolaos G. Bourbakis, Miltiadis Alamaniotis, Lefteri H. Tsoukalas, "A
Smart Car Model based on autonomous intelligent agents for reducing
accidents", 2017 IEEE Transportation Electrification Conference and
[Fig. 7 Detecting a Right Turn] Expo (ITEC), pp. 767 – 772, Chicago, IL, USA, 2017.
The interface shown in figure 7 now gives the prediction [3] Denis Osipychev, Duy Tran, Weihua Sheng, Girish Chowdhary,
"Human intention – based collision avoidance for autonomous cars",
for the user to turn right on the track. 2017 American Control Conference (ACC), pp. 2974 – 2979,
Seattle,WA, USA, 2017.
[4] Farhat W., Faiedh H., Souani C., et al, "Real – time embedded system
for traffic sign recognition based on ZedBoard", Journal of Real – Time
Image Processing, Springer Berlin Heidelberg, 2017.
[5] Saeed Asadi Bagloee, MadjidTavana, Mohsen Asadi, Tracey Oliver,
“Autonomous vehicles: challenges, opportunities and future implications
for transportation policies”, Journal Modern Transportation, Springer
Berlin Heidelberg, 2016.
[6] CananTastimur, Mehmet Karakose, Yavuz Celik, Erhan Akin, “Image
processing-based traffic sign detection and recognition with fuzzy
integral”, 2016 International Conference on Systems, Signals and Image
Processing (IWSSIP), Bratislava, Slovakia, 2016.
[7] Madhu Bala Myneni, M. Seetha, “Hybrid feature Extraction and
selection using Bayesian classifier” 2nd National Conference on
[Fig. 8 Detecting a Straight Path] Advancements in the Era of Multi Disciplinary Systems - AEMDS 2013,
Kurukshetra, Haryana, India. Proceedings in Elsevier : science &
Figure 8 shows how the command line issues a Technology, 2013.
prediction to continue straight along the path. [8] Madhu Bala Myneni, M. Seetha, “Comparative Analysis on Scene
Image Classification using Selected Hybrid Features”, International
Journal of computer application, 63(2), pp.44-47, 2013.
V. CONCLUSION AND FUTURE SCOPE [9] B Padmaja, P V Narasimha Rao, A Krishna Chaitanya, E Krishna Rao
This paper demonstrates the effective working of Patro, “IoT Based Surveillance of Agriculture and Smart Detection of
Infestation”, Journal of Advanced Research in Dynamical and Control
autonomous car by neural networking. A street sign Systems, 2018.
acknowledgment calculation has been executed on this stage [10] B Padmaja, V V Rama Prasad, K V N Sunitha, G Vineeth Reddy, “Deep
with promising outcomes. The framework execution is RNN Based Human Activity Recognition using LSTM Architecture on
enhanced significantly when contrasted with the current Smartphone Sensor Data”, Journal of Fundamental and Applied
framework. The framework can identify effectively all signs Sciences, 10(5S), pp 1102 – 1115, June 2018.
in test pictures with couple of most extreme positive results. [11] B Padmaja, V V Rama Prasad, KVN Sunitha, “TreeNet analysis
The proposed framework can work efficiently in breaking of human stress behavior using socio-mobile data”, Journal of Big Data,
Springer, 3(24), pp. 1-15, 2016.
down the sign at both sunlight and night enlightenments.
[12] B Padmaja, VV Rama Prasad, KVN Sunitha, “Connecting Productivity
Further make a speedy move identified with sign. Then with with Social Capital via Daily Mobile Phone Logs”, 5(2), pp. 62-74,
the assistance of sensors, the car able to be maximum keeps 2016.
away from crash.
This system is further developed by enhancing the
calculation by calculator figuring out how to do it. The
present calculation performs the tasks on every one of the
edges. It is exact yet its proficiency could be additionally
upgraded on the off chance that it begins learning by itself
and maintain a strategic distance from pointless figurines of
the areas which are as of now known or commonplace. Once
the auto begins voyaging on the streets, it decides the

978-1-5386-1442-6/18/$31.00 ©2018 IEEE 21