Illegal Parking notification system using Image Feature extraction

Method

Abstract:

Parking areas have become overstressed due to the growing numbers of

vehicles on the roads today. The Automatic Number Plate Recognition System
(ANPR) plays an important role in addressing these issues as its application ranges
from parking admission to monitoring urban traffic and to tracking automobile
thefts. There are numerous ANPR systems available today which are based on
different methodologies. In this paper, we attempt to review the various techniques
and their usage. The ANPR system has been implemented using template
Matching and its accuracy was found to be 80.8% for Indian number plates. We
propose an advanced Automatic number-plate recognition (ANPR) system, which
not only recognizes the number and the issuing state, but also the type and location
of the vehicle in the input image. The system is based on a combination of existing
methods, modifications to neural network architectures and improvements in the
training process. The proposed system uses machine-learning approach and
consists of three main parts: segmentation of input image by Fully Convolutional
Network for localization of license plate and determination of vehicle type;
recognition of the characters of the localized plate by a Maxout CNN and LSTM;
determination of the state that has issued the license plate by a CNN. The training
of these neural network models is accomplished using a manually labeled custom
dataset, which is expanded with data augmented techniques. The resulting system
is capable of localizing and classifying multiple types of vehicles (including
motorcycles and emergency vehicles) as well as their license plates. The achieved
precision of the localization is 99.5%. The whole number recognition accuracy is
96.7% and character level recognition accuracy is 98.8%. The determination of
issuing state is precise in 92.8% cases.

INTRODUCTION

The continuing growth of world population leads to more vehicles on the road. The
increasing road traffic intensity reinforces the need for automation of traffic
monitoring systems. Such automatic systems include identification of vehicles by
accurate recognition of number plates. If the license plate recognition is used to
detect traffic offenders, it is useful to simultaneously detect the type of the vehicle,
since the traffic regulations might be different for light vehicles, trucks, and
emergency vehicles, such as ambulance and police cars. The number plates also
include the information about the issuing country, which may be detected by a
versatile monitoring system. Recent achievements in the computer vision field
indicate that recognition of number plates, detection of issuing country, and
detection of vehicle type can be addressed noninvasively by the use of cameras,
eliminating the need for other sensors. A survey on classical computer vision
approaches in ANPR systems. However, in the last few years, the field of
computer vision has experienced significant advancements thanks to a combination
of big data, an increasing availability of computation power, and deep learning
methods. These advances allow one to create more accurate camera-based object
recognition systems than was possible only a few years ago. However, in order to
compare the results, existing state-of-the-art deep learning methods are usually
trained and compared on a widely known and general datasets, such as PASCAL
VOC, ImageNet, and MS COCO. Therefore, a successful use of deep learning on
specific real world problems, such as license plate localization, recognition, and
detection of the issuing state, requires additional research, which is offered in the
current paper. The paper first introduces the related work and then proposes the
system with these main parts: The localization of vehicles and license plates, which
is accomplished by Fully Convolutional Network (FCN). Fast training of
localization and classification network was achieved by the application of transfer
learning. For the recognition, the localized license plate is divided into smaller
regions using sliding window approach and those regions are processed by a
character reader neural network consisting of customized Feature Extraction
Maxout CNN and an Long Short-Term Memory (LSTM) network. Finally, the
recognition of the issuing state is accomplished by a classifying Convolutional
Neural Network (CNN), which gives better result than conventional regular
expression (regex) approach. System implementation, training, and testing were
done using Python language and Tensorflow framework. The base training data
was acquired from real life radar cameras angled downwards. In order to acquire
an even larger labeled data set to improve the robustness of supervised machine
learning and resulting localization and recognition system, a custom image
augmentation system was developed for generating additional training images,
adjusting the lighting, rotation, zoom levels, blurring and noise levels. All parts of
the overall system were tested on the real-world examples. The tests suggested,
that proposed system is accurate and versatile ANPR system. An end-to-end
inference for a single image takes 0.2 s on a GPU and 8 s on a single core CPU.

Literature Survey:

1. Topic: Standard Definition ANPR System on FPGA and an Approach to

Extend it to HD

Author: Xiaojun Zhai, Faycal Bensaali,

Abstract: Automatic Number Plate Recognition (ANPR) system becomes an

important research topic in Intelligent Transportation systems (ITS). More
recently, high-definition (HD) cameras are used for providing better performance
in ANPR system. However, most known approaches for standard definition (SD)
number plate localisation (NPL) are not suitable for real-time HD image
processing as the real-time requirement cannot be met due to the computationally
intensive cost of localising the number plate. In this paper, a solution to link
previously designed architectures for NPL, character segmentation and character
recognition in a SD ANPR system is first described. The system is to be
implemented on a single stand-alone FPGA-based processing unit. An approach to
extend the SD ANPR system to HD ANPR system without significantly increasing
the computational cost is then introduced.

Disadvantages:

Its not Portable for mobile environment.

There is no notification system

2. Topic: Number Plate Recognition for use in different countries using an

improved segmentation

Author: A Roy and D.P Ghoshal,

Abstract: Automatic Number Plate Recognition (ANPR) is a real time embedded

system which identifies the characters directly from the image of the license plate.
It is an active area of research. ANPR systems are very useful to the law
enforcement agencies as the need for Radio Frequency Identification tags and
similar equipments are minimized. Since number plate guidelines are not strictly
practiced everywhere, it often becomes difficult to correctly identify the non-
standard number plate characters. In this paper we try to address this problem of
ANPR by using a pixel based segmentation algorithm of the alphanumeric
characters in the license plate. The non-adherence of the system to any particular
country-specific standard & fonts effectively means that this system can be used in
many different countries – a feature which can be especially useful for trans-border
traffic e.g. use in country borders etc. Additionally, there is an option available to
the end-user for retraining the Artificial Neural Network (ANN) by building a new
sample font database. This can improve the system performance and make the
system more efficient by taking relevant samples. The system was tested on 150
different number plates from various countries and an accuracy of 91.59% has
been reached.

Disadvantages:

The entire system was designed on Matlab (Version R2008a) platform but
for real time implementation this needs to be developed in C or any similar IDE
specific to the hardware used. This would also lower the cost of the system as the
C compiler is cheaper than Matlab. As well as its not suitable for mobile
environment.

3. Topic: Vehicle Recognition Using Contourlet Transform and SVM

Author: Saeid Rahati, Reihaneh Morvejian, Ehsan M. Kazemi and Farhad M.

Kazem,

Abstract: This paper proposes the performance of a new algorithm for vehicles
recognition system. This recognition system is based on extracted features on the
performance of image’s curvelet transform & achieving standard deviation of
curvelet coefficients matrix in different scales & various orientations. The curvelet
transform is a multiscale transform with frame elements indexed by location, scale
and orientation parameters, and have time-frequency localization properties of
wavelets but also shows a very high degree of directionality and anisotropy. This
paper presents the application of three different types of classifiers to the vehicle
recognition. They include of Support vector machine (one versus one), k nearest-
neighbor and Support vector machine (one versus all). In addition, the proposed
recognition system is obtained by using different scales information as feature
vector. So, we could clarify the most important scales in aspect of having useful
information. The performed numerical experiments for vehicles recognition have
shown the superiority of curvelet and standard deviation preprocessing, which are
associated with the Support vector machine structure (one versus one). The results
of this test show, the right recognition rate of vehicle’s model in this recognition
system, at the time of using total scales information numbers 3&4 curvelet
coefficients matrix is about 99%. We’ve gathered a data set that includes of 300
images from 5 different classes of vehicles. These 5 classes of vehicles include of:
PEUGEOT 206, PEUGEOT 405, Pride, RENAULT55 and Peykan. We’ve
examined 230 pictures as our train data set and 70 pictures as our test data set.

Disadvantages:

This is not suitable for real time implementation.

Its requires high computation cost and high storage space.

4. Topic: Road Vehicle Recognition in Monocular Images

Author: M.A. Sotelo , J. Nuevo , L.M. Bergasa and M. Ocana

Abstract: This paper describes a monocular vision-based Vehicle Recognition

System in which the basic components of road vehicles are first located in the
image and then combined with a SVM-based classifier. The challenge is to use a
single camera as input. This poses the problem of vehicle detection and recognition
in real, cluttered road images. A distributed learning approach is proposed in order
to better deal with vehicle variability, illumination conditions, partial occlusions
and rotations. The vehicle searching area in the image is constrained to the limits
of the lanes, which are determined by the road lane markings. By doing so, the rate
of false positive detections is largely decreased. A large database containing
thousands of vehicle examples extracted from real road images has been created
for learning purposes. We present and discuss the results achieved up to date.

Disadvantages:

There is no notification system .

Detection accuracy also very less.

5. Topic: OCR Based Neural Network for ANPR

Author: Xiaojun Zhai, Faycal Bensaali and Reza Sotudeh

Abstract: Optical Character Recognition (OCR) is the last stage in an Automatic

Number Plate Recognition System (ANPRs). In this stage the number plate
characters on the number plate image are converted into encoded texts. In this
paper, an Artificial Neural Network (ANN) based OCR algorithm for ANPR
application is presented. A database of 3700 UK binary character images have
been used for testing the performance of the proposed algorithm. Results achieved
have shown that the proposed algorithm can meet the real-time requirement of an
ANPR system and can averagely process a character image in 8.4ms with 97.3%
successful recognition rate.

Disadvantages:

Detection accuracy is high but there no support for the retrieving geo location of
cars.
Problem Statements:

Low accuracy of detecting the car numbers

It requires lots of training data

There is no accuracy to locate the car

Motivation:

The growing affluence of urban India has made the ownership of vehicles a
necessity. This has resulted in an unexpected civic problem - that of traffic control
and vehicle identification. Parking areas have become overstressed due to the
growing numbers of vehicles on the roads today. The Automatic Number Plate
Recognition System (ANPR) plays an important role in addressing these issues as
its application ranges from parking admission to monitoring urban traffic and to
tracking automobile thefts.

Existing System:

Computer vision and character recognition, algorithms for license plate recognition
play an important role in video analysis of the number plate image. Therefore they
form the core modules in any ANPR system. The system for automatic car license
plate recognition includes a camera, a frame grabber, a computer, and custom
designed software for image processing, analysis and recognition. Vehicle
identification has been an active research for over the last few years. A number of
researches have been carried out to identify the type of vehicle such as a car, truck,
scooter or motorcycle. In , Soble filter was used to address this issue to find the
edges of the vehicle which in turn is applied to recognize the type of vehicle. The
Contourlet Transform and Support Vector Machine (SVM) were used in to find out
the model of the vehicle. They showed numerical results on data set of about 70
pictures. However, they did not apply the technique to real-time video stream. In
monocular images are used for vehicle recognition. They applied canny edge
detection to detect the presence of vehicle and SVM to recognize the vehicle. In,
Maximum Average Correlation Height (MACH) filter and Log r-theta Mapping
techniques were applied to recognize the type of vehicle irrespective of scale and
rotation variation of vehicles. The MACH filter was used for detection of targets in
cluttered environment. In, MACH was used to filter recognize the target to
orientation invariance and they used log r-theta mapping to create in plane rotation
and scale invariance while recognition.

In, Optical Character Recognition (OCR) technique was used, which is a

widely used technology which translates scanned images of printed text into
machine encoded text. Here, an OCR algorithm based on feed-forward neural
network is being proposed where two non-overlapping real character image data
sets are used for training and testing the proposed neural network. The two non-
overlapping image data sets were used to emulate real-world scenarios where the
neural network will be subjected to. Artificial Neural Networks (ANN) are widely
used intelligent computing architecture for pattern recognition. The most common
used ANN is the multilayer feed-forward neural network which has a simple
internal architecture that can classify inputs into a set of target categories.
Typically, the works done in and use features extraction and binary pixels value
to organize the inputs of neural network respectively, the former one is the most
common used method for neural network, which can achieve good performance
even under difficult environment. However, the feature extraction normally needs
complex computation or multiple stages to extract features. Similar methods are
present which use extra procedures during the training stage or after obtaining the
results of neural network to handle difficult characters that belong to the sets of
ambiguous characters. Additional training is used for the difficult characters (e.g.
I/1, B/8 and O/D) and in comparison of distinguishing parts of ambiguous
characters is performed. The statistical classifiers can be divided into two sub-
classes: single stage classifier and multistage classifier. In the work presented in ,
character features are extracted from the elastic mesh, and the entire address
character string is taken as the object of study. This was tested using Japanese
Number Plates and the Support Vector Machine (SVM) integration uses the
features to recognize numbers, Kana (Japanese script), and the strings of characters
that represent the area. The recognition rates for numbers, Kana and strings of
characters are 99.5%, 98.6% and 97.8% respectively. In, a two-stage hybrid OCR
system is presented to improve the recognition rate. It firstly uses four statistical
sub classifiers to independently recognize the input character and then the results
are combined using the Bayes' method .Secondly, if the recognized character from
the first stage belong to the sets of ambiguous characters (e.g. I/1, B/8 and O/D), a
structural stage is used for a further decision Common pattern matching technique
is a simple technique for the recognition of single font and fixed size character,
which is a suitable approach for ANPR systems. Incorrectly segmented characters
from the character segmentation stage, where characters are not in the expected
position or few of them are missed, may affect the OCR recognition. The neural
networks and statistical classifiers, which give better outcome compare to common
pattern matching technique, can overcome this problem because of their strong
memory and self-adapting ability. However, in order to achieve good performance,
large amount of samples and neurons are needed to obtain the neural networks. In ,
MATLAB has been used for the implementation of the algorithm on a PC
equipped with a Dual Core 2.4GHz and 3G RAM. It has also been used to generate
the weights of neural network. 6436 binary images with varying resolutions from
the previous character segmentation stage were used. First of all, the binary images
of the characters are resized to the same size. To choose the right size, several sizes
of input images have been used for neural network training. High recognition rates
can be achieved by using large character images but this will result in a more
complex structure of the neural network as the number of weights will increase.
The size corresponding to the best suitable result is used for the final neural
network. Each system proposed for vehicle identification and number plate
recognition in the literature survey has its own pros and cons.

2. SYSTEM STUDY

2.1 FEASIBILITY STUDY

The feasibility of the project is analyzed in this phase and business

proposal is put forth with a very general plan for the project and some cost

estimates. During system analysis the feasibility study of the proposed system is to

be carried out. This is to ensure that the proposed system is not a burden to the

company. For feasibility analysis, some understanding of the major requirements

for the system is essential.

Three key considerations involved in the feasibility analysis are

  ECONOMICAL FEASIBILITY

 TECHNICAL FEASIBILITY

 SOCIAL FEASIBILITY

ECONOMICAL FEASIBILITY

This study is carried out to check the economic impact that the system will

have on the organization. The amount of fund that the company can pour into the

research and development of the system is limited. The expenditures must be

justified. Thus the developed system as well within the budget and this was

achieved because most of the technologies used are freely available. Only the

customized products had to be purchased.

TECHNICAL FEASIBILITY

This study is carried out to check the technical feasibility, that is, the

technical requirements of the system. Any system developed must not have a

high demand on the available technical resources. This will lead to high demands

on the available technical resources. This will lead to high demands being placed

on the client. The developed system must have a modest requirement, as only

minimal or null changes are required for implementing this system.

SOCIAL FEASIBILITY

The aspect of study is to check the level of acceptance of the system by the

user. This includes the process of training the user to use the system efficiently.

The user must not feel threatened by the system, instead must accept it as a

necessity. The level of acceptance by the users solely depends on the methods

that are employed to educate the user about the system and to make him familiar
with it. His level of confidence must be raised so that he is also able to make some

constructive criticism, which is welcomed, as he is the final user of the system.

THE PROPOSED SYSTEM

The system consists of three trainable parts: (1) segmentation of the input
image, which simultaneously determines the type of the vehicle and the location of
the license plate; (2) recognition of the number plate; (3) determination of the state
that has issued the localized number plate. The localization of the number plate, as
well as the localization of vehicle and determination of the vehicle class, is carried
out by a Fully Convolutional Network.
 Identify
owners mobile Data base of
Car Details
number and
Location

Send Notification
sms to owner
mobile
CONCLUSION AND FUTURE WORK In this work, existing methodologies
and algorithms proposed in literature for Vehicle and Number Plate recognition
were reviewed. Due to the unavailability of such an ANPR system off the shelf in
tune with our requirements, it is our endeavor to customize an ANPR system for
educational institution. Template matching was implemented on number plates
obtained from static images and an average accuracy of 80.8% was obtained. This
accuracy can be improved greatly by positioning the camera suitably to capture the
best frame and using two layers of neural networks. The implementation of the
proposed system can be extended for the recognition of number plates of multiple
vehicles in a single image frame by using multi-level genetic algorithms.

Work Plan of Action:

Duration Description of the Work

2 weeks
Studying about Literature
survey

1week
Analyzing Problems and
Problem formation

3 weeks
Designing and
implementation
 2 weeks
Testing

Bibiliography:

[1] Xiaojun Zhai, Faycal Bensaali, “Standard Definition ANPR System

on FPGA and an Approach to Extend it to HD” in 2013 IEEE GCC
Conference and exhibition, November 17-20, Doha, Qatar. pp.214

[2] H. Erdinc Kocer and K. Kursat Cevik, "Artificial neural networks

based vehicle license plate recognition," Procedia Computer Science,
vol. 3, pp. 1033-1037, 2011
[3] A Roy and D.P Ghoshal, "Number Plate Recognition for use in
different countries using an improved segmentation," in 2nd National
Conference on Emerging Trends and Applications in Computer
Science(NCETACS), 2011, pp. 1-5

[4] Fikriye Öztürk and Figens Özen, "A New License Plate Recognition
System Based on Probabilistic NeuralNetworks," Procedia Technology,
vol. 1, pp. 124128,2012

[5] Anton Satria Prabuwono and Ariff Idris, "A Study of Car Park
Control System Using Optical Character Recognition," in International
Conference on Computer and Electrical Engineering, 2008, pp. 866-870

[6] Ch. Jaya Lakshmi, Dr. A. Jhansi Rani, Dr. K. Sri Ramakrishna, and
M. Kanti Kiran, "A Novel Approach for Indian License Recognition
System," International Journal of Advanced Engineering Sciences and
Technologies, vol. 6, no. 1, pp. 10-14, 2011

[7] Jianbin Jiao, Qixiang Ye, and Qingming Huang, "A configurable
method for multi-style license platerecognition," Pattern Recognition,
vol. 42, no. 3, pp. 358-369, 2009

[8] Zhigang Zhang and Cong Wang, "The Research of Vehicle Plate
Recognition Technical Based on BP Neural Network," AASRI Procedia,
vol. 1, pp. 74-81, 2012
[9] Ying Wen, "An Algorithm for License Plate recognition Applied to
Intelligent Transportation System", IEEE Transactions of Intelligent
Transportation Systems. pp. 1-16, 2011

[10] Chirag Patel, Dipti Shah, Atul Patel," Automatic Number Plate
Recognition System (ANPR): A Survey", International Journal of
Computer Applications,

[11] Yang Yang, Xuhui Gao, and Guowei Yang, "Study the Method of
Vehicle License Locating Based on Color Segmentation," Procedia
Engineering, vol. 15, pp. 13241329, 2011

[12] SaimaRafique, Mahboob Iqbal and Hafiz Adnan Habib, “Space

Invariant Vehicle Recognition for Toll Plaza Monitoring and Auditing
System”, Multitopic Conference, 2009. INMIC 2009,IEEE 13th
International, pp. 1-6

[13] Fajas F., Farhan Yousuf, Remya P. R., Adarsh P. Pavanan, Sajan
Ambadiyil and Varsha Swaminathan, ”Automatic Number Plate
Recognition for Indian Standard Number Plates”, Ultra Modern
Telecommunications and Control Systems and Workshops (ICUMT),
2012 4th International Congress, pp. 1026-1028

[14] Weihua Wang, "Reach on Sobel Operator for Vehicle Recognition,

"International Joint Conference on Artificial Intelligence, pp.448-451,
2009
[15] Saeid Rahati, Reihaneh Morvejian, Ehsan M. Kazemi and Farhad
M. Kazem “Vehicle Recognition Using Contourlet Transform and
SVM,” Proceedings of the Fifth International Conference on Information
Technology, 2008

[16] M.A. Sotelo , J. Nuevo , L.M. Bergasa and M. Ocana, “Road

Vehicle Recognition in Monocular Images,” IEEE Symposium on
Industrial Electronics, 2005

[17] Bone P, Young R, Chatwin C. “Position, rotation, scale, and

orientation-invariant multiple object recognition from cluttered scenes,”
Opt Eng2006; 45:077203

[18] Xiaojun Zhai, Faycal Bensaali and Reza Sotudeh, “OCRBased

Neural Network for ANPR” in IEEE, 2012. Pp1

[19] Y. Amit, D. Geman, and X. Fan, “A coarse-to-fine strategy

formulticlass shape detection,” IEEE Transactions on Pattern Analysis
and Machine Intelligence, vol. 26, pp. 1606-1621, 2004

[20] C. Oz, and F. Ercal, “A Practical License Plate Recognition System

for Real-Time Environments. Computational Intelligence and Bio-
inspired System,” Lecture Notes in Computer Science, vol. 3512/2005,
pp.497-538, 2005
SYSTEM CONFIGURATION:

HARDWARE CONFIGURATION:

 Processor - Intel i3
 Speed - 2.4 Ghz
 RAM - 4GB (min)
 Hard Disk - 200GB
 Keyboard - Standard Keyboard
 Mouse - Two or Three Button Mouse
 Monitor - LCD/LED Monitor

SOFTWARE CONFIGURATION:

 Operating System - Windows 7

 Programming Language - Java/J2EE
 Software Version - android 7.0
 Back end - mysql