Keywords

1 Introduction

With an increasing number of vehicles, Intelligent Transportation System is developed and applied for better vehicle monitor and management. As an important part of Intelligent Transportation System, License Plate Recognition has many applications in our daily life, such as electronic toll collection, traffic law enforcement, park entrance management and so on.

License Plate Recognition system is usually composed of three parts: license plate extraction, character segmentation, character recognition. License plate extraction is the most important step in the LPR system. The result of extraction can directly impact the subsequent processes and the whole efficiency of the system. Many factors like partial occlusion, uneven lights, different angels, complex background may disturb the results of extraction.

Plentiful methods have been proposed by scholars for license plate extraction in the past years. Traditional license plate extraction methods are mainly based on color [1–3], edge [4–6] or texture [7–9].

Different from these traditional methods, methods using local features are applied into license plate extraction recently. In [10], a two-stage method is used to extract license plate. Several possible license plate regions are first extracted by AdaBoost classifier. In the second stage, a SVM classifier based on SIFT descriptors is used to filter false positives.

Traditional edge-based methods require the continuity of the edges in common. Our approach extracts windows with high edge magnitudes as possible license plate regions, which overcomes the shortcoming of discontinues license plate. To increase the accuracy of license plate extraction, we also add the step of verification. Instead of point-based feature for verification, we select region-based feature: T-HOG for better characterizing license plate.

Figure 1 shows the framework of our approach, which consists of two key components: vertical edge density detector and SVM filter. Initially, we will perform vertical edge detection on the gray image. Then we get some candidate license plate regions by analyzing the vertical edge density. All the detected windows are passed to the SVM classifier in the second stage to filter the non-character regions. Finally, we get the license plate if exists.

Fig. 1.
figure 1

System architecture

Full size image

The rest of the paper is organized as follows. In Sect. 2, the method of extracting candidate license plate windows is presented. We describe the SVM filter algorithm in Sect. 3. The experiment results are shown in Sect. 4. Section 5 concludes the paper.

2 Candidate License Plate Extraction

In this step, we aim at extracting areas with high vertical gradient density as candidate license plate regions.

2.1 Preprocessing

The input image is first converted into gray image. Then we apply illumination normalization to reduce the influence of the uneven light. An example image with 256 gray levels and its enhanced image are shown in Fig. 2.

Fig. 2.
figure 2

(a) Gray image (b) Enhanced image

Full size image

2.2 Vertical Edge Detection

The magnitude of the vertical edges on the license plate is considered a robust extraction feature, while using the horizontal edges only can result in errors due to car bumper. Therefore we select Sobel vertical operator to detect edges. After convolving the gray image with Sobel vertical operator, we get the vertical gradient map. Finally, we use the mean of the absolute gradient values as the threshold and apply non-maximum suppression to horizontal direction of the gradient image.

2.3 Background Curves and Noise Removing

We remove the long background edges and short random noise edges by the algorithm mentioned in [12]. This algorithm only requires us to scan the edge image for three times. The first scan will record the edge lengths away from the top (or left) start points. And the second scan will record the edge lengths away from the bottom (or right) end points. And the last scan will add up the two kinds of lengths to denote the actual edge lengths. If the edge points are too long or too short, then these points will be removed from the edge image. There are two thresholds of edge length: \( T_{l} \) and \( T_{s} \). \( T_{l} \) is related to the estimated height of the license plate, and \( T_{s} \) is shorter than most of the length of the plate edges. The vertical edge image and the edge image after removing the background edges and noise edges are shown respectively as Fig. 3.

Fig. 3.
figure 3

(a) Vertical edge image (b) The edge image after removing background and noise edges

Full size image

2.4 Candidate License Plate Region Location

In this step, we shift a rectangle window in the edge image processed after the last section, and record the number of the edge points in the window. Finally, we select the top three windows whose edge points are most as candidate license plate regions. The candidate regions need to be far away from each other in distance. When the background areas are edge densely, as a result, the true license plate is not selected among the candidate regions. In this case, we can change the parameter of the number of the candidate regions. In our test dataset, most of the license plates are included in these candidate regions. Figure 4 shows the result of candidate license plates extraction.

Fig. 4.
figure 4

Result of candidate license plates extraction

Full size image

The parameters of our license plate sizes are predefined, while plates in real scene are of different sizes, we can resize the input image to ensure the real plate detected. In order to find large/small plate, we need to apply the algorithm on a small/large size image.

3 License Plate Verification

As license plate is usually composed of several characters, we can validate the candidate license plate by examining whether the region is textual. The variation of license plate types and environments causes the challenge in plate text verification. Experience shows that point-based features like SIFT can’t represent the features of texts well in common. Instead of point-based feature for text verification, we use the region-based method: T-HOG descriptor, which is based on the multi-cell HOG.

3.1 Histograms of Oriented Gradients

The distribution of local intensity gradients or edge directions can be used to characterize local object appearance and shape well even without precise information of gradient or edge positions. In practice, we divide the image window into small spatial regions called ‘cells’, then accumulate a local 1-D histogram of gradient directions over the pixels of the cell. The entrances to the histogram form the representation. A block is made up of several cells. Finally, a contrast-normalization is applied to the descriptor block in order to be invariant to illumination. The normalized descriptor is named as the HOG descriptor. This descriptor is robust to local geometric and photometric transformations: translations or rotations when the change is smaller than the cell or orientation bin size.

For each sample in the candidate image, we firstly compute its gradient:

$$ \nabla I(x,y) = \frac{1}{2}(I(x + 1,y) - I(x - 1,y),I(x,y + 1) - I(x,y - 1)) $$
(1)

The gradient magnitude m(x, y) is just the gradient norm, and orientation ÆŸ(x, y) is expressed as an angle in the range 0 to 2Ï€.

Inspired by the R. Minetto and N. Thome [13] on sing line text description, we participate the candidate region into horizontal stripes. For each stripe, we accumulate these samples into an orientation histogram. The histogram has eight bins covering 360 degrees range of orientation. Each sample contributes to the histogram with its gradient magnitude as well as a Gaussian-weighted function. To avoid boundary effect in which the descriptor changes abruptly as one sample orientation shifts smoothly from one bin to another, we distribute the gradient into two nearest bins in a linear way. Similarly, to reduce the problem of the displacement of the sample from one cell to another, a Gaussian bell function is used to assign a weight to each sample point. The values of all the orientation histogram entries form the feature vector. Then the vector is normalized to reduce the effects of illumination change.

Instead of overlapping blocks and block weight functions in the HOG, T-HOG divides the image into horizontal stripes as cell grids, and applies overlapping cell weight functions to these cells.

3.2 False Positive Removal with SVM

This paper uses SVM to train the license plate classifier. SVM is a supervised learning method proposed in [14]. It has been widely used in two-class classification for its good performance. The core idea of SVM is to find an optical separating hyper-plane which maximizes the margin between the two classes. In the non-linear situation, a kernel function is used to map the input data to higher dimensions so that we can solve the problem in a linear way. The SVM classifier is defined as:

$$ f(x) = \sum\limits_{i = 1}^{N} {\alpha_{i} y_{i} K(x_{i} ,x) + b} $$
(2)

where \( K \) is the kernel function; \( x_{i} \) is the support vector, \( y_{i} \) is corresponding class-label; \( \alpha_{i} \) and \( b \) are the parameters got from the SVM training step.

4 Experiments and Analysis

We collected 3000 license plate images which were of little affine distortion as positive samples as well as another 3000 negative samples in the training step. To evaluate the performance of the proposed approach, we select 2000 images as our testing dataset. These images are taken from traffic intersection by using a camera in China. License plates in these images are in different sizes, colors and orientations and exhibit different conditions, such as varying illumination and background clutter.

In the second stage of license plate verification, we perform extensive experiments in order to determine the optical parameters for our T-HOG descriptor. Best result was obtained with L2 whole-descriptor normalization when the number of horizontal stripes is 9, the number of bins is 8.

We evaluate the performance of the proposed method with two widely used measurement criterions, recall and precision. Recall rate reflects the ability of a classifier to classify a labeled positive license plate region as positive license plate. Precision rate measures the proportion of the labeled positive license plates to the classified positives.

In our experiment, the vertical edge detector is adopted to detect the license plate with the recall rate 0.85 and the precision rate 0.28, however, we can get the recall rate 0.93 and the precision rate 0.87 after the candidate verification process. In the first stage, in order to detect as many license plates as possible, we locate three candidate license plate regions for each image. As a result, almost all license plates are contained in the candidate regions as well as many non-plate regions, which dues to a high recall rate and a low precision rate. The false positive is removed after the candidate verification processing, the precision rate is improved. The final results show that we can get both a high precision rate and a recall rate.

Our license plate detection result has been compared with other methods using p, r criteria for images of our dataset. Table 1 shows the comparison results. We can see that the proposed approach has a higher recall rate than method [6]. It can be seen from the results that the proposed method and method [15] perform better than method [10] in precision. This is mainly because the former are region-based, while the latter is point-based. The experiments verify the fact that the expression ability of point-based features is lower than the region-based features. Further more, we compare our method with method [15], our algorithm shows better performance in precision rate, which proves T-HOG is more distinct than HOG in license plate classification.

Table 1. Performance of difference algorithms
Full size table

Method [6] used canny detector in the edge detection, which was a time-consuming method. However in our work, Sobel operator was used, so it would reduce the run time greatly.

5 Conclusion and Future Work

In this paper, we have proposed a way to extract license plate using an edge detector and a SVM classifier with T-HOG descriptor to remove the false positives. Experiments show the efficiency of our approach. Since some parameters in the first stage are relative to the size of the candidate license plate, the results will be better when all the license plates in the dataset have similar sizes.

To extract the license plate information in the image, we also need to segment the verified license plate and recognize them, which will be done in our future work.