YOLO Vehicle Plate Recognition
YOLO Vehicle Plate Recognition
Abstract—Introducing the meaning of vehicle license plates has factors. In research conducted by Silva et al. [4], and research
a broad impact on various aspects of life, especially enforcing carried out by Laroca et al. [5], the initial step applied is
traffic regulations. One of the latest innovations is an electronic vehicle detection before proceeding with license plate detection.
ticketing system with government surveillance cameras. Each
license plate has a unique serial number based on county vehicle The results of this approach have been proven to enhance the
registration, allowing for accurate identification and tracking. The performance of ALPR.
license plate recognition system consists of three main stages: One proven practical approach is utilizing deep learning,
license plate detection, character segmentation, and segmented particularly Convolutional Neural Network (CNN) algorithms,
character recognition. The main challenge in this research is
to detect vehicles and vehicle license plates. In this context,
obtaining image data from irregular and dynamic environments,
which can significantly affect system accuracy. To overcome this, some detectors that have shown great performance include
the YOLO (You Only Look Once) method detects vehicle license YOLO [3]. To recognize characters on license plates, Optical
plates using the Rodosol-ALPR dataset. However, the Rodosol- Character Recognition (OCR) technology based on YOLO is
ALPR dataset does not have a bounding box and label information employed, which is a process where the system can identify
at the character level. Therefore, this study manually adds bound-
each character in the previously detected license plate image.
ing boxes and labels at the character level to facilitate a robust
segmentation approach. Experimental results show impressive
accuracy performance, reaching 93.925% for all types of vehicles, II. L ITERATURE R EVIEW
98.525% for cars, and 89.325% for motorbikes. These findings
show that this system competes with state-of-the-art methods based This section will present related research utilizing deep
on deep learning techniques. This increase in accuracy emphasizes learning approaches in ALPR. Below is a summary of studies
the effectiveness of using YOLO for license plate detection and related to vehicle license plate recognition.
the importance of detailed manual labelling for character-level
recognition, indicating potential for future development. A. Related Work
Index Terms—computer vision, license plate recognition, yolo
The study by Al-batat et al. utilizes YOLOv2 as the primary
I. I NTRODUCTION object detection method, considering its ability to detect objects
Traditional paper-based government administration processes quickly without sacrificing significant accuracy. To detect and
are being replaced by digitization, known as e-government, recognize number plates, this study adopted YOLOv4-tiny,
for improved services and administrative efficiency. Automatic the latest version available when the study was conducted.
License Plate Recognition (ALPR) assists in various tasks This research incorporates data augmentation at each stage
like parking management, road traffic control, and Electronic to improve performance, using augmentation techniques such
Traffic Law Enforcement (ETLE), allowing law enforcement as adding shadows, light blobs, and blur. The results of this
without direct officer-driver interaction [1]. ETLE implementa- study show the following accuracy levels: Caltech cars: 97.1%;
tion boosts efficiency in police administration, prevents bribery English LP: 95.5%; AOLP: 98.0%; Open ALPR EU: 98.7%;
cases involving police personnel [2], and is supported by UFPR ALPR: 73.3% [7].
ALPR systems for driver identification through license plate The study by Lee et al. utilizes YOLOv3 to detect number
recognition. plates using only two data sets, AOLP-RP and VTLP, with
ALPR is a system that utilizes cameras and software, em- an accuracy of 99.18% and 93.08%, respectively. This research
ploying image processing techniques and artificial intelligence does not recognize the vehicle first, so it immediately starts with
to automatically identify vehicle license plates from images or the number plate recognition stage. In the AOLP-RP dataset,
videos [3]. Typically, ALPR systems undergo three stages in images contain close-up images of license plates, eliminating
recognizing license plates: detecting license plates from vehi- the need for vehicle detection [8].
cles, segmenting each character, and recognizing each character. The study conducted by Quang et al. license plate recognition
In the ALPR system, several obstacles can lead to failures has two stages: number plate detection and character recog-
in license plate recognition. This is due to specific conditions nition. This research used YOLOv5 and was tested with the
such as poor image quality, suboptimal camera angles, inade- UFPR dataset, which consists of time series data. It achieved
quate lighting, interference from surrounding objects, and other an accuracy of 96.7%. Therefore, this research approaches time
Authorized licensed use limited to: UNIVERSIDADE FEDERAL DO AMAZONAS. Downloaded on December 05,2024 at 17:55:19 UTC from IEEE Xplore. Restrictions apply.
979-8-3503-6535-1/24/$31.00 ©2024 IEEE 397
2024 International Conference on Data Science and Its Applications (ICoDSA)
Authorized licensed use limited to: UNIVERSIDADE FEDERAL DO AMAZONAS. Downloaded on December 05,2024 at 17:55:19 UTC from IEEE Xplore. Restrictions apply.
398
2024 International Conference on Data Science and Its Applications (ICoDSA)
to reduce the number of objects identified as license plates, formats, including both single-line and two-line formats. The
allowing for more precise and efficient focus in license plate dataset can be seen in Fig. 2.
recognition and analysis.
Vehicle license plates typically have a rectangular shape
in various countries, although their colours may vary. By
leveraging the vehicle detection process, identifying vehicle
license plates becomes easier, as their distinctive rectangular
shape can be recognized more effectively.
After the detection process is completed, the license plate
needs to be recognized. The OCR process is a crucial step
where the system can identify characters in the license plate
image that have been successfully detected beforehand.
Fig. 2. Rodosol-ALPR dataset.
C. You Only Look Once (YOLO)
YOLO is an algorithm in deep learning used for object The Rodosol-ALPR dataset consists of 20,000 images cap-
detection in images. The image is divided into evenly spaced tured at fixed camera angles in both daytime and morning. The
grid cells along the X and Y axes in the YOLO network. Each images are taken at a resolution of 1280 x 720 pixels. Among
grid cell generates several candidate bounding boxes distributed these 20,000 images, there are 5,000 images of each car and
evenly within the cell. These bounding boxes play a crucial motorcycle with Brazilian license plates and 5,000 images of
role in detecting objects and predicting the confidence level each car and motorcycle with Mercosur license plates.
associated with the presence of objects in each bounding box There is a difference in format between the Mercosur license
[12]. plate and the old license plate format. Both formats have seven
The confidence score is a crucial metric for assessing object characters, with the first three characters being letters. But there
detection accuracy. It is derived from the Intersection over is a difference in the fifth character. The fifth character is a letter
Union (IoU) metric, which measures the extent of overlap in the Mercosur license plate format, while it is a number in
between the predicted bounding box and the reference bounding the old license plate format.
box. Both bounding boxes contain predictions for key param- Acknowledging that the Rodosol-ALPR dataset lacks bound-
eters such as position (x, y), width (w), and height (h). The ing box annotations for characters on vehicle license plates
confidence score quantifies the degree of agreement between is imperative. Consequently, developing a robust license plate
these predictions and the ground truth, indicating how well the recognition model requires manually add bounding box and
predicted bounding box aligns with the actual object’s position label for each character.
and size. A higher confidence score suggests a more accurate
detection, signifying a substantial overlap between the predicted B. Research Methodology
and reference bounding boxes [13]. Based on Fig. 3, there are five stages of the process in the
YOLOv8 is similar to the YOLOv5 architecture but with system to be developed in the research. This dataset underwent
some changes. One of the changes is the addition of two a data split process using the traditional split method with the
additional convolutions in the CSPLayer, now called C2f. These following composition: 40% for training, 20% for validation,
additional convolutions combine detailed information from the and 40% for testing. The dataset owner has provided this
image with its global context, thus enhancing object detection configuration so that research results can be compared with
accuracy. In testing the MS COCO test-dev 2017 dataset, those of other studies.
YOLOv8x achieved an AP of 53.9% with an image size of One notable drawback of the method is that it requires
640 pixels. This figure is higher than the mAP of YOLOv5 at the manual addition of bounding boxes and character-level
the same input size, which is 50.7%. YOLOv8x can also detect labels for both the training and validation datasets, which can
objects faster, achieving 280 frames per second on NVIDIA be a time-consuming and labor-intensive process, potentially
A100 and TensorRT [14]. limiting the scalability and efficiency of the system’s imple-
mentation.
III. M ETHOD AND M ATERIAL The detector was not built from scratch in this research be-
In this section, the system design used for vehicle license cause vehicle detection is already common practice. Therefore,
plate recognition will be outlined. Additionally, we will review this research uses YOLOv8 with transfer learning from the
the dataset used in this research to train and test the proposed Common Objects in Context (COCO) dataset. In the vehicle
system. A comprehensive understanding of the system design, detector, filter the class name to detect only cars, motorcycles,
dataset, evaluation metrics, and testing scenarios will be pro- buses, and trucks.
vided. Vehicle license plate detection uses vehicle detection results,
isolating car and motorbike objects from the image. During
A. Dataset the license plate detection training stage, data augmentation is
In this study, the authors utilized the Rodosol-ALPR dataset carried out by cropping, blurring, and rotation to enrich the
[15]. This dataset consists of images with varying positions variety and diversity of the data. The results of vehicle detection
and resolutions, and there are differences in the license plate have varying image sizes, so a resizing process is carried out
Authorized licensed use limited to: UNIVERSIDADE FEDERAL DO AMAZONAS. Downloaded on December 05,2024 at 17:55:19 UTC from IEEE Xplore. Restrictions apply.
399
2024 International Conference on Data Science and Its Applications (ICoDSA)
Authorized licensed use limited to: UNIVERSIDADE FEDERAL DO AMAZONAS. Downloaded on December 05,2024 at 17:55:19 UTC from IEEE Xplore. Restrictions apply.
400
2024 International Conference on Data Science and Its Applications (ICoDSA)
several detection results, demonstrating the model’s ability to Another challenge is the position of the motorbike license
detect vehicle license plates under various conditions. plates that tend to tilt. This tilt often causes the license plate
image must be cropped so that the characters cannot be fully
recognized. As a result, the number plate fails to be produced.
The following are several examples of detection and recognition
results produced on cars and motorbikes, shown in Fig. 5.
Authorized licensed use limited to: UNIVERSIDADE FEDERAL DO AMAZONAS. Downloaded on December 05,2024 at 17:55:19 UTC from IEEE Xplore. Restrictions apply.
401
2024 International Conference on Data Science and Its Applications (ICoDSA)
Additionally, exploring additional license plate datasets, partic- [12] C. Liu, Y. Tao, J. Liang, K. Li, and Y. Chen, “Ob-
ularly those featuring dynamic formats, will expand the model’s ject detection based on yolo network,” in 2018 IEEE
adaptability to variations in the number of characters on each 4th information technology and mechatronics engineering
license plate, presenting unique challenges that require inno- conference (ITOEC), pp. 799–803, IEEE, 2018.
vative solutions. Incorporating such datasets into our research [13] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You
will enrich the model’s training data and expand its application only look once: Unified, real-time object detection,” in
scope to meet a wider range of real-world scenarios. Proceedings of the IEEE conference on computer vision
and pattern recognition, pp. 779–788, 2016.
R EFERENCES [14] J. Terven, D.-M. Córdova-Esparza, and J.-A. Romero-
[1] F. A. Abdullah and F. Windiyastuti, “Electronic traffic González, “A comprehensive review of yolo architectures
law enforcement (etle) sebagai digitalisasi proses tilang,” in computer vision: From yolov1 to yolov8 and yolo-
Jurnal Kewarganegaraan, vol. 6, no. 2, pp. 3004–3008, nas,” Machine Learning and Knowledge Extraction, vol. 5,
2022. no. 4, pp. 1680–1716, 2023.
[2] S. Aditya and A. Safriani, “Problematika penerapan tilang [15] R. Laroca., E. V. Cardoso., D. R. Lucio., V. Estevam.,
elektronik dalam mengurangi pelanggaran lalu lintas,” and D. Menotti., “On the cross-dataset generalization in
Alauddin Law Development Journal, vol. 2, no. 1, pp. 74– license plate recognition,” in Proceedings of the 17th
80, 2020. International Joint Conference on Computer Vision, Imag-
[3] J. Shashirangana, H. Padmasiri, D. Meedeniya, and C. Per- ing and Computer Graphics Theory and Applications
era, “Automated license plate recognition: a survey on (VISIGRAPP 2022) - Volume 5: VISAPP, pp. 166–178,
methods and techniques,” IEEE Access, vol. 9, pp. 11203– INSTICC, SciTePress, 2022.
11225, 2020. [16] R. Laroca, A. B. Araujo, L. A. Zanlorensi, E. C.
[4] S. M. Silva and C. R. Jung, “License plate detection and De Almeida, and D. Menotti, “Towards image-based auto-
recognition in unconstrained scenarios,” in Proceedings matic meter reading in unconstrained scenarios: A robust
of the European conference on computer vision (ECCV), and efficient approach,” Ieee Access, vol. 9, pp. 67569–
pp. 580–596, 2018. 67584, 2021.
[5] R. Laroca, L. A. Zanlorensi, G. R. Gonçalves, E. Todt, [17] R. Atienza, “Vision transformer for fast and efficient scene
W. R. Schwartz, and D. Menotti, “An efficient and layout- text recognition,” in Document Analysis and Recognition –
independent automatic license plate recognition system ICDAR 2021 (J. Lladós, D. Lopresti, and S. Uchida, eds.),
based on the yolo detector,” IET Intelligent Transport (Cham), pp. 319–334, Springer International Publishing,
Systems, vol. 15, no. 4, pp. 483–503, 2021. 2021.
[6] R. Laroca, L. A. Zanlorensi, V. Estevam, R. Minetto,
and D. Menotti, “Leveraging model fusion for improved
license plate recognition,” in Progress in Pattern Recogni-
tion, Image Analysis, Computer Vision, and Applications
(V. Vasconcelos, I. Domingues, and S. Paredes, eds.),
(Cham), pp. 60–75, Springer Nature Switzerland, 2024.
[7] R. Al-Batat, A. Angelopoulou, S. Premkumar, J. Hemanth,
and E. Kapetanios, “An end-to-end automated license
plate recognition system using yolo based vehicle and li-
cense plate detection with vehicle classification,” Sensors,
vol. 22, no. 23, p. 9477, 2022.
[8] Y. Lee, J. Lee, H. Ahn, and M. Jeon, “Snider: Single noisy
image denoising and rectification for improving license
plate recognition,” in Proceedings of the IEEE/CVF In-
ternational Conference on Computer Vision Workshops,
pp. 0–0, 2019.
[9] H. C. Quang, T. Do Thanh, and C. T. Van, “Character
time-series matching for robust license plate recognition,”
in 2022 International Conference on Multimedia Analysis
and Pattern Recognition (MAPR), pp. 1–6, IEEE, 2022.
[10] S. M. Silva and C. R. Jung, “A flexible approach for
automatic license plate recognition in unconstrained sce-
narios,” IEEE Transactions on Intelligent Transportation
Systems, vol. 23, no. 6, pp. 5693–5703, 2021.
[11] S. M. Silva and C. R. Jung, “Real-time license plate
detection and recognition using deep convolutional neural
networks,” Journal of Visual Communication and Image
Representation, vol. 71, p. 102773, 2020.
Authorized licensed use limited to: UNIVERSIDADE FEDERAL DO AMAZONAS. Downloaded on December 05,2024 at 17:55:19 UTC from IEEE Xplore. Restrictions apply.
402