algorithms

Review
Object Detection in Autonomous Vehicles under Adverse
Weather: A Review of Traditional and Deep
Learning Approaches
Noor Ul Ain Tahir 1,† , Zuping Zhang 1, *,† , Muhammad Asim 2,3, *,† , Junhong Chen 3,4,†
and Mohammed ELAffendi 2

1 School of Computer Science and Engineering, Central South University, Changsha 410083, China;
214718021@csu.edu.cn
2 EIAS Data Science and Blockchain Laboratory, College of Computer and Information Sciences, Prince Sultan
University, Riyadh 11586, Saudi Arabia; affendi@psu.edu.sa
3 School of Computer Science and Technology, Guangdong University of Technology,
Guangzhou 510006, China; junhong.chen@uhasselt.be
4 Expertise Centre for Digital Media, Flanders Make, Hasselt University, 3500 Hasselt, Belgium
* Correspondence: zpzhang@csu.edu.cn (Z.Z.); asimpk@gdut.edu.cn or masim@psu.edu.sa (M.A.)
† These authors contributed equally to this work.

Abstract: Enhancing the environmental perception of autonomous vehicles (AVs) in intelligent

transportation systems requires computer vision technology to be effective in detecting objects and
obstacles, particularly in adverse weather conditions. Adverse weather circumstances present serious
difficulties for object-detecting systems, which are essential to contemporary safety procedures, infras-
tructure for monitoring, and intelligent transportation. AVs primarily depend on image processing
algorithms that utilize a wide range of onboard visual sensors for guidance and decisionmaking.
Ensuring the consistent identification of critical elements such as vehicles, pedestrians, and road lanes,
even in adverse weather, is a paramount objective. This paper not only provides a comprehensive
review of the literature on object detection (OD) under adverse weather conditions but also delves
Citation: Tahir, N.U.A.; Zhang, Z.;
Asim, M.; Junhong, C.; ELAffendi, M.
into the ever-evolving realm of the architecture of AVs, challenges for automated vehicles in adverse
Object Detection in Autonomous weather, the basic structure of OD, and explores the landscape of traditional and deep learning
Vehicles under Adverse Weather: A (DL) approaches for OD within the realm of AVs. These approaches are essential for advancing
Review of Traditional and Deep the capabilities of AVs in recognizing and responding to objects in their surroundings. This paper
Learning Approaches. Algorithms further investigates previous research that has employed both traditional and DL methodologies for
2024, 17, 103. https://doi.org/ the detection of vehicles, pedestrians, and road lanes, effectively linking these approaches with the
10.3390/a17030103 evolving field of AVs. Moreover, this paper offers an in-depth analysis of the datasets commonly
Academic Editors: Myrto employed in AV research, with a specific focus on the detection of key elements in various environ-
Konstantinidou and Sotirios mental conditions, and then summarizes the evaluation matrix. We expect that this review paper will
Kontogiannis help scholars to gain a better understanding of this area of research.

Received: 14 January 2024

Keywords: intelligent transportation system; autonomous vehicles; object detection; deep learning;
Revised: 16 February 2024
traditional approaches
Accepted: 19 February 2024
Published: 26 February 2024

1. Introduction
Copyright: © 2024 by the authors. The World Health Organization (WHO) releases statistics each year on the number
Licensee MDPI, Basel, Switzerland. of people who have been hurt in traffic accidents. Around 1.3 million people worldwide
This article is an open access article die each year, and 20 to 50 million suffer serious injuries [1], with young males under
distributed under the terms and
25 constituting the majority, 73%, of fatal traffic accidents. Extensive research on the wor-
conditions of the Creative Commons
rying prevalence of traffic fatalities has led to the development of upgraded intelligent
Attribution (CC BY) license (https://
transportation systems (ITS) with expanded capabilities. The cutting-edge of contem-
creativecommons.org/licenses/by/
porary automotive technology is represented by automated driving systems (ADS) and
4.0/).

Algorithms 2024, 17, 103. https://doi.org/10.3390/a17030103 https://www.mdpi.com/journal/algorithms

Algorithms 2024, 17, 103 2 of 36

autonomous vehicles (AVs). AVs, especially self-driving cars, have generated a great deal
of interest in the fields of intelligent transportation, computer vision, and artificial intel-
ligence [2]. No other invention since the creation of vehicles has had such a dramatic
impact on the automotive industry as self-driving vehicles. As per the forecasts presented
in Figure 1 [3], the market for self-driving cars is expected to grow from 23.80 million
units in 2022 to 80.43 million units by 2032. These vehicles primarily rely on cameras
and sensors, and examples include LiDAR (light detection and ranging), radar, ultrasonic,
etc., which have a number of autonomous features, including reduced use of fuel, cutting
carbon dioxide emissions by up to 10%, decreased traffic congestion [4], decreased road
accidents, and utilizing roads more efficiently to optimize the transportation system. AVs
can assist in managing and studying traffic by gathering effective and valuable traffic
data. Along with other road-related capabilities, they are capable of recognizing static
and non-static objects such as vehicles, pedestrians, and road lanes. Object detection (OD)
is critical for AVs as it ensures safety by identifying and avoiding moving obstacles in
their path. Numerous research works have investigated methods for identifying objects
in AVs, broadly divided into manual, semi-automated, and fully automated systems [5],
as shown in Figure 2. Roadside artifacts are visually assessed by human inspectors using
manual procedures; nonetheless, subjectivity and labor intensity are drawbacks. While
semi-automated methods gather data from moving cars and process them afterward, they
are still labor-intensive [6]. High-resolution cameras and sensors on cars record pictures
and videos in completely autonomous systems. Software-based models that have been
trained to detect objects evaluate the data either after they have been collected or in real
time. These techniques seek to solve safety problems while improving the precision and
effectiveness of item identification on roadways and streets. Vehicle-based surveillance is a
widely used technique since it is a quick and efficient way to examine objects. However,
it has always been difficult for autonomous programs to identify objects under adverse
weather conditions such as rain, fog, snow, haze, storms, and low lighting effects.

81.00 $80.43

72.90 $70.93

64.80 $62.6

56.70 $55.3
$48.9
48.60
$43.27
40.50 $38.33
$33.98
32.40 $30.15
$26.78
24.30 $23.8

16.20

8.10

0.00
2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032

Figure 1. The global self-driving cars market size 2022–2032 in USD Million [3].

The weather has a number of negative consequences for mobility and traffic. Pre-
cipitation is recorded roughly 11.0% of the time worldwide on average [7]. Studies have
unequivocally shown that, in comparison to typical weather, rainfall can cause a 70%
increase in the likelihood of accidents [8]. Moreover, snowfall occurs in 77% of the countries
worldwide. For example, according to US national data, annually, 24% of vehicle accidents
related to weather conditions take place on slippery, snowy, slushy, or icy roads, while
15% occur when snow is actively falling or mixed with sleet [9], highlighting the real risks
associated with snowy weather. Visibility is greatly reduced by environmental variables
such as fog, haze, sandstorms, and strong sunlight, which presents real difficulties for
drivers [10].
Algorithms 2024, 17, 103 3 of 36

Object Detection Strategies

Semi Fully
Manual
Automated Automated

Figure 2. Object identification strategies.

Traditional and deep learning (DL) algorithms are crucial for enhancing OD in au-
tonomous vehicles, especially in adverse weather. While traditional AI, based on statistical
learning, has been foundational, it faces limitations due to manual feature engineering
and limited adaptability to dynamic environments, requiring extensive retraining and
hindering its effectiveness in real-time applications.
DL offers a powerful alternative to traditional machine learning methods for OD
in AVs, addressing the limitations of manual feature engineering and adaptability in
dynamic environments. DL’s multi-layer neural networks automatically extract features
and learn from data, providing a more flexible and adaptable solution that enhances the
performance of surveillance systems, self-driving vehicles, and smart city applications [11].
DL’s neural network-based approach is particularly adept at handling complex models that
traditional techniques cannot [12]. In AV object detection, hybrid models combining one-
stage and two-stage detectors are effective, with two-stage models focusing on accuracy
and one-stage models on speed, although the reasons for the latter’s lower accuracy are
not fully understood.
The research significance of this paper lies in its comprehensive analysis of the chal-
lenges and advancements in OD for AVs in adverse weather conditions. It provides a
critical review of both traditional and DL approaches, offering insights into the limitations
and potential improvements of the current detection algorithms. The paper contributes to
the broader field of intelligent transportation systems by emphasizing the need for robust
and reliable detection systems that can operate effectively in a variety of weather scenarios,
which is crucial for the safe deployment of AVs in real-world conditions. Building on this
significance, this paper also contributes to the discussion regarding AV architecture and
challenges in adverse weather, and reviews the literature on detecting pedestrians, vehicles,
and road lanes using traditional and DL methods. It also summarizes common evaluation
metrics for OD. In this paper, we contribute to the field by examining the fundamental
architecture of AVs and the specific challenges they face in adverse weather conditions. We
have compiled comprehensive datasets, leveraging real-world statistics from LiDAR and
camera sensors, to provide a robust foundation for our analysis. We detail the core structure
of OD systems and elucidate both traditional and DL methodologies for AVs. Building
upon these approaches, we provide a critical review of the existing literature, focusing
on the detection of three primary objects—pedestrians, vehicles, and road lanes—under
challenging weather conditions. The structure of this paper is shown in Figure 3.
Algorithms 2024, 17, 103 4 of 36

Section 1
Introduction Sensor
Technologies
Section 2
RelatedSurvey Actuators

O Section 3
Overview of AVs Perception
R
G Section 4 Planning and
A Overview of OD Control
N Section 5 Challenges for
I Overviewof AVs under
Traditional / Adverse
Z DLApproaches weather
A
T Section 6
Previous work in
I OD
O Vehicle Pedestrian Road Lane
Section 7
N Discussion /
Detection Detection Detection
Limitation/ Future
Work
Section 8
Conclusion

Figure 3. Paper organization, where OD = object detection.

2. Related Survey Papers

This section critically evaluates the existing literature on autonomous vehicle (AV)
sensor performance under challenging weather conditions, with a focus on the integration
of traditional and DL detection techniques. One study [13] presents an in-depth analy-
sis of the impact of inclement weather on AV sensors, particularly the attenuation and
backscatter effects on millimeter-wave radar in heavy rain. Despite the study’s novel mod-
eling approach, it does not comprehensively address the multifaceted challenges posed
by other weather phenomena, such as fog and snowfall, which is crucial for a holistic
sensor performance evaluation. Research [14] offers a comprehensive review of sensor
performance in adverse weather conditions, emphasizing the importance of sensor fusion
for improved vehicle perception. However, the study’s focus on range and resolution does
not fully explore the nuanced challenges of object classification and sensor reliability in cold
weather conditions, which is critical for AV operation. The study in [15] aims to develop a
robust operational system for AVs in various weather conditions, advocating for sensor
fusion and infrastructure support. Yet, the study’s discussion on sensor reliability does not
adequately address the practical considerations of sensor installation costs and benefits,
which are significant factors in the practical implementation of AV technology. The study
in [16] investigates the effects of adverse weather on LiDAR and camera lenses in AVs,
suggesting improvements in sensor hardware and the application of sophisticated machine
learning methods. While the study’s focus on sensor fusion and network infrastructure
is commendable, it does not provide a comprehensive evaluation of these technologies
across different weather conditions. The research in [17] emphasizes the role of weather
recognition and categorization systems in AV decisionmaking, focusing on the detection
of road lanes, vehicles, and pedestrians. While the study provides valuable insights, it ac-
Algorithms 2024, 17, 103 5 of 36

knowledges that detecting smaller, obscured, or partially visible objects remains a challenge,
indicating a need for further research in this area. The study in [18] serves as a tutorial
on advanced techniques addressing the impact of rainy conditions on AV object detection.
The study evaluates the performance of detection methods under clear and rainy scenarios,
highlighting the efficacy of deraining methods, DL-based domain adaptation, and image
translation frameworks. However, it does not comprehensively address the limitations of
these methods in other adverse weather conditions. The study in [19] examines vehicle
detection approaches in video-based traffic surveillance, highlighting the challenges posed
by dim illumination, variable weather, occlusion, and shadows. The study’s exploration
of traditional approaches is valuable, but it does not fully integrate the advancements
in DL methods for improved detection. Other [20] research focuses on DL methods for
detecting on-road vehicles, providing a valuable list of works. However, the study’s limited
scope does not encompass the full range of challenges faced by AVs in adverse weather
conditions. Another paper [21] exclusively examines 3D methods for vehicle detection,
offering insights into a specific detection approach. Yet, the study’s narrow focus does
not address the broader context of AVs and their performance under various weather
conditions.
In summary, while the surveyed literature provides a valuable foundation for un-
derstanding AV sensor technologies and detection techniques, there is a critical gap in
the holistic examination of both traditional and DL approaches under adverse weather
conditions. This survey paper focuses on traditional approaches, deep learning approaches,
and their applications in object detection in AVs under adverse weather conditions. Fur-
thermore, we have detailed our literature review methodology, which involves a systematic
search across reputable digital repositories such as Springer Link, Elsevier, IEEE, MDPI, and
Wiley Online Library. We utilized targeted keywords to identify relevant research articles
and focused on studies published in peer-reviewed journals and international conferences
from 2001 to 2023. This approach ensured a comprehensive and high-quality selection of
literature for our review.

3. An overview of AVs
The Society of Automotive Engineers (SAE) has established a classification system
that divides Autonomous Driving Systems (ADS) into six levels of automation, ranging
from Level 0 (No Automation) to Level 5 (Full Automation), as illustrated in Figure 4 [22].
This system is pivotal in understanding the evolution of autonomous vehicles (AVs), which
are equipped with advanced sensors and software to navigate independently, thereby en-
hancing safety technologies and autonomy. Such a progression necessitates a collaborative
effort among scientists and engineers across various disciplines to address the complex
challenges associated with AV development [23]. The SAE automation levels provide a
standardized framework that is critical for evaluating the capabilities and limitations of
AVs at different stages of their development. By categorizing AVs into these distinct levels,
it becomes possible to systematically assess the technological advancements and hurdles
encountered at each level. This structured approach is instrumental in setting realistic
expectations about AV performance, shaping regulatory frameworks, and educating the
public on the operational capabilities of AVs under various scenarios. Thus, the classifi-
cation of AVs according to SAE levels is essential for advancing the field of autonomous
driving, guiding its regulatory landscape, and informing societal understanding of these
emerging technologies.
Algorithms 2024, 17, 103 6 of 36

Driver Partial Conditional High

No Automation Full Automation
Assistance Automation Automation Automation
Automated Under specific
capabilities, such as With automatic The vehicle operates
conditions, the
Full manual control, The vehicle is driver- standard steering and steering, braking, entirely
vehicle is capable
controlled, acceleration, are And acceleration, the autonomously,
where the driver is of performing all
with certain functions vehicle can operate undertaking all
responsible integrated, yet the driving
like braking capable independently, but the driving tasks without
for executing all of being automated by driver retains control functions, but the
driving tasks. and must oversee the driver driver always has the any need for
the vehicle. has to be ready for human attention or
remainder of the choice to take
handing over. interaction.
driving task. control.

Level 0 Level 1 Level 2 Level 3 Level 4 Level 5

Figure 4. SAE automation levels for AVs.

Initially, in the 1920s, AVs were referred to as “phantom autos” due to their lack of a
human driver, relying on remote control [24]. Significant advancements in AV technology
emerged in the 1980s, notably with Pomerleau’s “Autonomous Land Vehicle in a Neural
Network” project [25]. This project demonstrated the potential of neural networks to signif-
icantly enhance autonomous navigation systems. The Defense Advanced Research Projects
Agency (DARPA) initiated the inaugural Grand Challenge in 2004 with the primary aim
of stimulating research and the advancement of AVs. Subsequent to the 2004 challenge,
DARPA hosted additional grand challenge events in 2005 and 2007, famously recognized
as the Urban Challenge [25]. Developments in recent times have resulted in the global
implementation of AVs. Some manufacturers have achieved Level 4 SAE standard [26]
autonomy through the use of LiDAR technology. Google started secretly building its first
AV in 2009, and, on May 1, 2012, in Las Vegas, they successfully completed their first
autonomous driving test [27]. The UK government launched a competition in 2014 with
the goal of encouraging and developing AVs [28]. The Mcity driverless shuttle project was
started recently by the University of Michigan. It is the first Level 4 automated shuttle
project in the US [29]. However, due to safety considerations, AVs cannot operate in severe
rain or snow. Although testing and research have been carried out in inclement weather,
problems such as the persistent usage of windshield wipers may result in an operational
stoppage. Low temperatures impacted charging during the night and limited daily operat-
ing hours for the Sohjoa Baltic Project [30]. Tesla’s autopilot works well in mild rain and
snow and has obvious lane markers, but it struggles in inclement weather [31]. Regulation
(EU) 2019/2144 was introduced by the European Parliament and Council in 2019. This was
the first time that guidelines pertaining to automated and fully autonomous vehicles had
been established [32]. When Waymo (formerly Google’s self-driving division, rebranded
as Waymo in 2016) announced that their “Waymo Driver” had successfully completed
20 million self-driven miles and carried out simulations that equated to 15 billion miles, it
marked an important turning point in the growth of AVs [33]. While researchers are striving
to develop sensors for different weather scenarios, thorough studies that address every
facet are still absent. The AVs use environment-sensing skills to function with the least
amount of human interaction [34]. Using an autonomous driving framework, the weather
has a significant impact on the data obtained from car sensors and the general state of the
surroundings. Adverse weather conditions create obstacles to object identification, moni-
toring, and localization, necessitating modifications to control and planning techniques.
Furthermore, the efficiency of the vehicle can be directly impacted by meteorological factors
like wind, general weather, and road conditions.
This interaction results in an ongoing cycle whereby changes in the environment and
vehicle states impact one another, highlighting the weather’s crucial role in autonomous
driving. Figure 5 shows the architecture of AVs.
Algorithms 2024, 17, 103 7 of 36

External Factors

Hardware S/w Modules

Perception

Sensors Planning
Autonomous Car
Static & non static
objects Actuators Control

Weather External
forecasts V2V, V2I, VLC, V2X

Figure 5. An architecture of AVs.

3.1. Sensor Technologies

In the pursuit of improving OD in inclement weather, selecting the right sensor
technologies is essential to guaranteeing precise and dependable results. Unpredictable
natural events, like adverse weather, might have an impact on the operating environment
of AVs. These changes in the surrounding environment bring about differences in how
AV sensor systems, which are the fundamental parts of ADS, operate. The main sensory
elements used in AVs for perception are described in the section that follows, and the
comparison of sensors is shown in Table 1.

Table 1. Comparison of sensors [35].

Sensors Advantages Disadvantages

LiDAR High resolution Long range Wide FOV Sensitive to weather Expensive
Long range Detection of velocity
Radar Low resolution Very sensitive
Suitable for all types of weather
Camera High resolution Detection of colors Sensitive to weather Sensitive to lighting
Ultrasonic Non-InvasiveReal-time feedback Low resolution Expensive

3.1.1. LiDAR
LiDAR is considered the primary perceptive sensor in the self-driving car industry.
Even though 3D-LiDAR has only been used in automobiles for slightly over ten years, it has
already been shown to be crucial to the development of AVs and ADS [36]. Its exceptional
ability to measure things precisely and its resilience to changing lighting conditions high-
light how vital it is. Critical features of 3D scanning using lasers include its form aspect,
affordability, tolerance to external variables, measuring range, precision, pinpoint density,
scanning acceleration, configurational versatility, and spectral qualities [37]. These sensors
work well in a range of weather conditions by using laser beams to measure distances and
produce accurate 3D point clouds. However, there could be problems when there is a great
deal of rain, snow, or fog since laser beams can disperse and potentially skew the data.
Older types, such as the Velodyne HDL64-S2 [38], continue to function better in dense fog,
while the performance of more modern laser scanners decreases. While making manual
modifications can increase performance, it is still difficult to perceive reliably in deep fog.
For this reason, alternative technologies, including gated imaging, should be considered
for better performance in adverse conditions [39].
Algorithms 2024, 17, 103 8 of 36

3.1.2. Radar
The fundamental components of ADS and self-driving automobiles are automotive
radars and a variety of additional sensors. The successful use of these technical advances
depends on complex systems with long signal-processing chains that link the radars and
sensors to the controller. The vital tasks of identifying objects and obstructions, pinpointing
their exact location, and evaluating their relative speed in relation to the vehicle fall to
automotive radar equipment. Advances in millimeter-wave (mm-wave) semiconductor
technology, along with the development of signal processing techniques, form the basis
for the progress of car radar systems. A variety of signal processing methods have been
developed to improve resolution and reliability in a range of measuring parameters, includ-
ing target velocities in the vicinity, azimuth elevation perspectives, and the radius of the
vehicle [40]. Frequency Modulated Continuous Wave radar, or FMCW radar, modifies the
broadcast signal’s frequency continually at a predetermined pace. By measuring the gap
between the sent and reflecting signals, which is inversely related to the amount of time
required for the electrical signal to make its way to and from the object being targeted, we
can determine the range due to this modulation of frequency. FMCW radar has excellent
accuracy and range resolution, as well as speed measurement features, which make it
especially useful for autonomous driving applications [41].

3.1.3. Ultrasonic
One typical vehicle sensor that is frequently left out of discussions regarding ADS
is the ultrasonic sensor. Despite this error, it is positioned strategically on the vehicle’s
bumpers and throughout its body, making it indispensable for parking assistance and
blind spot surveillance. Surprisingly, ultrasonic sensors have proven to be dependable and
reasonably priced over a long period of time, which makes them a useful sensor choice [42].
The frequency range of ultrasound sounds, which are audible only to humans, is normally
between 30 and 480 kHz. In the field of ultrasonic sensing, the frequency range that is most
frequently utilized is between 40 and 70 kHz. The resolution and sensor range are highly
dependent on the selected frequency. Longer sensing ranges are correlated with the lower
ones. For example, the measuring range is up to 11 m, and the accuracy is one centimeter
(cm) at the commonly used frequency of 58 kHz. Conversely, higher frequencies such as
300 kHz provide amazing resolution, maybe down to one millimeter, but at the penalty of
a shorter range, capped at about 30 cm [43]. Ultrasonic sensors are useful for close-quarters
applications like parking because their normal operating range is 11 m [44]. However, they
can be used in autonomous driving; for example, Tesla’s “summon” capability can be used
to navigate garage doors and parking lots [45].

3.1.4. Camera
As the eyes and storytellers of automobiles, cameras are essential components of ADS.
They are a crucial component of ADS, capturing the dynamic story of the surroundings,
even though they are technologically more advanced than LiDAR. Installed on Windows,
dashcams record continuously and have made a substantial contribution to the initial ADS
datasets. Fisheye-lens professional camera setups nowadays increase the volume of data
that may be collected. On the other hand, adverse weather can cause visual problems for
cameras due to rain, snow, and fog. In low light, specialized cameras improve visibility,
such as night vision versions. The potential to overcome weather-related constraints in
sensor fusion methods through the integration of these cameras is encouraging. The
function of cameras will change further as autonomous driving technology develops,
influencing sensor technologies and how they interact with the outside world. In summary,
cameras are the storytellers of autonomous cars; they are sensitive to bad weather yet
versatile enough to be essential to safer autonomous systems.
Algorithms 2024, 17, 103 9 of 36

3.2. Actuators
AVs use data from the planning stage to carry out the vehicle’s motions in the control
stage. This entails giving actuators instructions for steering, braking, accelerating, and
signaling. A drive-by-wire method is frequently used to transmit commands effectively.
The control system is in charge of creating and monitoring trajectories, making sure that
intended routes are taken. Methods for generating trajectory include sensor-based, which
is appropriate for robotics, and dynamics-based, which is more appropriate for vehicles.
The majority of the time, trajectory tracking uses geometrical or model-based techniques.
Although they have limits and only react to errors as they happen [46], feedback con-
trollers like the proportional-integral-derivative (PID) are employed to prevent deviations
from intended trajectories. Two-degree independent controllers combine feedforward and
feedback controllers to circumvent these restrictions.

3.3. Perception
To highlight the importance of perception for AVs, it is important to emphasize how
it helps the AV determine its location and its surroundings. In order to extract road fea-
tures, detect objects, anticipate their behavior, and carry out Simultaneous Localization and
Mapping (SLAM) activities, the perception module uses data from sensors and commu-
nication equipment [47]. For detecting the surroundings, a variety of active and passive
sensors, such as cameras, LIDAR, and RADAR, can be employed; each has advantages
and disadvantages of its own, as shown in Table 1. Emphasized are the benefits and
drawbacks of various sensors, illustrating the compromises made in terms of resolution,
weather sensitivity, and accuracy. One way to overcome the limits of individual sensors
is to introduce sensor fusion, a technique that combines data from several sensors. There
are two main types of AV systems discussed: one is sensor fusion, which uses various
sensors such as LIDAR, RADAR, and cameras, and the other is pure vision, which uses
cameras and computer vision algorithms. It is mentioned that a pure vision-based system
has advantages like affordability and simplicity of implementation. In the parts that follow,
this research concentrates on the benefits of a pure vision-based method [48].

3.4. Planning and Control

Global route planning and local path planning are the two main planning challenges
in autonomous driving. When given an origin and destination, the global planner’s job
is to come up with possible road options and routes. Local path planning, on the other
hand, concentrates on carrying out the chosen path without running into obstructions or
breaking any traffic laws. Other modules, such as OD, which guarantees safe navigation,
frequently supplement this role. Further details regarding motion planning can be found
in [49] for a more thorough explanation. Like traditional cars, steering, throttle, and brake
inputs are the main controls for AVs [50]. These inputs control the AV’s direction, speed,
and acceleration, which in turn control its driving, under the supervision of the ADS
mainframe’s judgments based on sensor data and analysis. Low-level safety precautions
can also be put in place using sensors like sonar and electrical control units. ADS sensors
are challenged by weather conditions, which interfere with perception functions and have
an impact on detection, planning, and control, among other factors. Weather factors like
wind and variations in the road surface have an immediate effect on the vehicle and its
state, in addition to sensor-related problems. Environmental state changes result from these
state changes, which also have an impact on other nearby cars and sensor operations.

3.5. Challenges for AVs in Adverse Weather

Automation parts up to Level 3 have been added by automakers, and they rely on
sensors and cameras. These devices function best in optimal conditions; thus, unfavorable
weather presents obstacles. Weather has a major impact on AV performance, regardless of
whether it is road-related or generic. A few of these effects are shown in Figure 6, and these
are explained in more detail below.
Algorithms 2024, 17, 103 10 of 36

Technical parameters
General Weather Road Weather
Slow down when Extended
it's foggy outside. headways are
necessary in
foggy scenarios.
Sensor performance
Driver behavior General Weather Road Weather

General Weather Road Weather The sensor Snow cover

detection range lane markers
Decreased A longer might be affected have an impact
visibility impacts stopping by persistent rain on LiDAR
the speed of distance is or snow. performance.
conventional required in icy
vehicle drivers. conditions.
Communication
Cameras performance General Weather Road Weather
General Weather Road Weather
Thunderstorms Snow, slush, or
The cameras can't A detecting may interfere road debris can
identify obstacles error might with harm vehicle
in dense snow or arise from ice communication comm devices,
fog. reflection. resulting in link
disruption.

Figure 6. Weather impacts on AVs.

3.5.1. Performance of Sensors

Adverse weather conditions on roads provide serious difficulties for sensors in AVs.
The current sensing technologies, including LiDAR and cameras, perform well in clear
weather but have trouble when the roadway is covered with snow and may contain
erroneous reflections caused by rain or snow, making it difficult to discern between different
objects [13]. Even though forward radar is good at spotting large objects and piercing
through weather, it cannot improve safety applications on its own. In snowy circumstances,
ultrasonic detectors may set off false alarms, and glare and surface reflection are just two
examples of weather-related variables that might affect sensor accuracy.

3.5.2. Performance of Cameras

While camera systems are crucial for applications involving driverless vehicles, unfa-
vorable weather conditions such as fog, intense rain, snow, and dust reduce their effective-
ness. Although sophisticated techniques for image processing try to solve these problems,
accuracy and dependability issues still exist, which presents significant obstacles to the
advancement of autonomous driving [51]. In the right circumstances, glare can potentially
affect functionality. However, these systems may be able to identify hazardous weather
conditions immediately.

3.5.3. Technical Parameters of the Vehicle

Adverse weather affects a vehicle’s functioning factors. Slow speeds and longer fol-
lowing distances are necessary for safety due to decreased road friction and slick conditions.
As was previously mentioned, during bad weather, driver behavior can also have a parallel
impact on vehicle operating parameters.

3.5.4. Behavior of Driver

Driver conduct is still crucial once AVs are deployed, particularly in situations with
mixed traffic. In order to ensure operational safety, AV applications need to take into
account the existence of other drivers and environmental elements. These factors include
driver-state surveillance and the ability to react to different weather conditions.

3.5.5. V2V, V2I, and VLC Communications

The influence of meteorological conditions on network latency can sometimes prevent
vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications from connect-
ing in a direct line of sight. For safety applications, minimal latency is essential. While
radio-based communications are less impacted by snow, they are nevertheless susceptible
Algorithms 2024, 17, 103 11 of 36

to problems like solar flares and thunderstorms. Communication interruptions can result
from bad weather such as fog, heavy rain, snow, and hail obstructing the visible light
communication (VLC) line of sight.
The process of OD in AVs unfolds in several key steps, beginning with the collection
of image data by the sensor system. Following this, a Region of Interest (ROI) or regional
proposal strategy is employed, marking a critical phase in object tracking. This approach is
commonly utilized in devices such as cameras and stereoscopic systems, where ROIs serve
as the foundational elements for tracking objects. The subsequent step involves the use of
classifiers to meticulously analyze and refine shapes, edges, and lines. This analysis aids in
the classification of targets and their identification based on human categories, embedding
these ROIs within the image for object recognition.
Various techniques, including sliding windows, Locally De-correlated Channel Fea-
tures (LDCF), and constrained searches, are applied to locate ROIs efficiently. The extraction
of features from these ROIs represents the third stage of the process. At this juncture, ei-
ther DL-based methods or traditional manual OD algorithms can be employed for the
extraction and classification of features. While manual feature extraction techniques can
be advantageous for models that rely on lower-level features by facilitating manual ROI
suggestions [52], their efficacy diminishes when dealing with complex features, rendering
them less reliable. However, to derive useful and relevant data from video or photographs
using a computing device, a great deal of computation has to be completed [53]. In contrast,
DL methodologies enable systems to autonomously learn feature representations, thereby
enhancing the accuracy and adaptability of feature extraction in dynamic and complex
driving environments.
The evolution of research on OD in bad weather depends heavily on datasets. They
act as the building blocks for creating, refining, and testing algorithms and models. In this
part, we are going to discuss a few of the datasets that are frequently utilized in this field
of study. One of the most often used benchmarks for autonomous driving research is the
KITTI [54] dataset. It features a range of various metropolitan settings. The collection offers
LiDAR data, excellent-quality pictures, and annotations for cars. A2D2 [55] is a sizable
dataset and delivers extensive annotations for several items, including vehicles, together
with high-resolution camera and LiDAR data. Waymo’s dataset [56], which is renowned for
its comprehensive and varied gathering of data from AVs and includes information from
a range of weather and lighting circumstances, is a useful tool for research on inclement
weather. Among the noteworthy datasets, NuScenes [57], BDD100K [58], ApolloScape [59],
and SYNTHIA [60] datasets offer diverse and extensive resources for exploring vehicle
detection in challenging conditions. A quick overview of the weather conditions consid-
ered and the particular sensors utilized for gathering data for each dataset can be found in
Table 2. The University of Michigan used a Segway robot to collect four seasons of LIDAR
data for the NCLT [61] on campus. The first AV dataset, called the Canadian Adverse
Driving Conditions (CADC) [62] dataset, focuses on snow conditions. The Challenging
Conditions Dataset with Correspondences (ACDC) [54], created for training and testing lex-
ical techniques for segmentation in difficult visual conditions, was later introduced by the
same team. ACDC includes a high-quality pixel-level resolution collection of well-labeled
photos encompassing various categories of mist, at night, rain, and snowfall. The public’s
Digital Video Collection now includes the IUPUI [63] Driving Video/Image Benchmark.
Example views of varied lighting and roadway conditions captured by in-car webcams
are included in this baseline. In these movies, patrolling and disaster response situations
involving safety automobiles are shown. Together, these datasets provide priceless re-
sources for developing the discipline and promoting creativity in the creation of reliable
and weather-adaptive vehicle detection systems.
Algorithms 2024, 17, 103 12 of 36

Table 2. Publicly available AVs datasets.

Ref. Year Dataset LiDAR Radar Camera Adverse Weather

[37] 2020 LIBRE ✓ - ✓ Rain, fog, Daytime
[51] 2013 Kitti ✓ - ✓ -
[54] 2021 ACDC - - ✓ Rain, fog, snow
[55] 2020 A2D2 ✓ - ✓ Rain
[56] 2020 Waymo ✓ - ✓ Rain, night
[57] 2020 NuScenes ✓ ✓ ✓ Rain, night
[58] 2020 BDD100K - - ✓ Daytime, night
[59] 2020 ApolloScape ✓ - - Rain, Night
[60] 2016 SYNTHIA - - - Snow
[61] 2016 NCLT ✓ - ✓ Snow
[62] 2021 CADCD ✓ - ✓ Snow
[64] 2020 End of the earth ✓ - ✓ Snow
[65] 2020 nuImages - - ✓ -
[66] 2019 Argoverse ✓ - ✓ -
[67] 2019 Astyx ✓ ✓ ✓ -
[68] 2020 DENSE ✓ ✓ ✓ Rain, snow, fog, night
[69] 2018 Foggy Cityscape - - - Fog/haze
[70] 2020 Berkley DeepDrive - - ✓ Rain, fog, snow, night
[71] 2017 Mapillary - - ✓ Rain, fog, snow, night
[72] 2019 EuroCity - - ✓ Rain, fog, snow, night
[73] 2017 Oxford RobotCar ✓ ✓ ✓ Rain, snow, night
[74] 2020 A* 3D ✓ - ✓ Rain, night
[75] 2021 4Seasons - - ✓ Rain, night
[76] 2018 WildDash - - ✓ Rain, fog, snow, night
[77] 2018 KAIST multispectral - - ✓ Daytime, night
[78] 2021 Radiate ✓ ✓ ✓ Rain, fog, snow, night
[79] 2020 EU ✓ - ✓ Snow, night
[80] 2022 Boreas ✓ ✓ ✓ Rain, snow, night
[81] 2020 DAWN - - - Rain, snow, fog, sandstorm
[82] 2021 PVDN - - ✓ night

4. Overview of Object Detection in AVs

Most OD algorithms primarily adhere to a common framework, as shown in Figure 7.
In OD, we have considered the following three issues:
1. Vehicle Detection: Vehicle detection is the process by which AVs identify and locate
other vehicles on the road. This capability is crucial for AVs to make informed
decisions about their own movement, such as maintaining safe distances, changing
lanes, or responding to traffic situations.
2. Pedestrian Detection: Pedestrian detection involves the recognition and tracking of
people walking near or crossing the road. This is a vital safety feature for AVs as it
enables the vehicle to anticipate and react to the presence of pedestrians, preventing
collisions and ensuring the safety of both the vehicle’s occupants and those outside.
Algorithms 2024, 17, 103 13 of 36

3. Road Lane Detection: Road lane detection is the ability of AVs to identify and under-
stand the position and orientation of road lanes. This information is essential for the
vehicle to navigate within its designated lane, follow traffic rules, and make correct
turns, ensuring a smooth and safe driving experience.

1 2 3
a) Sensor
(a) (b)
b) ROI
(c)
c) Feature Extraction

(d)
d) Classification

Figure 7. The basic framework for object detection systems.

5. Overview of Traditional and DL Approaches for Object Detection in AVs

The detection techniques are composed of three parts: the DL approach, the traditional
technique, and a hybrid approach that utilizes both. The DL and traditional techniques are
explained in detail in the section that follows. The performance graph of both approaches
is shown in Figure 8.

Deep learning
performance

Traditional
approaches

Data

Figure 8. Performance graph of traditional and DL approaches.

5.1. Traditional Approach

Various algorithms were developed for the purpose of OD. Prominent traditional
feature extraction methods include Scale Invariant Feature Transform (SIFT) [83], Viola–
Jones rectangles [84], Haar-Like-wavelets and Histogram of Oriented Gradient (HOG) [85],
Adaptive Boosting (AdaBoost), non-linear Support Vector Machines (SVMs), and linear
SVMs, which are common classical approaches for object classification. A number of steps
are involved in the SIFT algorithm [83], which is well known for its ability to determine
scale and rotation-invariant features, making it resistant to partial occlusion, clutter, noise,
and lighting variations. These include the detection of scale-space extrema, the localization
of key points, the assignment of orientations, the creation of keypoint descriptors, and
the final step of using key point matching to recognize objects. Descriptors are used to
compare and identify items in a database using methods like nearest neighbor indexing
and the Hough transform. Nevertheless, SIFT has some drawbacks, including the absence
of ongoing key point consistency in dynamic objects, the significant dimensions of feature
descriptors that can affect matchmaking, and some restrictions related to patents.
Algorithms 2024, 17, 103 14 of 36

The Viola–Jones approach (Viola and Jones, [84] was developed for human face identi-
fication using Haar-like feature extractors. It consists of four primary parts; the primary
image algorithm for effective image participation is AdaBoost. Haar-like feature extraction
is used to uncover biased features. Cascade classifier implementation is used to filter out
background data and concentrate on regions that are more likely to contain the object
of interest. The algorithm performs remarkably well, taking only 0.067 s to process a
384 × 288 pixel image. Its detection rate is 91.4%, and it produces 50 false positives. Never-
theless, it is not well adapted for the generic recognition of objects and has slow periods
for training. The HOG method was created for identifying humans in digital photos and
was first presented in [85]. In order to determine the gradient’s length and direction, this
method examines input images employing gradient filters, both horizontal and vertical.
The filtered images are divided into 8 × 8 pixel cells, which are then further divided into
2 × 2 cell blocks that have 50% overlap. For every cell, orientation histograms are produced,
quantizing gradient directions into nine bins ranging from 0◦ to 180◦ . The magnitudes of
these bins are used as votes. A vector representing the HOG is produced by concatenating
these histograms for every cell in a block. To take into consideration changes in illumination
and contrast, these vectors are standardized. The final HOG descriptor, which consists of
all the normalized blocks, is fed into a Support Vector Machine (SVM) to determine if an
image is human or not. Although the HOG algorithm’s dense grid method results in a
larger processing burden and less efficient performance in cases involving obscured objects,
it is effective in decreasing false positives when compared with Haar wavelets.
These AI techniques, which have their roots in statistical principles, have traditionally
formed the basis of machine learning models. These techniques enable computers to
recognize trends and forecast outcomes based on past data. However, even with their
historical importance, traditional methods have clear drawbacks. Reliance on manual
feature engineering, as shown in Figure 9, in which experts carefully craft features for
the model necessitates domain knowledge and frequently fails to capture the nuances of
intricate datasets. Furthermore, these models’ ability to adjust to unfavorable weather
or dynamic surroundings is hindered by their inability to quickly incorporate new data
without requiring significant retraining, which limits their usefulness in situations that
change quickly. The development of more sophisticated methods, or DL approaches, to
overcome these obstacles and improve the power of AI systems has been made possible by
this realization of their limitations.

(a) Traditional Approach

Car/not
Features
a car

Input Feature Classification Output

Extraction

(b) Deep Learning Approach

Car/not
a car

Input Feature extraction Output

+ classification

Figure 9. Primary structure of feature extraction for traditional and DL approaches.

5.2. DL Approaches
DL emerged as a subfield of machine learning and artificial intelligence in the 1990s [86].
Unlike traditional methods, DL offers distinct advantages, including the ability to achieve
higher levels of abstraction, improved precision, and faster execution. These advantages
make DL a valuable choice for OD. The OD algorithm employing DL typically consists of
Algorithms 2024, 17, 103 15 of 36

three core components: a recurrent neural network (RNN), a depth belief network (DBN),
and a convolutional neural network (CNN). Overall, object detectors based on CNN [87]
can be categorized into two primary types.
1. Single or one-stage detectors are known as the “non-regional proposal method” and
“dense prediction”.
2. A two-stage detector is known as the “regional proposal method” and “sparse
prediction,” as shown in Figure 10. In a single-stage detector, all the tasks are integrated
into a unified system structure. Conversely, a two-stage detector separates the system
into distinct stages for region selection, classification, and localization. Some regional
proposal techniques consist of region-based CNN (R-CNN) [88], Fast R-CNN [89], Faster
R-CNN [90], Spatial Pyramid Pooling Networks (SPPNet) [91], and Feature Pyramid Net-
work (FPN) [92]. On the other side, the non-regional category includes Single Shot Multi-
box Detector (SSD) [93], You Only Look Once (YOLO 1-8) [94], EfficientDet [95], DEtection
TRansformer (DETR) [96], and Fully Convolutional One-Stage (FCOS) [97]. Since these
methods constitute the foundation of CNN, they have emerged as the standard for OD. The
amalgamation of one-stage and two-stage OD techniques has gained prominence in the
field of OD within AVs. These hybrid approaches have demonstrated effectiveness across
diverse scenarios, achieving promising results in precise OD and localization. Two-stage
algorithms tend to deliver superior accuracy, while one-stage algorithms offer faster pro-
cessing speeds. Notably, the reasons behind the lower accuracy of one-stage algorithms
remain unclear. A study examining the drawbacks of one-stage algorithms, especially those
with dense sampling, was conducted in Ref. [98]. The study found that there was a major
problem with the unbalanced background samples (negative instances) and foreground
values (positive examples).

Object Detection

Traditional Deep learning

Approaches Based

Viola HOG SIFT

Jones based based

One stage Two-stage

detector detector
SSD, RCNN, Fast-
EfficientDet, RCNN,
FCOS, YOLO Faster-RCNN,
series FPN, SPPNet

Figure 10. Traditional and DL approaches.

To solve this problem, the authors created Focal Loss, which alters the traditional
cross-entropy loss function to give problematic cases more weight and thereby puts the
emphasis on positive samples. After this adjustment, the RetinaNet network was created,
showing better accuracy and quicker detection times than one- and two-stage methods.
It should be mentioned too that the approach necessitates adjusting the focusing value,
an extra hyperparameter [99]. However, the two primary state-of-the-art methods for
DL-based detection these days are the YOLOv8 from the YOLO series (YOLOv1–YOLOv8)
and the Faster R-CNN from the R-CNN family.
Algorithms 2024, 17, 103 16 of 36

In order to speed up the region proposal step and allow the system to acquire knowl-
edge of optimal region proposals, Ren et al. [90] proposed Faster R-CNN as an improve-
ment to Fast R-CNN. The architecture of Faster R-CNN is described in Figure 11. This
was achieved mainly by integrating the region proposal network (RPN). RPN’s main ob-
jective is to produce proposals with various ratios of aspects and sizes. Similar to Fast
R-CNN, it employs high-resolution feature maps as input for RPN to identify the regions
of interest in an image rather than depending on selective search for region recommenda-
tions. While Faster R-CNN improves the precision of detection and reaches an evaluation
time of 0.2 s (five frames per second), it still requires a large amount of processing power
and cannot fully meet the real-time system demands. It is also more complex because
it requires training two interconnected neural networks, which makes system optimiza-
tion difficult [100]. While Faster R-CNN algorithms demonstrate satisfactory accuracy on
COCO and ImageNet datasets, they encounter difficulties when it comes to detecting small
objects and objects at varying scales. This approach, while being more memory-efficient
and faster, faces challenges in achieving consistent performance during both training and
test-time inferences.
Feature maps ROI Pooling

Image Conv layers

Classifier

Region Proposal Network

Proposals

Figure 11. The architecture of Faster R-CNN [90].

Jocher [100] implemented YOLOv8; however, that work has not yet been published in
a journal. The structure of YOLOv8 is shown in Figure 12. Different versions are available
with different scales of YOLO (N, S, M, L, and X) according to the scaling coefficient. The
two main differences are the variety of kernels at different locations in the network and
the number of feature extraction modules. These modifications are intended to meet the
unique requirements of various situations. The backbone, neck, and head are the three
primary modules that make up the YOLOv8 networks. First, the backbone module extracts
features from the input photos. The neck component then performs feature fusion to
create features in three different sizes: large, medium, and small. The final output is the
detection results. Eventually, these characteristics of various sizes are sent to the head
section for identification of the object. To improve deep image extraction features, the
YOLOv8 system uses a layered C2f technique inside the backbone section. The main
function of the SPPF layer, which is the tenth layer in the backbone module, is to transform
input feature maps—regardless of their original size—into a fixed-size feature vector. The
goal of this change is to increase the responsiveness of the network. YOLOv8 uses the
traditional top-down as well as bottom-up feature fusion technique referred to as PAN
(Path Aggregation Network) in the neck component. This tactic successfully combines
elements from several scales. YOLOv8 adopts the widely used decoupled head architecture
in the head section, which divides the categorization head from the identification head.
Furthermore, there is a noticeable change in the network’s OD technique: the anchor-free
strategy replaces the anchor-based method.
Algorithms 2024, 17, 103 17 of 36

Conv Neck Head

Conv

C2f
C2f Detect
Conv
Conv
Concat
C2f
upsample

Conv Concat
C2f

C2f Concat
C2f Detect
Conv
upsample Conv
C2f

SPPF Concat

C2f Detect
Backbone
Figure 12. Structure of YOLOv8.

The distinction between traditional and DL approaches is justified by their fundamen-

tally different methodologies and performance in various weather conditions. Traditional
methods, while effective in ideal conditions, struggle with adverse weather due to their
sensitivity to lighting, occlusion, and the need for precise feature engineering. DL methods,
on the other hand, have shown superior performance in handling these challenges by learn-
ing robust representations from large datasets, which include diverse weather conditions.
This allows DL-based systems to generalize better and maintain high detection accuracy
even under challenging circumstances.

6. Traditional and DL Approaches for Object Detection in AVs

AVs need to be able to identify moving items like cyclists, livestock, pedestrians, and
different kinds of automobiles, as well as stationary elements like traffic signals, road
signs, and parked vehicles because pedestrians are especially vulnerable and because AVs
frequently and critically engage with other types of vehicles. This section focuses only on
the previous literature using traditional and DL approaches for vehicles, pedestrians, and
road lane detection.

6.1. Traditional Approaches for Object Detection in AVs

In this section, we will summarize all the works on traditional approaches for OD
in AVs.

6.1.1. Traditional Approaches for Vehicle Detection

Traditional vehicle detection techniques, which rely on appearance- and motion-based
algorithms, have drawbacks, including being sensitive to changes in the camera’s position,
having trouble telling moving objects apart, and requiring a large number of photos
to accurately identify motion. Appearance-based algorithms that make use of features,
such as HOG, Haar-Like, and SIFT, can improve identification, but they are vulnerable
Algorithms 2024, 17, 103 18 of 36

to exterior objects, adverse weather, and lighting variations. Even though they help to
identify objects, biased classifiers like SVM and ANN might perform poorly in scenarios
involving complicated traffic, which can result in false positives and make them susceptible
to changes in the background, obstacles, lighting, and dimensions.
One study [101] investigated issues with physical detection techniques and equipment
maintenance by examining the application of SVM and Pyramid Histogram of Words
(PHOW) for traffic and road condition detection. The experimental findings show an
accuracy of more than 82.5% on average under different weather circumstances. The
suggested technique efficiently divides photos into four categories based on the type of
traffic and road conditions. However, the paper does not detail the dataset’s diversity, which
could impact the model’s robustness and generalizability. The research in [102] tackled the
requirement for a reliable traffic surveillance system that takes into consideration variables
like headlights, reflections, and shadows and operates well in a variety of weather and
lighting circumstances. The suggested method is based on pattern classification and uses a
distinctive image’s histogram as a distinguishing characteristic for classification. Because
of its excellent generalization skills, SVM is used as a classifier. In tests, the system can
discriminate between headlights and shadows with good accuracy and recognize moving
cars in a variety of weather and lighting scenarios. However, this paper does not address
the system’s performance under high-density traffic or its computational efficiency, which
could be critical limitations for real-time applications. Using webcams as weather sensors,
researchers [85] came up with a novel technique that allowed them to identify over 180,000
annotated photographs depending on indoor or outdoor circumstances with a 98% accuracy
rate using SVM. The results of the research [103] indicate that DPM effectively addresses
occlusion difficulties, even though it might not be required for ordinary detection settings.
Accuracy is improved by adding image context information, and system speed is increased
by adding other features. On the other hand, the multi-scale system only slightly improves
accuracy, and the classifier has little effect on detector quality [21,104,105].
The vehicle detection method in [106], which utilizes pseudo-visual pursuit and HOG-
LBP feature fusion, showcases the superiority of traditional machine learning techniques in
achieving high detection accuracy and speed in severe weather conditions. This approach,
with its reliance on radar information, demonstrates a clear advantage over methods that
do not incorporate such advanced sensor data, particularly in scenarios where visibility
is compromised. In contrast, the work by Abdullah et al. [107], while also effective, high-
lights the limitations of traditional methods when solely relying on MOG 2 background
subtraction and histogram equalization. Their system’s high effectiveness rate of 96.4%
across various weather conditions suggests that traditional techniques can still perform
well. The vehicle detection system by Wu and Juang [108], which uses histogram extension
and the Gray-level Differential Value Method (GDVM), shows that traditional methods
can be effective in complex traffic conditions. However, the system’s reduced accuracy in
rainy and dark conditions suggests a limitation of traditional approaches when dealing
with significant environmental variations. The authors in [109] developed a system for con-
current vehicle and weather detection using optical flow and color histogram approaches.
While this system demonstrates the utility of traditional methods in distinguishing between
raindrops and normal conditions, it also indicates that these methods may struggle with
the nuanced recognition tasks required in adverse weather.
Tian et al. [110] proposed a method for vehicle detection and tracking at night using
HOG features and SVM for object recognition, effectively leveraging traditional machine
learning techniques to enhance video surveillance in low-light conditions. Kuang et al. [111]
achieved a high detection rate of 95.82% with minimal false positives in their multiclass
system for nighttime conditions, employing tensor decomposition and object proposal
methods. Furthermore, the research in [112] presents a system that integrates rule-based
algorithms with a shallow neural network for nighttime vehicle detection, demonstrat-
ing significant improvements over traditional methods in terms of detection speed and
proactive capabilities. These studies highlight the strengths of both traditional and hybrid
Algorithms 2024, 17, 103 19 of 36

approaches in the realm of nighttime vehicle detection, providing a foundation for future
advancements in this field.
A vehicle tracking and counting system with camera shaking was presented in [113]
for difficult sandy weather conditions. The method used particle filter tracking for vehicle
detection, integrated headlight analysis, and background subtraction. While the method
worked well in difficult situations, such as evenings and sandy weather, with a vibrating
camera, it had trouble identifying white automobiles in snowy settings. Similarly, the
background subtraction method encountered difficulties when there was movement in
the backdrop or vibrations from the camera. A vehicle detection and tracking system for
complex traffic was described in [114]. It used a background reduction technique with
“low rank + sparse” disassembly and performed well in both qualitative and quantitative
evaluations. Table 3 tabulates these works.

Table 3. Vehicle detection previous work with traditional approaches.

Ref. Year Technique Weather Classes Accuracy

[101] 2014 SVM+PHOW Rain, Fog, Snow, Low light 82.5%
[106] 2022 HOG–LBP several weather 96.4%
[108] 2012 histogram GDVM Rain, Night 92.4%
[111] 2019 tensor decomposition Night 95.82%
[113] 2017 Particle Filtering Dust, Snow 94.78%

6.1.2. Traditional Approaches for Pedestrian Detection

This section delves into traditional techniques for pedestrian detection, comparing
their effectiveness and limitations in various conditions. A notable study [115] presents a
method that leverages an infrared camera for nighttime pedestrian detection, combining
Haar-Cascade and Histogram of Oriented Gradients-Support Vector Machines (HOG-SVM)
to reduce false alerts, demonstrating the superiority of this approach in low-light conditions.
Another study [116] employed LiDAR point clouds and SVM for object clustering and
classification in snowy environments, showcasing the robustness of this method for side-
walk snowplow machines at night. The real-world validation of this approach underscores
its practical advantages over traditional vision-based systems in adverse weather. The
work in [117] stands out for its real-time pedestrian detection system (PDS) that uses stereo-
based segmentation and SVM to distinguish pedestrians from objects with vertical edge
features. The high speed and detection rate of this method highlight its effectiveness, partic-
ularly in dynamic scenarios where camera motion is a factor. Research [118] addresses the
challenge of detecting pedestrians from a moving vehicle in inclement weather, using SURF
key points to compensate for camera motion and SVM for classification. The impressive
performance of this approach, as evidenced by its speed and detection rate, underscores
the potential of hybrid feature extraction methods in complex environments. Another
study [119] combines Haar and HOG features to improve multi-person recognition and
tracking, particularly in occluded and congested settings. The success of approaches in de-
tecting both the side and frontal faces of pedestrians, as demonstrated through experiments
on various datasets, highlights the limitations of single-feature approaches. The work
in [120] introduces a novel method for pedestrian recognition in hazy environments by
integrating a dark channel prior haze removal model with HOG descriptors. The improved
performance of this method over conventional HOG-based techniques on the ’INRIA’
dataset indicates the benefits of adaptive preprocessing for robust detection in challenging
conditions. With an emphasis on various weather situations, the study in [121] presents
a robust vision-based pedestrian detection system for intelligent transportation systems.
It incorporates feature extraction and SVM-based classification and addresses issues like
variability and occlusions in real traffic photos. In a more constrained environment, the
work in [122] demonstrates a pedestrian recognition method using SVM and HOG fea-
Algorithms 2024, 17, 103 20 of 36

tures, achieving an impressive 98.00% accuracy in counting individuals in a limited field

of view. This high accuracy, achieved through training on a dataset of 2000 road photos,
underscores the effectiveness of the model in controlled settings, potentially outperforming
other approaches that do not account for such precision. Another study [123] provides a
two-phase video recording pedestrian recognition method that combines local and global
data with HOG features. The system’s efficacy in pedestrian detection is demonstrated by
increased detection rates and decreased false positives obtained through testing with the
PETS 2009 dataset.

6.1.3. Traditional Approaches for Road Lane Detection

In the study in [124], lane detection and sliding-mode control (SMC) were employed
to demonstrate a novel approach for autonomous tracking control in intelligent electric
vehicles (IEVs). This method, incorporating an optimal preview linear quadratic regulator
(OP-LQR) with SMC and a 2-DOF vehicle model, outperforms traditional controllers in
route tracking and lane marking recognition. Additionally, it enables efficient allocation
of unequal braking force to all four wheels, showcasing superiority over conventional
methods. Addressing the challenge of adverse weather conditions, Ref. [125] proposed
a weather-resistant lane recognition method using horizontal optical flow, highlighting
its capability for reliable obstacle assignment and lateral control essential for driver assis-
tance and autonomous driving. Moreover, Ref. [126] presents an adaptive lane marking
identification method for low-light conditions, demonstrating its accuracy and robustness
compared to existing techniques. Similarly, Ref. [127] developed an advanced driver
assistance system (ADAS) focusing on nighttime conditions, showcasing its effectiveness
through successful highway tests. Lastly, Ref. [128] introduced a real-time lane detection
technique for intelligent vehicles, highlighting its performance in complex road conditions
and its potential for improving driving safety.

6.2. DL Approaches for Object Detection in AVs

DL has become much more popular and prominent than regular algorithms in recent
times. This change in direction is attributed to DL’s exceptional work, reliable outcomes,
and breadth of industry experience. Jones and Viola introduced the well-known VJ infrared
technology, which improved immediate detection efficacy and capabilities [84]. In their
research [129], Romero and Antonio mainly concentrated on defining DL methods. Previous
relevant research on OD, e.g., vehicles, pedestrians, and road lanes, is examined in this
section under adverse weather.

6.2.1. DL Approaches for Vehicle Detection

A key element of ITS is vehicle identification, which is used in many applications such
as traffic monitoring, ADAS, AVs, and traffic data. Robustness, acceleration, precision,
and cost-effectiveness are the hallmarks of an intelligent system. For this, a variety of
imaging sources are used, including satellite imagery, in-car cameras, UAV cameras, and
traffic monitoring equipment. This study is mostly focused on on-road vehicle surveillance,
wherein in-car cameras are essential. Nonetheless, there are a variety of difficulties in the
field of vehicle detection. There are several different classifications of vehicles, each having
a distinct size, color, structure, and design. Additionally, they can be viewed at a variety of
scales and positions, frequently in intricate traffic scenarios. Adverse weather conditions
create extra challenges since they distort visibility and add noise to the sensors, which can
result in missed objects and false alarms. This section will examine the major research
works and methodologies that have influenced DL approaches for vehicle detection under
adverse weather conditions, offering a basis for the discussion of novel strategies and
developments in this crucial domain. Table 4 summarizes the previous work on vehicle
detection under adverse weather conditions.
One paper [130] underscores the efficacy of CNNs in image processing through a
tailored methodology for multiclass weather image categorization. Experimentation with
Algorithms 2024, 17, 103 21 of 36

various batch sizes (128, 256, 512, and 1024) is recommended to enhance accuracy and
generalization. Testing on larger datasets enhances classifier accuracy and widens learning
scope. In the realm of two-stage detectors, Ref. [131] introduces a practical approach for
vehicle detection in challenging conditions like fog, low light, and sun glare, leveraging
three trained CNNs (ResNet, GoogleNet, and SqueezeNet) with transfer learning. No-
tably, ResNet achieves a validation accuracy of 100%, while GoogleNet and SqueezeNet
achieve 65.50% and 90.0%, respectively. Additionally, Ref. [132] employs Fast R-CNN
for day and night vehicle detection, yielding impressive results under adverse conditions,
validated by high recall (98.44%), accuracy (94.20%), and precision (90%). Furthermore,
the on-road vehicle detection method proposed in [133] utilizes multiple region proposal
networks (RPNs) in Faster R-CNN to identify vehicles across diverse sizes and weather
scenarios, outperforming the existing techniques with high average precision rates on
various datasets, including DAWN (89.48%), CDNet 2014 (91.20%), and LISA (95.16%). On
the other hand, regarding one-stage detectors, another study [134] introduces an improved
SSD-based front vehicle recognition algorithm for smart automobiles. Tested on the KITTI
dataset, it achieves a high mAP of 92.18% and processes frames quickly at 15 ms per
frame. The system enhances smart car safety in congested traffic and adverse weather
conditions, prioritizing both accuracy and real-time performance. Another study [135]
enhances a YOLO-based algorithm for precise vehicle detection in foggy weather by inte-
grating a dehazing module with multi-scale retinex. This enhanced model, trained with
augmented data, surpasses traditional YOLO in foggy conditions. Additionally, Ref. [136]
proposes a modified YOLO model for on-road vehicle detection and tracking across various
weather conditions. Utilizing a single CNN, it exhibits robustness and outperforms the
existing techniques in intelligent transportation systems. Furthermore, Miao et al. [137]
developed a nighttime vehicle detection method using fine-tuned YOLOv3, outperforming
Faster R-CNN and SSD in terms of accuracy and efficiency. They achieved an average
precision of 93.66%. Another study [138] utilized YOLOv4 with SPP-NET layers for ve-
hicle detection, achieving an 81% mAP. In contrast, the study in [139] focused on harsh
weather conditions, introducing YOLOv4 with an anchor-free and decoupled head, albeit
achieving a 60.3% mAP and focusing exclusively on a single class. Moreover, the goal
of [140] was to enhance self-driving vehicle detection in adverse weather using YOLOv5
with Transformer and CBAM modules, achieving an impressive mAP of 94.7% and FPS of
199.86. The DL approach proposed in [141] for nighttime vehicle detection in autonomous
cars, combining a Generative Adversarial Network for image translation and YOLOv5
for detection, achieved a high accuracy of 96.75%, significantly enhancing the reliability
of AV recognition models for night conditions. This study in [12] presents a DL-based
intelligent AV weather-detecting system. Using a combined dataset from MCWDS2018 and
DAWN2020, the performance of three deep convolutional neural networks was evaluated
by categorizing six weather categories: overcast, rainy, snowy, sandy, sunny, and sunrise.
The CNNs are SqueezeNet, ResNet-50, and EfficientNet-b0. The ResNet-50, EfficientNet-
b0, and SqueezeNet models achieved 98.48%, 97.78%, and 96.05% accuracy, respectively,
in the experiments, demonstrating remarkable accuracy rates while preserving quick in-
ference times using GPU acceleration. By combining previously disparate datasets, the
study’s novel methodology makes a significant addition to DL applications for autonomous
systems’ weather detection.
In simple traffic situations, the aforementioned extraction and classification algorithms
have shown themselves to be successful at detecting vehicles. However, issues including
scale sensitivity, occlusion, and a high number of false positive results limit their efficacy
in more complex traffic conditions [142]. These algorithms perform well in simple traffic
scenarios, but they are limited in complex traffic conditions due to a significant number of
false positive detections. As a result, an explanation connected with vehicle detection is
intended for this part, and the difficulties associated are also highlighted.
Algorithms 2024, 17, 103 22 of 36

Table 4. Vehicle detection previous work with DL.

Ref. Year Technique Weather Classes Accuracy

cloudy, rainy, snowy, sandy, shine,
[12] 2022 SqueezeNet, ResNet-50, EfficientNet-b0 98.48% 97.78% 96.05%
and sunrise
[137] 2020 YOLOv3 night 93.66%
[140] 2022 YOLOv5 Fog, rain, snow, sand 94.7%
[130] 2023 CNN Cloudy, Fog, Rain, Shine Sunrise 98%
[131] 2022 ResNet, GoogleNet, SqueezeNet Fog, Low light, sun glare 100% 65.50% 90.0%
[133] 2021 Faster R-CNN Rain, Fog, Snow, Sand 95.16%
[134] 2020 SSD Sunny, overcast, Night, rain 92.18%
[141] 2022 YOLOv5 night 96.75%

6.2.2. DL Approaches for Pedestrian Detection

A useful deformation model algorithm for managing occlusions is the Deformable
Part Model (DPM), which scores blocks or sections [143]. It prevents objects from obscuring
pedestrians. Classification algorithms use object features to determine which objects are
pedestrians and which are not. A DL- and feature-selection-driven pedestrian detection
system (PDS) is presented in [144]. The way the technology works is that a camera installed
on the car’s dashboard records the surroundings as a video. After that, this video data
are put through a number of preprocessing stages until pedestrians are finally classified
and localized. One of the paper’s key findings is that the composition and quality of the
datasets for training and testing that are used in the detection phase have a direct impact
on the system’s efficacy.
The research presented in [145] demonstrates a significant advancement in nighttime
human detection using visible light cameras, particularly through the application of con-
volutional neural networks (CNNs). The method’s effectiveness is evident in its superior
performance across various contexts, including open and self-constructed databases such as
DNHD-DB1, KAIST, and CVC. This achievement is a notable improvement over previous
techniques and is crucial for the development of intelligent surveillance systems. Another
study [146] addresses the challenges of pedestrian identification in adverse weather condi-
tions by introducing a novel DL network framework. This framework’s ability to adjust to
different lighting conditions through multispectral fusion and illumination perception is
a clear advantage. The integration of cloud computing and the Internet of Vehicles (IoV)
ecosystem further expands its potential for real-time applications in autonomous vehicles,
offering a versatile and adaptable solution. Furthermore, Ref. [147] introduces an image
fusion module (MSRCR-IF) that significantly enhances the Mask R-CNN-based pedes-
trian identification in low-light conditions. The 4.66% improvement in detection accuracy,
reaching 85.05% on a self-built dataset, is a substantial gain over the existing mainstream
techniques. This advancement is particularly valuable for surveillance systems operating
in challenging lighting conditions. Table 5 summarizes the work. Another study [148]
provides valuable insights into the challenges of data collection across diverse meteorologi-
cal conditions and introduces the ZUT dataset, which boasts over 122k annotations. The
research emphasizes the potential benefits of integrating Automobile CAN data with ADAS
systems and suggests temperature-based normalization as a strategy for improvement.
The recommendation to use 16-bit images for a 10.67% increase in detection accuracy and
the consideration of car speed for parameter adjustments are noteworthy. The study also
acknowledges the need for onboard precipitation sensors and a more extensive dataset
that spans multiple seasons to enhance pedestrian detection accuracy, as demonstrated by
the comparison with the SCUT dataset. The study in [149] focuses on metropolitan areas
with a speed restriction of 48–70 km/h and showcases impressive performance on both
rainy and dry roads. The method utilizes the SMP technique for depth calculation and
Algorithms 2024, 17, 103 23 of 36

stereo-vision technology with two cameras. The integration of data from various sensors,
such as radar or LIDAR, is proposed to enhance autonomous driving performance. The
research also advocates for the application of DL methods to refine feature extraction and
categorization, achieving an AP of 89.85% and a mAP of 90% on the CVC-09 and LSIFIR
databases. In the realm of nighttime pedestrian recognition, Ref. [150] introduces an en-
hanced Faster R-CNN architecture, which delivers significant improvements, particularly
for distant pedestrians. The study in [151] tackles the issue of optical camera-based pedes-
trian recognition under adverse weather conditions by employing a cascaded classification
technique that incorporates both local and global features, resulting in increased detection
rates. The effectiveness of this strategy is demonstrated in challenging weather conditions
such as snow or rain. The use of YOLO-v5 in [152] introduces a novel multispectral method
for pedestrian detection, combining independent subnetworks for RGB and IR images
to create an effective detector. The method’s performance on open databases, with high
mAP scores and IoU of 0.5 across various datasets, is encouraging. and another study [153]
evaluates YOLOv7, Faster R-CNN, SSD, and HoG for both vehicle and pedestrian detection
in different weather conditions. YOLOv7’s superior performance in terms of accuracy,
precision, recall, and mAP underscores the critical role of weather conditions in selecting
appropriate DL methods for autonomous driving.

Table 5. Pedestrian detection previous work with DL.

Ref. Year Technique Weather Classes Accuracy

[141] 2023 YOLOv3 several weathers 74.38% (AP)
[145] 2017 CNN night time 98.3%
[147] 2021 Mask R-CNN lowlight 85.05%
[152] 2022 YOLOv5 Day, Night 96.6%
YOLOv7, CNN, Faster 0.73, 0.883, 0.71, 0.657,
[153] 2023 Rain, Fog, Snow, Sand
R-CNN, SSD, HOG 0.70

6.2.3. DL Approaches for Road Lane Detection

A color-based road lane-detecting system was used in research [154–157]. In favorable
weather, Refs. [154,155] achieved above 90% accuracy. However, in adverse weather
conditions, the accuracy considerably decreased. Some researchers used LiDAR technology
for lane detection in order to reduce the impact of ambient influences on the detection
algorithm. With the use of both images and LiDAR data, Shin et al. [158] developed a lane
detection algorithm that combines camera and LiDAR information, improving the capacity
to detect lanes in difficult environmental situations. By combining data from cameras, GPS,
and LiDAR, the authors in [159] were able to develop a precise distance estimation between
the vehicle and lateral lane that was accurate to the centimeter. However, the high cost of
LiDAR and its limited sensitivity to unfavorable weather conditions make it difficult to
apply to large-scale lane detection systems.
A different method [160] uses the HSV color space to achieve an accuracy of 86.21%
regarding the rate of identification. This method works best in daylight or with white light
because it takes advantage of the near-color correspondence between white and yellow lane
markers. Consequently, global thresholding is necessary for precisely dividing color planes
and obtaining lane markers in both the YCbCr and HSV color models. In contrast, in tunnel
environments with distinct color illumination, the lane markers diverge from their real
hues, making reliable detection difficult. Sattar et al. [161] developed a different technique
called “SafeDrive,” which identifies the lane marking in areas with poor lighting. They
located other distinct images of highways at the same place using the vehicle’s location
information, and they used those images to identify the lanes. Other researchers [162] also
carried out an additional study that, because of the solid and easily visible lane markings,
was able to identify lanes in several types of weather conditions.
Algorithms 2024, 17, 103 24 of 36

Using already installed roadside webcams, real-time road weather and road state
detection algorithms were developed in [163]. Transfer learning was utilized to train
detection models utilizing three previously learned CNN architectures. When it came
to accuracy, ResNet18 outperformed the other models, scoring about 97% for weather
and 99% for surface conditions. These models can be useful in improving road safety
since they can automatically recognize and relate real-time circumstances to road networks
without the need for human intervention. They can be incorporated into advanced traveler
information systems (ATIS) for drivers and utilized to optimize winter maintenance routes.
Additionally, the models could replace manual reporting in snowplows, increasing driver
safety and accuracy. Research [164] presents a novel approach that combines computer
vision and DL to extract meteorological data from street-level photos without any image
restrictions. With recognition accuracy ranging from 91% to 95.6% across several categories,
the study presents four deep CNN models designed to identify a variety of visibility
circumstances, including dawn and dusk, daytime, evening, glare, and weather elements
like rain and snow. A study [165] presents a DL-based method for lane marker detection
in order to address the difficulties associated with classifying road markers during rainy
weather. Specifically designed for bad weather, the approach gives priority to the best
feature selection from video frames in order to counteract rain-induced blurriness, leading
to adequate classification precision even in difficult weather circumstances. In other
research [166], two different models for AV lane prediction in highway traffic situations are
presented. The study’s main focus is on the use of AI for lane prediction using a sizable
dataset from NGSIM. To showcase the effectiveness of these models, two distinct subsets
with 5000 car samples each were used. The strategy employed a range of classifiers in
the Identification Learner and different methods in Neural Net Fitting, emphasizing the
methodology’s importance in accomplishing good lane prediction without diving into
particular accuracy measures.
Due to different illumination circumstances, global thresholds for lane marker iden-
tification frequently produce disappointing outcomes. An efficient lane departure (LD)
technique is presented, enabling lane marker identification that may be used in daylight
highway scenarios as well as tunnel situations with artificial lighting. In order to obtain
lane features, the usual LD technique begins by performing pre-processing to eradicate
perspective-related distortion and isolate an ROI. In order to identify suitable lane markers
using color, shape, alignment, or geometrical measurements using the road scenario data,
two categorization methods, model-based and feature-based, are used [17]. To minimize
false positives, prospective lane markers are further refined and validated using curved
or linear lane fitting. This helps with features like adaptive speed control, automatic lane
focusing, and lane departure alert. LD devices may not be as successful due to a number of
issues, such as road obstructions, ambient lighting sources, and noisy weather. To address
these issues, a shadow-resistant methodology that makes use of Fourier transformation
and the maximally stable extreme region (MSER) approach in the blue color stream is used
as a reference [167]. The research findings [168] suggest that the minimal safe separation
between the self-absorbed car and the vehicle should be used to estimate the extraction
of ROI. According to the study, a car traveling at 110 km/h may cover a distance of 35 m
with a height of just 150 pixels. Usually, a predetermined model or set of features is used to
construct lane markers using the ROI.
In [169], the authors modified YOLOv3 and presented a brand-new BGRU-Lane
(BGRU-L) model; the method integrates spatial distribution with visual data. High accuracy
(90.28 mAP) and real-time detection speed (40.20 fps) are achieved in difficult settings
through integration utilizing the Dempster–Shafer algorithm, as demonstrated by datasets
from KIT, Toyota Technological Institute, and Euro Truck Simulator 2.
The dynamic environment of research and innovation in the field of road lane detection
for autonomous cars is shown by our thorough analysis of the literature. The abundance of
information and developing techniques indicates the ongoing commitment to increasing
the potential of self-governing systems.
Algorithms 2024, 17, 103 25 of 36

6.3. Practical Implications

The practical implementation of DL models for pedestrian and vehicle detection in AVs
is a critical area of research as it directly impacts the safety and reliability of these systems.
Studies such as Kim et al. [145] have demonstrated the effectiveness of CNNs in low-light
conditions, a significant advancement for nighttime AV navigation. Lai et al. [147] have
further optimized the Mask R-CNN algorithm for low-light environments, enhancing
detection rates and computational efficiency. Montenegro et al. [152] have shown that
YOLO-v5 can be fine-tuned to maintain high detection accuracy across varying lighting
conditions, ensuring consistent performance for AVs. Zaman et al. [153] have addressed
the challenge of adverse weather conditions, a common obstacle for AVs, by developing DL
models that can adapt to weather-induced distortions, thereby improving system reliability.
These studies have identified key performance indicators (KPIs) such as detection accuracy,
false negative rate, computational efficiency, and real-time processing capabilities, which
are essential for evaluating the practicality of DL models in AVs, as shown in Table 6. The
challenges faced, including limited visibility and dynamic lighting conditions, have been
mitigated through model optimization, data augmentation, and sensor fusion techniques.
The insights from these studies are invaluable for AV developers as they provide a roadmap
for creating systems that can operate effectively in a wide range of environmental conditions,
bringing us closer to a future where autonomous vehicles are a safe and integral part of our
transportation infrastructure.

Table 6. Practical implications of DL approaches.

Ref. Approach Practical Implications KPIs Challenges Addressing Challenges

Enhances pedestrian Utilizes CNNs trained
Detection Accuracy,
detection in low-light Limited visibility, on nighttime images to
CNN-based Human False Negative Rate,
[145] conditions, improving background noise recognize pedestrian
Detection at Nighttime Computational
safety and navigation interference features, potentially
Efficiency
for AVs. reducing false negatives.
Adjusts model
Tailors the Mask R-CNN
Optimization for parameters, uses data
Optimized Mask R-CNN algorithm for low-light Detection Accuracy,
low-light conditions, augmentation, and
[147] for Low-Light environments, Precision, Recall,
computational employs hardware
Pedestrian Detection maintaining high Real-time Processing
complexity acceleration for real-time
detection rates.
processing.
Fine-tunes the YOLO-v5
Demonstrates the
model on diverse
YOLO-v5 for Pedestrian versatility of YOLO-v5 Balancing speed with
Detection Speed, datasets, ensuring
[152] Detection in Daytime across different lighting accuracy in
Accuracy, Robustness generalization across
and Nighttime conditions, ensuring varying lighting
different lighting
consistent performance.
scenarios.
Uses DL models that
Improves detection learn to recognize
DL Approaches for Detection Accuracy, Degradation of
reliability in adverse patterns despite weather
[153] Adverse Weather Robustness, System performance due to
weather, crucial for AV distortions and integrate
Detection Reliability weather conditions
safety and functionality. multi-sensor data for
enhanced detection.

7. Discussion, Limitations, and Future Research Trends

7.1. Discussion
Recent literature highlights the increasing use of DL approaches for OD, yielding
promising results under typical conditions. However, there is a pressing need for further
advancements, particularly in adverse weather and complex scenarios. Enhancing OD
in such conditions is crucial, especially for AVs, to prevent accidents and ensure safety.
In the following discussion, we delve into the key insights and implications drawn from
our research on OD in adverse weather conditions for AVs. Our research has explored
both traditional and DL methods for AV object detection, focusing on vehicles, pedestrians,
and road lanes. Traditional methods, despite their foundational role, struggle with high
Algorithms 2024, 17, 103 26 of 36

computational demands, slow processing, and occasional misidentification. DL, on the

other hand, excels by learning complex patterns from data, offering faster and more precise
detection, especially in challenging weather conditions. This makes DL a more effective and
adaptable solution for AV systems. Traditional vehicle detection techniques, which rely on
appearance- and motion-based algorithms, face significant challenges in adverse weather
conditions. Appearance-based algorithms like HOG, Haar-like, and SIFT are sensitive
to exterior objects, adverse weather, and lighting variations, making them vulnerable in
situations like heavy rain or fog. These algorithms often struggle to maintain accuracy due
to the reduced visibility of vehicle features and the difficulty in differentiating vehicles
from their surroundings. Motion-based detection methods, which track objects based on
movement relative to the camera, also encounter issues in adverse weather. They can have
trouble distinguishing between moving vehicles and other dynamic elements, such as
pedestrians or debris, especially when motion cues are obscured by weather conditions. DL
approaches are more accurate and perform better, as shown in Tables 4 and 5. A comparison
of traditional and DL approaches is shown in Table 7.

Table 7. Comparison of traditional and DL approaches.

Ref. Approaches Technique Superiority Limitations Additional Considerations

Research into adaptive
Limited in adverse weather,
color models or fusion with
Color-based Lane Simple to implement, low reliance on good weather
[154,155] Traditional other sensor data could
Detection computational cost conditions for high
improve performance in
accuracy
challenging conditions.
Cost reduction and
Reduces reliance on visual High cost, sensitivity to miniaturization of LiDAR
[158,159] LiDAR Integration Traditional data, robust in various unfavorable weather sensors could make this
weather conditions. technology more accessible
for widespread use.
Requires global Enhancing the color model
Works well in daylight or thresholding, less effective with machine learning
[160] HSV Color Space Traditional with white light; uses color in tunnel environments could improve its
correspondence with distinct color adaptability to different
illumination. lighting conditions.
Incorporating real-time
Utilizes historical data and May not generalize well to
weather data and vehicle
[161] SafeDrive Technique Traditional vehicle location for lane all environments, relies on
dynamics could improve
detection in low-light areas available historical data.
the technique’s robustness.
High accuracy with Continuous learning and
Requires pre-trained
CNN-based Real-time transfer learning automates model updating could help
[163] DL models, may not adapt
Road Condition Detection real-time condition maintain high accuracy as
quickly to new conditions
recognition conditions change.
Using smaller, more
Prioritizes feature selection May require extensive specialized models could
Lane Marker Detection in
[165] DL to counteract rain-induced training data, complex reduce computational
Rainy Weather
blurriness model architecture demands and improve
real-time performance.
Research into GANs for
Requires significant
Can generate realistic data adverse weather
Generative Adversarial computational resources,
[141] DL for training, improve simulation could provide
Networks (GANs) may struggle with certain
feature extraction more robust training data
types of adverse weather.
for DL models.
Further development of
Combines different Requires specialized multispectral imaging
[152] Multispectral Imaging DL imaging modalities for hardware; may be complex techniques could lead to
improved detection to integrate more reliable detection
systems.
Fine-tuning transfer
May not perform as well as
Allows models to adapt to learning models for specific
[163] Transfer Learning DL task-specific models if the
new tasks with fewer data AV tasks could enhance
transfer is not well-aligned.
their effectiveness.
Algorithms 2024, 17, 103 27 of 36

Figure 13 presents the overall percentage distribution of papers, while Figure 14

describes the papers of traditional and DL approaches related to the studied three object
detection issues. From Figure 13, one can see that vehicle detection is frequently studied
using traditional and DL approaches, followed by pedestrian and road lane detection.
As shown in Figure 14, DL approaches were more frequently used for all three issues
in AVs compared to traditional approaches. Prior to 2008, traditional feature extraction
methods were prevalent for detection and classification but had limitations in adverse
conditions. Manual feature extraction made them less suitable for complex applications.
DL has become much more popular and prominent than regular algorithms in recent
times. This change in direction is attributed to DL’s exceptional work, reliable outcomes,
and breadth of industry experience. Romero and Antonio [129] also mainly concentrated
on defining DL methods. Peer review has been applied to DL one-stage and two-stage
detectors from the investigations. Currently, the leading methods are YOLOv8 from the
YOLO series and Faster R-CNN from the R-CNN family, renowned for their superior
accuracy and performance. Figure 15 shows the number of summarized papers for one-
stage and two-stage detectors, which shows that one-stage detectors are frequently used
for vehicle detection, while, for pedestrian detection, both are equally applied. Two-stage
detectors are frequently used for road lanes compared with single-stage detectors.

Vehicles Pedestrians Road Lanes

32% 40%

28%

Figure 13. Overall percentage distribution of papers.

25

20
Number of Papers

15

10

0
Traditional Approaches DL Approaches
Vehicles Pedestrians Road Lanes

Figure 14. Distribution of papers for traditional and DL approaches.

Algorithms 2024, 17, 103 28 of 36

One-Stage detectors Two-Stage detectors Both detectors

16

14 2

12

Number of Papers
4
10

2
6

4
9 3

2
1
3
2
0

Vehicles Pedestrians Road Lanes

Figure 15. Paper distribution for one-stage and two-stage detectors.

In order to improve OD, Faster R-CNN was integrated into the RPN. This resulted
in shorter evaluation times, but it also required a significant amount of processing power
and had difficulties maintaining consistent performance for small and variously scaled
objects.YOLOv8 is designed to cater to specific requirements. The YOLOv8 structure
comprises backbone, neck, and head modules, enhancing feature extraction with C2f and
employing the PAN feature fusion technique. It also adopts a decoupled head architecture
and introduces an anchor-free strategy for OD. Furthermore, we analyzed high-quality
datasets related to AVs, and some standard evaluation metrics were also reviewed and
summarized. Our discussion underscores the advancements in OD for autonomous vehicles
under adverse weather, paving the way for enhanced safety and reliability in ADs.

7.2. Limitations
1. Our literature review primarily focuses on the detection of pedestrians, vehicles, and
road lanes, which may not encompass all possible objects and scenarios relevant to
AVs in adverse weather conditions. There may be other critical elements, such as
traffic signals or animals, that warrant further investigation.
2. The detection algorithms discussed in our review may have inherent limitations, such
as difficulties in detecting small or occluded objects, which could impact the accuracy
and reliability of AVs in certain situations.
3. Our study primarily considers a range of adverse weather conditions, but there may
be other environmental factors, such as dust storms or heavy fog, that were not
extensively covered in the reviewed literature.
4. The field of AV technology is rapidly evolving, and new detection algorithms and
sensor technologies may emerge that could significantly impact the findings of our
review. Our study may not capture the most recent advancements in this area.
5. While our study includes datasets that simulate adverse weather conditions, the
simulation environments may not perfectly replicate real-world driving scenarios.
The complexity of real-world conditions, including unpredictable human behavior
and dynamic traffic patterns, can introduce additional challenges not fully captured
in simulated datasets.
6. The ethical considerations and societal acceptance of AVs, especially in challenging
conditions, are not addressed in our study. Public trust and the ethical use of AV
technology are essential factors for their successful integration into smart cities.
Algorithms 2024, 17, 103 29 of 36

7.3. Future Research Trends

1. It has become more important to address the real-time requirements for OD in real-
world applications. Deep neural networks, however, frequently require large amounts
of computational power, which presents difficulties for embedded systems. To prop-
erly fulfill these objectives, resource-efficient technique development has become
essential. To ensure the practical usefulness of the suggested methodologies, future
research should focus heavily on their computational components, offering both
quantitative and qualitative analysis.
2. The existing deep neural network techniques for difficult item detection mainly
depend on large-scale annotated datasets. However, creating these databases is
expensive and time-consuming. Consequently, there is an urgent need to create OD
algorithms that can train with little to no labeled information.
3. The employment of various evaluation metrics and IoU criteria for OD in difficult
situations has resulted in the absence of a clear benchmark standard, which makes
comparing methods difficult. For future research in this area to be uniform and
comparable, a global baseline must be established.
4. Creation of extensive simulation environments should occur that imitate inclement
weather to thoroughly test and improve object identification algorithms.
5. To develop comprehensive solutions for adverse weather OD, researchers, engineers,
and policymakers should collaborate more closely.
6. It is necessary to study the psychology and behavior of human drivers in adverse
weather, with an emphasis on developing successful communication and trust with AVs.
7. Creation of novel tactics for the real-time modification of OD algorithms for AVs in
response to changing environmental circumstances should be achieved.
8. Investigation into cutting-edge techniques for combining current weather data with
weather forecasts to enable proactive decisionmaking during unfavorable weather
conditions is necessary.
9. Improvements in sensor fusion methods should be attained, which integrate information
from several sensor types to provide more accurate and dependable identification in
adverse weather.
10. To develop behavior prediction models for AVs, leveraging machine learning and
deep learning to forecast the actions of vehicles, pedestrians, and cyclists should
occur. These models will operate effectively in adverse weather, improving AV
decisionmaking for enhanced safety and efficiency.

8. Conclusions
This paper reviewed the traditional and DL approaches for vehicles, pedestrians,
and road lane detection in AVs under adverse weather conditions. We first studied the
architecture of AVs with sensor technologies and other components and also discussed
the challenges for AVs in adverse weather. After an overview of almost all the datasets
related to AVs covering different weather conditions, we explained the basic structure of
OD and the evaluation matrices used for it. Then, we explained the traditional approaches
and DL approaches and discussed the traditional feature extraction methods that were
prevalent for detection and classification but had limitations in adverse conditions. Manual
feature extraction made them less suitable for complex applications. DL has become
much more popular and prominent than regular algorithms in more recent times. DL
approaches explain the structure of YOLOV8 (one-stage detectors) and Faster R-CNN (two-
stage detectors). Two-stage algorithms tend to deliver superior accuracy, while one-stage
algorithms offer faster processing speeds. Notably, the reasons behind the lower accuracy
of one-stage algorithms remain unclear. In addition, the statistics about the status quo of
traditional and DL approaches for OD in AVs were provided based on the works collected
in this survey paper. We found that DL was intensively used for vehicles, pedestrians,
and road lane detection in AVs compared with the traditional approaches. Specifically,
one-stage detectors were frequently used for vehicle detection compared with two-stage
Algorithms 2024, 17, 103 30 of 36

detectors. In addition, vehicle detection was frequently studied using both traditional and
DL approaches followed by road lane and pedestrian detection. Finally, we presented a
useful discussion along with some future research directions.

Author Contributions: All authors have equally contributed. All authors have read and agreed to
the published version of the manuscript.
Funding: This work was supported by EIAS Data Science Lab, CCIS, Prince Sultan University and
also by the Special Research Fund (BOF) of Hasselt University (No. BOF23DOCBL11), and Chen
Junhong was sponsored by the China Scholarship Council (No. 202208440309).
Acknowledgments: The authors would like to thank Prince Sultan University for their support.
Conflicts of Interest: The authors declare no conflicts of interest.

References
1. World Health Organization. WHO: Global Status Report on Road Safety: Summary; Technical Report. Available online:
https://www.who.int/health-topics/road-safety#tab=tab_1 (accessed on 18 February 2024).
2. Iftikhar, S.; Asim, M.; Zhang, Z.; El-Latif, A.A.A. Advance generalization technique through 3D CNN to overcome the false
positives pedestrian in autonomous vehicles. Telecommun. Syst. 2022, 80, 545–557. [CrossRef]
3. Self-Driving Cars Global Market Size. Available online: https://precedenceresearch.com/self-driving-cars-market (accessed on
18 December 2023).
4. Badue, C.; Guidolini, R.; Carneiro, R.V.; Azevedo, P.; Cardoso, V.B.; Forechi, A.; Jesus, L.; Berriel, R.; Paixao, T.M.; Mutz, F.; et al.
Self-driving cars: A survey. Expert Syst. Appl. 2021, 165, 113816. [CrossRef]
5. Bansal, P.; Kockelman, K.M. Are we ready to embrace connected and self-driving vehicles? A case study of Texans. Transportation
2018, 45, 641–675. [CrossRef]
6. Chehri, A.; Sharma, T.; Debaque, B.; Duclos, N.; Fortier, P. Transport systems for smarter cities, a practical case applied to traffic
management in the city of Montreal. In Sustainability in Energy and Buildings 2021; Springer: Singapore, 2022; pp. 255–266.
7. Trenberth, K.E.; Zhang, Y. How often does it really rain? Bull. Am. Meteorol. Soc. 2018, 99, 289–298. [CrossRef]
8. Andrey, J.; Yagar, S. A temporal analysis of rain-related crash risk. Accid. Anal. Prev. 1993, 25, 465–472. [CrossRef]
9. National Oceanic and Atmospheric Administration, Getting Traction: Tips for Traveling in Winter Weather. Available online:
https://www.weather.gov/wrn/getting_traction (accessed on 24 October 2023).
10. Mehra, A.; Mandal, M.; Narang, P.; Chamola, V. ReViewNet: A fast and resource optimized network for enabling safe autonomous
driving in hazy weather conditions. IEEE Trans. Intell. Transp. Syst. 2020, 22, 4256–4266. [CrossRef]
11. Abu Al-Haija, Q.; Krichen, M.; Abu Elhaija, W. Machine-learning-based darknet traffic detection system for IoT applications.
Electronics 2022, 11, 556. [CrossRef]
12. Al-Haija, Q.A.; Gharaibeh, M.; Odeh, A. Detection in adverse weather conditions for autonomous vehicles via deep learning. AI
2022, 3, 303–317. [CrossRef]
13. Zang, S.; Ding, M.; Smith, D.; Tyler, P.; Rakotoarivelo, T.; Kaafar, M.A. The impact of adverse weather conditions on autonomous
vehicles: How rain, snow, fog, and hail affect the performance of a self-driving car. IEEE Veh. Technol. Mag. 2019, 14, 103–111.
[CrossRef]
14. Mohammed, A.S.; Amamou, A.; Ayevide, F.K.; Kelouwani, S.; Agbossou, K.; Zioui, N. The perception system of intelligent
ground vehicles in all weather conditions: A systematic literature review. Sensors 2020, 20, 6532. [CrossRef]
15. Yoneda, K.; Suganuma, N.; Yanase, R.; Aldibaja, M. Automated driving recognition technologies for adverse weather conditions.
IATSS Res. 2019, 43, 253–262. [CrossRef]
16. Zhang, Y.; Carballo, A.; Yang, H.; Takeda, K. Perception and sensing for autonomous vehicles under adverse weather conditions:
A survey. ISPRS J. Photogramm. Remote Sens. 2023, 196, 146–177. [CrossRef]
17. Arthi, V.; Murugeswari, R.; Nagaraj, P. Object Detection of Autonomous Vehicles under Adverse Weather Conditions. In
Proceedings of the 2022 International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI), Chennai, India,
8–10 December 2022; Volume 1, pp. 1–8.
18. Hnewa, M.; Radha, H. Object detection under rainy conditions for autonomous vehicles: A review of state-of-the-art and
emerging techniques. IEEE Signal Process. Mag. 2020, 38, 53–67. [CrossRef]
19. Abbas, A.F.; Sheikh, U.U.; Al-Dhief, F.T.; Mohd, M.N.H. A comprehensive review of vehicle detection using computer vision.
TELKOMNIKA Telecommun. Comput. Electron. Control 2021, 19, 838–850. [CrossRef]
20. Yang, Z.; Pun-Cheng, L.S. Vehicle detection in intelligent transportation systems and its applications under varying environments:
A review. Image Vis. Comput. 2018, 69, 143–154. [CrossRef]
21. Muneeswaran, V.; Nagaraj, P.; Rajasekaran, M.P.; Reddy, S.U.; Chaithanya, N.S.; Babajan, S. IoT based Multiple Vital Health
Parameter Detection and Analyzer System. In Proceedings of the 2022 7th International Conference on Communication and
Electronics Systems (ICCES), Coimbatore, India, 22–24 June 2022; pp. 473–478. [CrossRef]
22. SAE International. Available online: https://www.sae.org/standards/content/j3016_202104/ (accessed on 11 November 2023).
Algorithms 2024, 17, 103 31 of 36

23. Bimbraw, K. Autonomous cars: Past, present and future a review of the developments in the last century, the present scenario and
the expected future of autonomous vehicle technology. In Proceedings of the 2015 12th International Conference on Informatics
in Control, Automation and Robotics (ICINCO), Colmar, France, 21–23 July 2015; Volume 1, pp. 191–198.
24. Lafrance, A. Your Grandmother’s Driverless Car. The Atlantic, 29 June 2016. Available online: https://www.theatlantic.com/
technology/archive/2016/06/beep-beep/489029/ (accessed on 27 October 2023).
25. Pomerleau, D.A. Alvinn: An autonomous land vehicle in a neural network. In NIPS’88: Proceedings of the 1st International
Conference on Neural Information Processing Systems, 1 January 1988; MIT Press: Cambridge, MA, USA; pp. 305–313.
26. SAE International. Surface Vehicle Recommended Practice. Joint SAE/RCCC Fuel Consumption Test Procedure (Short Term
in-Service Vehicle) Type I. 1986. Available online: https://ca-times.brightspotcdn.com/54/02/2d5919914cfe9549e79721b12e66/j3
016-202104.pdf (accessed on 1 May 2021)
27. How Google’s Autonomous Car Passed the First US State Self-Driving Test. 2014. Available online: https://spectrum.ieee.org/
how-googles-autonomous-car-passed-the-first-us-state-selfdriving-test (accessed on 31 December 2023).
28. Claire, P. The Pathway to Driverless Cars: Summary Report and Action Plan; OCLC: London, UK, 2015.
29. Briefs, U. Mcity Grand Opening. Res. Rev. 2015, 46, 1–3.
30. Bellone, M.; Ismailogullari, A.; Müür, J.; Nissin, O.; Sell, R.; Soe, R.M. Autonomous driving in the real-world: The weather
challenge in the Sohjoa Baltic project. In Towards Connected and Autonomous Vehicle Highways: Technical, Security and Social
Challenges; Springer International Publishing: Cham, Switzerland, 2021; pp. 229–255.
31. Lambert, F. Watch Tesla Autopilot Go through a Snowstorm. Available online: https://electrek.co/2019/01/28/tesla-autopilot-
snow-storm/ (accessed on 2 January 2023).
32. EUR-Lex. E.L. EUR-Lex-32019R2144-EN-EUR-Lex. 2014. Available online: https://eur-lex.europa.eu/legal-content/en/ALL/
?uri=CELEX%3A32019R2144 (accessed on 13 October 2023).
33. Waymo, W. Waymo Safety Report. 2020. Available online: https://waymo.com/safety/ (accessed on 18 August 2023).
34. Gehrig, S.K.; Stein, F.J. Dead reckoning and cartography using stereo vision for an autonomous car. In Proceedings of the 1999
IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High
Intelligence and Emotional Quotients (Cat. No.99CH36289), Kyongju, Republic of Korea, 17–21 October 1999; Volume 3, pp.
1507–1512. [CrossRef]
35. Rawashdeh, N.A.; Bos, J.P.; Abu-Alrub, N.J. Drivable path detection using CNN sensor fusion for autonomous driving in the
snow. In Proceedings of the SPIE 11748, Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure
2021, Online, FL, USA, 12–17 April 2021; Volume 1174806. [CrossRef]
36. Thrun, S.; Montemerlo, M.; Dahlkamp, H.; Stavens, D.; Aron, A.; Diebel, J.; Fong, P.; Gale, J.; Halpenny, M.; Hoffmann, G.; et al.
Stanley: The robot that won the DARPA Grand Challenge. J. Field Robot. 2006, 23, 661–692. [CrossRef]
37. Carballo, A.; Lambert, J.; Monrroy, A.; Wong, D.; Narksri, P.; Kitsukawa, Y.; Takeuchi, E.; Kato, S.; Takeda, K. LIBRE: The multiple
3D LiDAR dataset. In Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020; pp. 1094–1101.
[CrossRef]
38. Velodyne, HDL-64E Spec Sheet. Available online: https://velodynesupport.zendesk.com/hc/en-us/articles/115003632634
-HDL-64E-Spec-Sheet (accessed on 13 October 2023).
39. Bijelic, M.; Gruber, T.; Ritter, W. A benchmark for lidar sensors in fog: Is detection breaking down? In Proceedings of the
Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 760–767. [CrossRef]
40. Patole, S.M.; Torlak, M.; Wang, D.; Ali, M. Automotive radars: A review of signal processing techniques. IEEE Signal Process.
Mag. 2017, 34, 22–35. [CrossRef]
41. Navtech Radar, FMCW Radar. Available online: https://navtechradar.com/explore/fmcw-radar/ (accessed on 22 November 2021).
42. Carullo, A.; Parvis, M. An ultrasonic sensor for distance measurement in automotive applications. IEEE Sens. J. 2001, 1, 143.
[CrossRef]
43. Frenzel, L. Ultrasonic Sensors: A Smart Choice for Shorter-Range Applications. 2018. Available online: https:
//www.electronicdesign.com/industrialautomation/article/21806202/ultrasonic-sensors-a-smart-choicefor-shorterrange-
applications (accessed on 14 August 2023).
44. Kamemura, T.; Takagi, H.; Pal, C.; Ohsumi, A. Development of a long-range ultrasonic sensor for automotive application. SAE
Int. J. Passeng. Cars-Electron. Electr. Syst. 2008, 1, 301–306. [CrossRef]
45. Tesla, Summon Your Tesla from Your Phone. Available online: https://www.tesla.com/blog/summon-your-tesla-your-phone
(accessed on 2 November 2023).
46. Pendleton, S.D.; Andersen, H.; Du, X.; Shen, X.; Meghjani, M.; Eng, Y.H.; Rus, D.; Ang, M.H., Jr. Perception, planning, control,
and coordination for autonomous vehicles. Machines 2017, 5, 6. [CrossRef]
47. Nasseri, A.; Shlomit, H. Autonomous Vehicle Technology Report. 2020. Available online: https://www.wevolver.com/article/
(accessed on 28 October 2023).
48. Rosique, F.; Navarro, P.J.; Fernández, C.; Padilla, A. A systematic review of perception system and simulators for autonomous
vehicles research. Sensors 2019, 19, 648. [CrossRef] [PubMed]
49. Paden, B.; Čáp, M.; Yong, S.Z.; Yershov, D.; Frazzoli, E. A survey of motion planning and control techniques for self-driving
urban vehicles. IEEE Trans. Intell. Veh. 2016, 1, 33–55. [CrossRef]
Algorithms 2024, 17, 103 32 of 36

50. Shladover, S.E.; Desoer, C.A.; Hedrick, J.K.; Tomizuka, M.; Walrand, J.; Zhang, W.B.; McMahon, D.H.; Peng, H.; Sheikholeslam,
S.; McKeown, N. Automated vehicle control developments in the PATH program. IEEE Trans. Veh. Technol. 1991, 40, 114–130.
[CrossRef]
51. Geiger, A.; Lenz, P.; Stiller, C.; Urtasun, R. Vision meets robotics: The kitti dataset. Int. J. Robot. Res. 2013, 32, 1231–1237.
[CrossRef]
52. Sun, H.; Zhang, W.; Yu, R.; Zhang, Y. Motion planning for mobile robots—Focusing on deep reinforcement learning: A systematic
review. IEEE Access 2021, 9, 69061–69081. [CrossRef]
53. Rai, M.; Khosla, B.; Dhawan, Y.; Kharotia, H.; Kumar, N.; Bandi, A. CLEAR: An Efficient Traffic Sign Recognition Technique for
Cyber-Physical Transportation Systems. In Proceedings of the 2022 IEEE/ACM 15th International Conference on Utility and
Cloud Computing (UCC), Vancouver, WA, USA, 6 December 2022; pp. 418–423.
54. Sakaridis, C.; Dai, D.; Van Gool, L. ACDC: The adverse conditions dataset with correspondences for semantic driving scene
understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17
October 2021; pp. 10765–10775.
55. Geyer, J.; Kassahun, Y.; Mahmudi, M.; Ricou, X.; Durgesh, R.; Chung, A.S.; Hauswald, L.; Pham, V.H.; Mühlegg, M.; Dorn, S.; et al.
A2d2: Audi autonomous driving dataset. arXiv 2020, arXiv:2004.06320.
56. Sun, P.; Kretzschmar, H.; Dotiwalla, X.; Chouard, A.; Patnaik, V.; Tsui, P.; Guo, J.; Zhou, Y.; Chai, Y.; Caine, B.; et al. Scalability in
perception for autonomous driving: Waymo open dataset. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision
and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 2443–2451. [CrossRef]
57. Caesar, H.; Bankiti, V.; Lang, A.H.; Vora, S.; Liong, V.E.; Xu, Q.; Krishnan, A.; Pan, Y.; Baldan, G.; Beijbom, O. nuscenes: A
multimodal dataset for autonomous driving. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern
Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 11621–11631. [CrossRef]
58. Yu, F.; Chen, H.; Wang, X.; Xian, W.; Chen, Y.; Liu, F.; Madhavan, V.; Darrell, T. Bdd100k: A diverse driving dataset for
heterogeneous multitask learning. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 2636–2645. [CrossRef]
59. Huang, X.; Wang, P.; Cheng, X.; Zhou, D.; Geng, Q.; Yang, R. The apolloscape open dataset for autonomous driving and its
application. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 42, 2702–2719. [CrossRef] [PubMed]
60. Ros, G.; Sellart, L.; Materzynska, J.; Vazquez, D.; Lopez, A.M. The synthia dataset: A large collection of synthetic images for
semantic segmentation of urban scenes. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), Las Vegas, NV, USA, 26 June–1 July 2016; pp. 3234–3243. [CrossRef]
61. Carlevaris-Bianco, N.; Ushani, A.K.; Eustice, R.M. University of Michigan North Campus long-term vision and lidar dataset. Int.
J. Robot. Res. 2016, 35, 1023–1035. [CrossRef]
62. Pitropov, M.; Garcia, D.E.; Rebello, J.; Smart, M.; Wang, C.; Czarnecki, K.; Waslander, S. Canadian adverse driving conditions
dataset. Int. J. Robot. Res. 2021, 40, 681–690. [CrossRef]
63. Zheng, J.Y. IUPUI driving videos and images in all weather and illumination conditions. arXiv 2021, arXiv:2104.08657.
64. Bos, J.P.; Chopp, D.; Kurup, A.; Spike, N. Autonomy at the end of the earth: An inclement weather autonomous driving data set.
In Proceedings of the Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2020, Online, CA,
USA, 27 April–9 May 2020; Volume 11415, pp. 36–48. [CrossRef]
65. Vora, S.; Lang, A.H.; Helou, B.; Beijbom, O. Pointpainting: Sequential fusion for 3d object detection. In Proceedings of the 2020
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 4603–4611.
[CrossRef]
66. Chang, M.F.; Lambert, J.; Sangkloy, P.; Singh, J.; Bak, S.; Hartnett, A.; Wang, D.; Carr, P.; Lucey, S.; Ramanan, D.; et al. Argoverse:
3d tracking and forecasting with rich maps. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern
Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019; pp. 8748–8757. [CrossRef]
67. Meyer, M.; Kuschk, G. Automotive radar dataset for deep learning based 3d object detection. In Proceedings of the 2019 16th
European Radar Conference (EuRAD), Paris, France, 2–4 October 2019; pp. 129–132.
68. Bijelic, M.; Gruber, T.; Mannan, F.; Kraus, F.; Ritter, W.; Dietmayer, K.; Heide, F. Seeing through fog without seeing fog: Deep
multimodal sensor fusion in unseen adverse weather. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and
Pattern Recognition (CVPR), Seattle, WA, USA, 15–20 June 2020; pp. 11682–11692.
69. Sakaridis, C.; Dai, D.; Van Gool, L. Semantic foggy scene understanding with synthetic data. Int. J. Comput. Vis. 2018, 126, 973–992.
[CrossRef]
70. Kim, J.; Rohrbach, A.; Darrell, T.; Canny, J.; Akata, Z. Textual Explanations for Self-Driving Vehicles. In Computer Vision–ECCV
2018; Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y., Eds.; Lecture Notes in Computer Science; Springer: Cham, Switzerland,
2018; Volume 11206.
71. Neuhold, G.; Ollmann, T.; Rota Bulo, S.; Kontschieder, P. The mapillary vistas dataset for semantic understanding of street
scenes. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017;
pp. 4990–4999.
72. Braun, M.; Krebs, S.; Flohr, F.; Gavrila, D.M. Eurocity persons: A novel benchmark for person detection in traffic scenes. IEEE
Trans. Pattern Anal. Mach. Intell. 2019, 41, 1844–1861. [CrossRef]
Algorithms 2024, 17, 103 33 of 36

73. Maddern, W.; Pascoe, G.; Linegar, C.; Newman, P. 1 year, 1000 km: The oxford robotcar dataset. Int. J. Robot. Res. 2017, 36, 3–15.
[CrossRef]
74. Pham, Q.H.; Sevestre, P.; Pahwa, R.S.; Zhan, H.; Pang, C.H.; Chen, Y.; Mustafa, A.; Chandrasekhar, V.; Lin, J. A 3D dataset:
Towards autonomous driving in challenging environments. In Proceedings of the 2020 IEEE International Conference on Robotics
and Automation (ICRA), Paris, France, 31 May–31 August 2020; pp. 2267–2273.
75. Wenzel, P.; Wang, R.; Yang, N.; Cheng, Q.; Khan, Q.; von Stumberg, L.; Zeller, N.; Cremers, D. 4Seasons: A cross-season dataset
for multi-weather SLAM in autonomous driving. In Pattern Recognition; Akata, Z., Geiger, A., Sattler, T., Eds.; DAGM GCPR 2020.
Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2021; Volume 12544, pp. 404–417.
76. Zendel, O.; Honauer, K.; Murschitz, M.; Steininger, D.; Dominguez, G.F. Wilddash-creating hazard-aware benchmarks. In Pro-
ceedings of the Computer Vision–ECCV 2018: 15th European Conference, Munich, Germany, 8–14 September 2018; Proceedings,
Part VI, pp. 402–416.
77. Choi, Y.; Kim, N.; Hwang, S.; Park, K.; Yoon, J.S.; An, K.; Kweon, I.S. KAIST multi-spectral day/night data set for autonomous
and assisted driving. IEEE Trans. Intell. Transp. Syst. 2018, 19, 934–948. [CrossRef]
78. Sheeny, M.; De Pellegrin, E.; Mukherjee, S.; Ahrabian, A.; Wang, S.; Wallace, A. RADIATE: A radar dataset for automotive
perception in bad weather. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an,
China, 30 May–5 June 2021; pp. 1–7.
79. Yan, Z.; Sun, L.; Krajník, T.; Ruichek, Y. EU long-term dataset with multiple sensors for autonomous driving. In Proceedings of
the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24 October 2020–24
January 2021; pp. 10697–10704.
80. Burnett, K.; Yoon, D.J.; Wu, Y.; Li, A.Z.; Zhang, H.; Lu, S.; Qian, J.; Tseng, W.K.; Lambert, A.; Leung, K.Y.; et al. Boreas: A
multi-season autonomous driving dataset. Int. J. Robot. Res. 2023, 42, 33–42. [CrossRef]
81. Kenk, M.A.; Hassaballah, M. DAWN: Vehicle detection in adverse weather nature dataset. arXiv 2020, arXiv:2008.05402.
82. Saralajew, S.; Ohnemus, L.; Ewecker, L.; Asan, E.; Isele, S.; Roos, S. A dataset for provident vehicle detection at night. In
Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic,
27 September–1 October 2021; pp. 9750–9757.
83. Lowe, D.G. Object recognition from local scale-invariant features. In Proceedings of the Seventh IEEE International Conference
on Computer Vision, Kerkyra, Greece, 20–25 September 1999; Volume 2, pp. 1150–1157.
84. Viola, P.; Jones, M. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer
Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Kauai, HI, USA, 8–14 December 2001. [CrossRef]
85. Dalal, N.; Triggs, B. Histograms of oriented gradients for human detection. In Proceedings of the 2005 IEEE Computer Society
Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 20–26 June 2005; Volume 1, pp. 886–893.
86. Asim, M.; Wang, Y.; Wang, K.; Huang, P.Q. A review on computational intelligence techniques in cloud and edge computing.
IEEE Trans. Emerg. Top. Comput. Intell. 2020, 4, 742–763. [CrossRef]
87. LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998,
86, 2278–2324. [CrossRef]
88. Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation.
In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014;
pp. 580–587.
89. Girshick, R. Fast r-cnn. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile,
7–13 December 2015; pp. 1440–1448. [CrossRef]
90. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans.
Pattern Anal. Mach. Intell. 2016, 39, 1137–1149. [CrossRef]
91. He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans.
Pattern Anal. Mach. Intell. 2015, 37, 1904–1916. [CrossRef]
92. Lin, T.Y.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature pyramid networks for object detection. In
Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July
2017; pp. 936–944. [CrossRef]
93. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Berg, A.C. Ssd: Single shot multibox detector. In Computer Vision–
ECCV 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Lecture Notes in Computer Science; Springer: Cham, Switzerland,
2016; Volume 9905, pp. 21–37.
94. Terven, J.; Cordova-Esparza, D. A comprehensive review of YOLO: From YOLOv1 to YOLOv8 and beyond. arXiv 2023,
arXiv:2304.00501.
95. Tan, M.; Pang, R.; Le, Q.V. EfficientDet: Scalable and efficient object detection. In Proceedings of the 2020 IEEE/CVF Conference
on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 10778–10787. [CrossRef]
96. Carion, N.; Massa, F.; Synnaeve, G.; Usunier, N.; Kirillov, A.; Zagoruyko, S. End-to-end object detection with transformers. In
Computer Vision–ECCV 2020; Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M., Eds.; Lecture Notes in Computer Science; Springer:
Cham, Switzerland, 2020; Volume 12346, pp. 213–229.
Algorithms 2024, 17, 103 34 of 36

97. Tian, Z.; Shen, C.; Chen, H.; He, T. Fcos: Fully convolutional one-stage object detection. In Proceedings of the 2019 IEEE/CVF
International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019; pp. 9626–9635.
[CrossRef]
98. Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal loss for dense object detection. In Proceedings of the 2017 IEEE
International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2999–3007. [CrossRef]
99. Galvao, L.G.; Abbod, M.; Kalganova, T.; Palade, V.; Huda, M.N. Pedestrian and vehicle detection in autonomous vehicle
perception systems—A review. Sensors 2021, 21, 7267. [CrossRef]
100. Jocher, G. Ultralytics/Yolov8, GitHub, GitHub Repository. 2023. Available online: https://github.com/ultralytics/ultralytics
(accessed on 15 December 2023).
101. Jia, S.J.; Zhai, Y.T.; Jia, X.W. Detection of Traffic and Road Condition Based on SVM and PHOW. Appl. Mech. Mater. 2014,
513, 3651–3654. [CrossRef]
102. Gao, D.; Zhou, J.; Xin, L. SVM-based detection of moving vehicles for automatic traffic monitoring. In Proceedings of the ITSC
2001. 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.01TH8585), Oakland, CA, USA, 25–29 August 2001;
pp. 745–749. [CrossRef]
103. Zheng, F.; Luo, S.; Song, K.; Yan, C.W.; Wang, M.C. Improved lane line detection algorithm based on Hough transform. Pattern
Recognit. Image Anal. 2018, 28, 254–260. [CrossRef]
104. Vamsi, A.M.; Deepalakshmi, P.; Nagaraj, P.; Awasthi, A.; Raj, A. IOT based autonomous inventory management for warehouses.
In EAI International Conference on Big Data Innovation for Sustainable Cognitive Computing; Haldorai, A., Ramu, A., Mohanram, S.,
Onn, C., Eds.; EAI/Springer Innovations in Communication and Computing; Springer: Cham, Switzerland, 2020; pp. 371–376.
105. Nagaraj, P.; Muneeswaran, V.; Sudar, K.M.; Ali, R.S.; Someshwara, A.; Kumar, T.S. Internet of Things based smart hospital saline
monitoring system. In Proceedings of the 2021 5th International Conference on Computer, Communication and Signal Processing
(ICCCSP), Chennai, India, 24–25 May 2021; pp. 53–58. [CrossRef]
106. Wang, Z.; Zhan, J.; Duan, C.; Guan, X.; Yang, K. Vehicle detection in severe weather based on pseudo-visual search and HOG–LBP
feature fusion. Proc. Inst. Mech. Eng. Part D J. Automob. Eng. 2022, 236, 1607–1618. [CrossRef]
107. Abdullah, M.N.; Ali, Y.H. Vehicles detection system at different weather conditions. Iraqi J. Sci. 2021, 62, 2040–2052. [CrossRef]
108. Wu, B.F.; Juang, J.H. Adaptive vehicle detector approach for complex environments. IEEE Trans. Intell. Transp. Syst. 2012,
13, 817–827. [CrossRef]
109. Padilla, D.A.; Villaverde, J.F.; Magdaraog, J.J.T.; Oconer, A.J.L.; Ranjo, J.P. Vehicle and Weather Detection Using Real Time Image
Processing Using Optical Flow and Color Histogram. In Proceedings of the 2019 5th International Conference on Control,
Automation and Robotics (ICCAR), Beijing, China, 19–22 April 2019; pp. 880–883. [CrossRef]
110. Tian, Q.; Zhang, L.; Wei, Y.; Zhao, W.; Fei, W. Vehicle Detection and Tracking at Night in Video Surveillance. Int. J. Online Eng.
2013, 9, 60–64. [CrossRef]
111. Kuang, H.; Chen, L.; Chan, L.L.H.; Cheung, R.C.; Yan, H. Feature selection based on tensor decomposition and object proposal
for night-time multiclass vehicle detection. IEEE Trans. Syst. Man Cybern. Syst. 2018, 49, 71–80. [CrossRef]
112. Ewecker, L.; Asan, E.; Ohnemus, L.; Saralajew, S. Provident vehicle detection at night for advanced driver assistance systems.
Auton. Robot. 2023, 47, 313–335. [CrossRef]
113. Yaghoobi Ershadi, N. Improving vehicle tracking rate and speed estimation in dusty and snowy weather conditions with a
vibrating camera. PLoS ONE 2017, 12, e0189145. [CrossRef]
114. Yang, H.; Qu, S. Real-time vehicle detection and counting in complex traffic scenes using background subtraction model with
low-rank decomposition. IET Intell. Transp. Syst. 2018, 12, 75–85. [CrossRef]
115. Govardhan, P. Night Time Pedestrian Detection for Advanced Driving Assistance Systems (ADAS) Using Near Infrared Images.
Ph.D. Thesis, National Institute of Technology, Rourkela, India, 2014.
116. Sasaki, Y.; Emaru, T.; Ravankar, A.A. SVM based pedestrian detection system for sidewalk snow removing machines. In
Proceedings of the 2021 IEEE/SICE International Symposium on System Integration (SII), Iwaki, Japan, 11–14 January 2021;
pp. 700–701.
117. Kang, S.; Byun, H.; Lee, S.W. Real-time pedestrian detection using support vector machines. In Pattern Recognition with Support
Vector Machines. SVM 2002; Lee, S.W., Verri, A., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany,
2002; Volume 2388, pp. 268–277.
118. Jegham, I.; Khalifa, A.B. Pedestrian detection in poor weather conditions using moving camera. In Proceedings of the 2017
IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), Hammamet, Tunisia, 30 October–3
November 2017; pp. 358–362.
119. Fan, Y. Research of Pedestrian Tracking Based on HOG Feature and Haar Feature. Comput. Sci. 2013, 40, 199–203.
120. Ding, B.; Liu, Z.; Sun, Y. Pedestrian detection in haze environments using dark channel prior and histogram of oriented gradient.
In Proceedings of the 2018 Eighth International Conference on Instrumentation & Measurement, Computer, Communication and
Control (IMCCC), Harbin, China, 19–21 July 2018; pp. 1003–1008.
121. Sotelo, M.; Parra, I.; Fernandez, D.; Naranjo, E. Pedestrian detection using SVM and multi-feature combination. In Proceedings
of the 2006 IEEE Intelligent Transportation Systems Conference, Toronto, ON, Canada, 17–20 September 2006; pp. 103–108.
[CrossRef]
Algorithms 2024, 17, 103 35 of 36

122. Satyawan, A.S.; Fuady, S.; Mitayani, A.; Sari, Y.W. HOG Based Pedestrian Detection System for Autonomous Vehicle Operated
in Limited Area. In Proceedings of the 2021 International Conference on Radar, Antenna, Microwave, Electronics, and
Telecommunications (ICRAMET), Bandung, Indonesia, 23–24 November 2021; pp. 147–152.
123. Jang, G.; Park, J.; Kim, M. Cascade-Adaboost for Pedestrian Detection Using HOG and Combined Features. In Advances in
Computer Science and Ubiquitous Computing; Springer: Singapore, 2016; pp. 430–435.
124. Zhang, X.; Zhu, X. Autonomous path tracking control of intelligent electric vehicles based on lane detection and optimal preview
method. Expert Syst. Appl. 2019, 121, 38–48. [CrossRef]
125. Gern, A.; Moebus, R.; Franke, U. Vision-based lane recognition under adverse weather conditions using optical flow. In
Proceedings of the Intelligent Vehicle Symposium 2002, Versailles, France, 17–21 June 2002; Volume 2, pp. 652–657.
126. Tran, T.T.; Son, J.H.; Uk, B.J.; Lee, J.H.; Cho, H.M. An adaptive method for detecting lane boundary in night scene. In Advanced
Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence; Huang, D.S., Zhang, X., Reyes García, C.A.,
Zhang, L., Eds.; ICIC 2010. Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2010; Volume 6216,
pp. 301–308.
127. Chen, T.Y.; Chen, C.H.; Luo, G.M.; Hu, W.C.; Chern, J.C. Vehicle detection in nighttime environment by locating road lane
and taillights. In Proceedings of the 2015 International Conference on Intelligent Information Hiding and Multimedia Signal
Processing (IIH-MSP), Adelaide, SA, Australia, 23–25 September 2015; pp. 60–63.
128. Guo, J.; Wei, Z.; Miao, D. Lane detection method based on improved RANSAC algorithm. In Proceedings of the 2015 IEEE
Twelfth International Symposium on Autonomous Decentralized Systems, Taichung, Taiwan, 25–27 March 2015; pp. 285–288.
129. Antonio, J.A.; Romero, M. Pedestrians’ Detection Methods in Video Images: A Literature Review. In Proceedings of the 2018
International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 12–14 December
2018; pp. 354–360.
130. Papadimitriou, O.; Kanavos, A.; Mylonas, P.; Maragoudakis, M. Advancing Weather Image Classification using Deep Convolu-
tional Neural Networks. In Proceedings of the 2023 18th International Workshop on Semantic and Social Media Adaptation &
Personalization (SMAP) 18th International Workshop on Semantic and Social Media Adaptation & Personalization (SMAP 2023),
Limassol, Cyprus, 25–26 September 2023; pp. 1–6.
131. Alhammadi, S.A.; Alhameli, S.A.; Almaazmi, F.A.; Almazrouei, B.H.; Almessabi, H.A.; Abu-Kheil, Y. Thermal-Based Vehicle
Detection System using Deep Transfer Learning under Extreme Weather Conditions. In Proceedings of the 2022 8th International
Conference on Information Technology Trends (ITT), Dubai, United Arab Emirates, 25–26 May 2022; pp. 119–123.
132. Arora, N.; Kumar, Y.; Karkra, R.; Kumar, M. Automatic vehicle detection system in different environment conditions using fast
R-CNN. Multimed. Tools Appl. 2022, 81, 18715–18735. [CrossRef]
133. Ghosh, R. On-road vehicle detection in varying weather conditions using faster R-CNN with several region proposal networks.
Multimed. Tools Appl. 2021, 80, 25985–25999. [CrossRef]
134. Cao, J.; Song, C.; Song, S.; Peng, S.; Wang, D.; Shao, Y.; Xiao, F. Front vehicle detection algorithm for smart car based on improved
SSD model. Sensors 2020, 20, 4646. [CrossRef] [PubMed]
135. Li, W. Vehicle detection in foggy weather based on an enhanced YOLO method. In Journal of Physics: Conference Series; IOP
Publishing: Bristol, UK, 2022; Volume 2284, p. 012015.
136. Ghosh, R. A modified yolo model for on-road vehicle detection in varying weather conditions. In Intelligent Computing and
Communication Systems; Springer: Singapore, 2021; pp. 45–54.
137. Miao, Y.; Liu, F.; Hou, T.; Liu, L.; Liu, Y. A nighttime vehicle detection method based on YOLO v3. In Proceedings of the 2020
Chinese Automation Congress (CAC), Shanghai, China, 6–8 November 2020; pp. 6617–6621.
138. Humayun, M.; Ashfaq, F.; Jhanjhi, N.Z.; Alsadun, M.K. Traffic management: Multi-scale vehicle detection in varying weather
conditions using yolov4 and spatial pyramid pooling network. Electronics 2022, 11, 2748. [CrossRef]
139. Wang, R.; Zhao, H.; Xu, Z.; Ding, Y.; Li, G.; Zhang, Y.; Li, H. Real-time vehicle target detection in inclement weather conditions
based on YOLOv4. Front. Neurorobot. 2023, 17, 1058723. [CrossRef]
140. Yao, J.; Fan, X.; Li, B.; Qin, W. Adverse Weather Target Detection Algorithm Based on Adaptive Color Levels and Improved
YOLOv5. Sensors 2022, 22, 8577. [CrossRef] [PubMed]
141. Razzok, M.; Badri, A.; Mourabit, I.E.; Ruichek, Y.; Sahel, A. Pedestrian detection under weather conditions using conditional
generative adversarial network. Int. J. Artif. Intell. 2023, 12, 1557–1568. [CrossRef]
142. Pawełczyk, M.; Wojtyra, M. Real world object detection dataset for quadcopter unmanned aerial vehicle detection. IEEE Access
2020, 8, 174394–174409. [CrossRef]
143. Yahiaoui, M.; Rashed, H.; Mariotti, L.; Sistu, G.; Clancy, I.; Yahiaoui, L.; Kumar, V.R.; Yogamani, S. Fisheyemodnet: Moving object
detection on surround-view cameras for autonomous driving. arXiv 2019, arXiv:1908.11789.
144. Ragesh, N.; Rajesh, R. Pedestrian detection in automotive safety: Understanding state-of-the-art. IEEE Access 2019, 7, 47864–47890.
[CrossRef]
145. Kim, J.H.; Hong, H.G.; Park, K.R. Convolutional neural network-based human detection in nighttime images using visible light
camera sensors. Sensors 2017, 17, 1065. [CrossRef] [PubMed]
146. Hsia, C.H.; Peng, H.C.; Chan, H.T. All-Weather Pedestrian Detection Based on Double-Stream Multispectral Network. Electronics
2023, 12, 2312. [CrossRef]
Algorithms 2024, 17, 103 36 of 36

147. Lai, K.; Zhao, J.; Liu, D.; Huang, X.; Wang, L. Research on pedestrian detection using optimized mask R-CNN algorithm in
low-light road environment. In Journal of Physics: Conference Series; IOP Publishing: Bristol, UK, 2021; Volume 1777, p. 012057.
148. Tumas, P.; Nowosielski, A.; Serackis, A. Pedestrian detection in severe weather conditions. IEEE Access 2020, 8, 62775–62784.
[CrossRef]
149. Shakeri, A.; Moshiri, B.; Garakani, H.G. Pedestrian detection using image fusion and stereo vision in autonomous vehicles. In
Proceedings of the 2018 9th International Symposium on Telecommunications (IST), Tehran, Iran, 17–19 December 2018; pp.
592–596. [CrossRef]
150. Galarza-Bravo, M.A.; Flores-Calero, M.J. Pedestrian detection at night based on faster R-CNN and far infrared images. In
Intelligent Robotics and Applications; Chen, Z., Mendes, A., Yan, Y., Chen, S., Eds.; ICIRA 2018. Lecture Notes in Computer Science;
Springer: Cham, Switzerland, 2018; Volume 10985, pp. 335–345.
151. Xu, Y.; Cao, X.; Qiao, H. Optical camera based pedestrian detection in rainy or snowy weather. In Fuzzy Systems and
Knowledge Discovery; Wang, L., Jiao, L., Shi, G., Li, X., Liu, J., Eds.; FSKD 2006. Lecture Notes in Computer Science; Springer:
Berlin/Heidelberg, Germany, 2006; Volume 4223, pp. 1182–1191.
152. Montenegro, B.; Flores-Calero, M. Pedestrian detection at daytime and nighttime conditions based on YOLO-v5. Ingenius. Rev.
Cienc. Tecnol. 2022, 85–95. [CrossRef]
153. Zaman, M.; Saha, S.; Zohrabi, N.; Abdelwahed, S. Deep Learning Approaches for Vehicle and Pedestrian Detection in Adverse
Weather. In Proceedings of the 2023 IEEE Transportation Electrification Conference & Expo (ITEC), Detroit, MI, USA, 21–23 June
2023; pp. 1–6.
154. Son, J.; Yoo, H.; Kim, S.; Sohn, K. Real-time illumination invariant lane detection for lane departure warning system. Expert Syst.
Appl. 2015, 42, 1816–1824. [CrossRef]
155. Kortli, Y.; Marzougui, M.; Atri, M. Efficient implementation of a real-time lane departure warning system. In Proceedings of the
2016 International Image Processing, Applications and Systems (IPAS), Hammamet, Tunisia, 5–7 November 2016; pp. 1–6.
156. Sun, T.Y.; Tsai, S.J.; Chan, V. HSI color model based lane-marking detection. In Proceedings of the 2006 IEEE Intelligent
Transportation Systems Conference, Toronto, ON, Canada, 7–12 September 2006; pp. 1168–1172.
157. Chiu, K.Y.; Lin, S.F. Lane detection using color-based segmentation. In Proceedings of the IEEE Proceedings. Intelligent Vehicles
Symposium, Las Vegas, NV, USA, 6–8 June 2005; pp. 706–711.
158. Shin, S.; Shim, I.; Kweon, I.S. Combinatorial approach for lane detection using image and LIDAR reflectance. In Proceedings of
the 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Goyangi, Republic of Korea,
28–30 October 2015; pp. 485–487.
159. Rose, C.; Britt, J.; Allen, J.; Bevly, D. An integrated vehicle navigation system utilizing lane-detection and lateral position
estimation systems in difficult environments for GPS. IEEE Trans. Intell. Transp. Syst. 2014, 15, 2615–2629. [CrossRef]
160. Hsieh, J.W.; Chen, L.C.; Chen, D.Y. Symmetrical SURF and its applications to vehicle detection and vehicle make and model
recognition. IEEE Trans. Intell. Transp. Syst. 2014, 15, 6–20. [CrossRef]
161. Yusuf, M.M.; Karim, T.; Saif, A.S. A robust method for lane detection under adverse weather and illumination conditions using
convolutional neural network. In Proceedings of the ICCA 2020: International Conference on Computing Advancements, Dhaka,
Bangladesh, 10–12 January 2020; pp. 1–8.
162. Nguyen, V.; Kim, H.; Jun, S.; Boo, K. A study on real-time detection method of lane and vehicle for lane change assistant system
using vision system on highway. Eng. Sci. Technol. Int. J. 2018, 21, 822–833. [CrossRef]
163. Khan, M.N.; Ahmed, M.M. Weather and surface condition detection based on road-side webcams: Application of pre-trained
convolutional neural network. Int. J. Transp. Sci. Technol. 2022, 11, 468–483. [CrossRef]
164. Ding, Y.; Xu, Z.; Zhang, Y.; Sun, K. Fast lane detection based on bird’s eye view and improved random sample consensus
algorithm. Multimed. Tools Appl. 2017, 76, 22979–22998. [CrossRef]
165. Ab Ghani, H.; Daud, A.M.; Besar, R.; Sani, Z.M.; Kamaruddin, M.N.; Syahali, S. Lane Detection Using Deep Learning for Rainy
Conditions. In Proceedings of the 2023 9th International Conference on Computer and Communication Engineering (ICCCE),
Kuala Lumpur, Malaysia, 15–16 August 2023; pp. 373–376.
166. Raj, N.; Dutta, K.K.; Kamath, A. Lane Prediction by Autonomous Vehicle in Highway Traffic using Artificial Neural Networks. In
Proceedings of the 2021 Fourth International Conference on Electrical, Computer and Communication Technologies (ICECCT),
Erode, India, 15–17 September 2021; pp. 1–6.
167. Krishnaveni, P.; Sutha, J. Novel deep learning framework for broadcasting abnormal events obtained from surveillance
applications. J. Ambient. Intell. Humaniz. Comput. 2020, 13, 4123. [CrossRef]
168. Ouyang, W.; Wang, X. A discriminative deep model for pedestrian detection with occlusion handling. In Proceedings of the 2012
IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; pp. 3258–3265.
169. Haris, M.; Hou, J.; Wang, X. Lane lines detection under complex environment by fusion of detection and prediction models.
Transp. Res. Rec. 2022, 2676, 342–359. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.