7.1 The Threat Model for General ADS
In this section, we construct a threat model to summarize the safety and security threats that each module may confront based on our survey results. To build the threat model, we first summarize the threats discovered in the papers that we survey in the module-level testing in Section
5; then, as a complement, we review the bugs shown in the empirical studies [
34,
35] on open source ADS to understand the concrete issues encountered in each module during system development. Our threat model is shown in Figure
5.
Threats to sensing. In this module, existing studies mainly concern about the hardware aspect, e.g., those physical sensors that are critical hardware used in an ADS for collecting the information of the external environments. A common threat such as harsh weather conditions could reduce the capabilities of the intelligent sensors. There are also many deliberate attack techniques, such as
jamming attack [
52,
53,
54] and
spoofing attack [
55,
56,
57,
58] (see details in Section
5.1.2) that target this module and could interfere with these sensors and harm their normal functionalities.
Threats to perception. The perception module is the most investigated, and we collect 23 testing techniques dedicated to this module. Common threat comes from
adversarial examples that are generated by adding perturbations to normal images, which can fool the deep learning models in the perception module to make incorrect predictions, as shown in References [
61,
62,
63,
64,
65,
65,
66,
67,
71,
72,
73,
74,
75,
76,
77,
78,
79,
80,
83]. Another type of threat is called
Trojan attack [
88,
89], in which malicious data are injected into the training data of the deep learning models. Moreover, in the case when the ADS requires an HD map from the cloud service,
Denial of service [
309] or fake HD map data [
310] can interfere with perception tasks such as localization.
Threats to planning. With the data produced by the perception module, the planning module takes charge of several tasks, e.g., object trajectory prediction and path planning, and we collect eight testing techniques for this module. A common threat comes from the unwanted maneuvers of NPCs [
137,
138,
139,
141,
142,
143,
144,
145], which can interfere with the prediction for moving objects and thus lead to an unsafe trajectory plan. Moreover, improper localization from the perception module can also threaten the accuracy of output trajectories.
Threats to control. This module mostly adopts those mature control techniques, e.g.,
MPC and
PID, and thus are relatively hard to attack. One threat comes from the injected faults [
151,
152], which could affect the longitudinal/lateral control of the vehicle. Another threat emerges when an emergency situation is encountered, e.g., when an emergency braking is needed, and the control module may fail to handle these cases. Moreover, the output signals are sent via the CAN bus to the ECU for controlling the vehicle. Since this process involves a data transmission between software and hardware, a potential threat is the interface mismatch [
311], e.g., an inappropriate steering angle rate, in practical usage.
7.2 Datasets and Tool Stacks for ADS Testing
Datasets. In the context of ADS, deep learning components handle safety-critical tasks, e.g., perception and end-to-end control, so it is necessary to validate their robustness under various scenarios. This process typically relies on data from the real world, which is, however, hard to obtain in general. Fortunately, there are a collection of publicly available datasets to solve the problem, which involve large quantities of real-world pictures and videos recorded by onboard sensors. For example, the KITTI dataset [
313] contains over 10,000 images of traffic scenarios, collected by a variety of sensors including high-resolution
RGB/grayscale stereo cameras and a
3D laser scanner.
In this section, we summarize the scenario-driven datasets for ADS testing in Table
10. The first column shows the time when each dataset was released. The next three columns give the name, brief description, and the size of each dataset, and the last column indicates the related works that adopt these datasets. Note that the datasets for other machine learning testing tasks [
337] that have nothing to do with ADS testing are not listed here; in other words, all the datasets listed here are dedicated to ADS testing.
As shown in Table
10, we collect 27 datasets released from 2004 to 2022, including popular ones like the KITTI dataset [
313] and emerging ones like the CrashD [
336] dataset. One observation is that these datasets span over various physical conditions, e.g., different times of the day [
317], different weather conditions [
329,
332], and different traffic density [
327]. They also span over various application scenarios, such as urban street [
316,
320,
322], highway [
325,
326,
338], and intersection [
331]. In addition, we find that some of these datasets are specific to a certain task, e.g., pedestrian detection [
319,
321] and traffic sign detection [
125,
314,
315].
As the column of references in Table
10 shows, several datasets such as the KITTI dataset [
313] and the Udacity dataset [
317] are frequently used in ADS testing due to the diverse tasks they support, such as object detection and semantic segmentation. However, we also find that a number of datasets have not been widely used, due to their own limitations. For example, the rounD [
333] and openDD [
334] can only be used for validating the behavior planning of ADS in the scenario of roundabouts; SYNTHIA [
318] and GTA5 [
339] contain synthetic images from virtual environments, which may be not realistic enough for ADS testing.
Tool stacks. As mentioned, simulation-based testing has become an important alternative approach for real-world testing. Simulators usually provide vehicle dynamics, e.g., longitudinal and lateral motion of the vehicle, and virtual traffic scenarios. Moreover, simulators can help generate those extreme scenarios for testing, e.g., harsh weather, which are rarely encountered in the real world. There have been many advanced simulation platforms developed for ADS testing in recent years. For example,
Carla [
345] is an open source simulator for ADS training and testing, which supports various sensor models and environmental conditions.
In this section, we summarize the simulation platforms, namely the simulators usually used for ADS testing in Table
11. As shown in the table, we collect 20 simulation platforms, including classical platforms such as
Matlab/Simulink [
340] and
CarSim [
341], and emerging popular simulators, such as
Carla and
LGSVL [
351]. Since these simulators have their own pros and cons, we compare them in the table and focus on several aspects of interest, e.g., their gap from real environment. Specifically, the first column lists the name and the second column shows the accessibility of each simulator. The third column is relevant to physical aspects, that is, whether the simulator allows for customizing a dynamic model and whether it is a soft-body or rigid-body based simulator. The fourth column indicates the level of support for mixed-reality testing, including
model-in-the-loop (MiL),
software-in-the-loop (SiL), HiL, and ViL. The fifth column presents the capability of these simulators to complement each other, e.g., whether they support co-simulation with other simulators. The last column indicates the related works that adopt these simulators in their research.
Based on the table, we can draw the following conclusions:
•
First, there exist many commercial simulators that are not open source, e.g.,
CarMaker [
222] and
PreScan [
303]. These simulators could be expensive and difficult for researchers to satisfy their research goals. In comparison, open source simulators like
Carla could have broader prospects for future research;
•
Second, accurate physical dynamic models are needed to bridge the gap between simulation-based testing and real-world testing and satisfy different testing requirements, e.g., smooth road needs a lower friction coefficient. We find that there have been simulators, i.e.,
BeamNG [
15] and
CarSim, dedicated to this aspect and allowing for dynamic model customization. In particular,
BeamNG is also a soft-body based simulator that supports more realistic collision effects (see more details in Section
2.3);
•
Third, we find that most simulators support software-in-the-loop and hardware-in-the-loop testing. Several simulators, e.g.,
CarMaker and
VTD [
343], support vehicle-in-the-loop testing, which closes the gap between hardware-in-the-loop testing and real-world testing.
•
Last, we find that a number of simulators have built-in interfaces to other simulators. This is essential to perform co-simulation for ADS testing, since these simulators have their own pros and cons, and co-simulation could complement each other for a more realistic testing environment. For example,
CarMaker (accurate vehicle dynamics) and
VISSIM (representational traffic flow) are combined into a co-simulation framework [
288] for generating more realistic testing scenarios.
Overall, it can be seen from the reference column that Matlab/Simulink and BeamNG have been widely used for ADS testing. Simulators like NVIDIA Drive Sim and Gazebo also have great potential for future research, since they cover multiple features we list in the table, e.g., whether they could perform ViL testing or co-simulation with other simulators.
Moreover, we also introduce several publicly available systems under test in Table
12.
OpenPilot,
Apollo,
Autoware, and
Pylot are all modular systems and have already been described in Section
2.3, so we will not repeat them here. In addition to modular ADS systems, there also exist open source end-to-end based systems:
LBC [
356] is an imitation learning controller that uses camera images and direction commands as input to control the direction of the vehicle in the lane and intersection;
DeepDriving [
357] and
Nvidia CNN Lane Follower [
358] are also widely used CNN-based end-to-end controllers; and Open source DNN models from the Udacity self-driving challenge, such as
Chauffeur [
179] and
Epoch [
180], are another line of end-to-end driving controllers. Besides, there are also driving agents and controllers from simulators, such as
BeamNG and
Carla.
BeamNG.AI, which has been mentioned in Section
2.3, is an AI agent in
BeamNG simulator and could accept virtual images in the simulator as input for path planning and trajectory tracking.
Carla PID is a specific module in
Carla that performs calculations at the motion planning stage and estimates the acceleration, braking, and steering inputs required to reach target positions.
Programming languages. To systematically generate test cases, it has become a trend to propose new programming languages for testing scenario description. In this way, the generation of a new test case boils down to writing a program that describes the scenario. Also, researchers can make use of existing coverage criteria for programs, such as the code coverage criteria, to assess the adequacy of the generated tests.
To define such a programming language, researchers need to formally express the basic elements in an ADS scenario, e.g., the ego car, other cars, pedestrians, and static objects. Since these languages are usually dependent on existing formats, they vary in their ways of expressing those elements, based on their dependent formats. For instance,
Scenic [
361], a pythonlike language, requires users to define those objects as variables; in contrast,
GeoScenario [
360] provides users with a graphical interface where users can drag the icons to describe a scenario. Moreover, these languages usually do not emerge independently; instead, they come with specific simulators or even specific ADS.
In this section, we summarize the state-of-the-art programming languages for test case generation in Table
13, and introduce their dependent formalism, their bonded simulator, other features, and their adoption in ADS testing. There exists literature, e.g., Reference [
367], that surveys programming languages for the test generation of ADS. Compared to Reference [
367], our main aim is to show the ecosystems and the landscape of the use of these languages in ADS testing, as a reference for the readers to better understand the testing techniques in Sections
5 and
6. Also, our study includes some latest achievements, e.g., paracosm [
366] and SceGene [
365], in this direction.
As shown by Table
13, we collect eight representative programming languages, including the classic ones, such as OpenScenario, that have been widely used in different stages of the development of ADS, and emerging ones, such as paracosm [
366]. As our findings, first, these languages offer distinct capabilities for different purposes, e.g., languages are designed for different purposes and attached with different features, e.g., Scenic [
361] allows probabilistic sampling for testing driving systems with machine learning components; SceGene [
365] designs bio-inspired operations, such as crossover and mutation, for scenario generation. Second, some of these languages provide more user-friendly features; for instance, some of the languages, e.g., SceML [
363] provide a graphical user interface for users to define their scenarios. However, as the column of reference shows, most of these languages have not been widely adopted in practice. This can be due to several reasons: One possibility is that some languages are still too specialized for practitioners to adopt them in their work; also, since many of the languages, such as GeoScenario [
360], are designed for specific systems, they are still ad hoc and not easily extensible to be adopted in other systems.
In conclusion, programming languages are increasingly deemed as powerful weapons for test case generation in ADS testing, but currently they have not been widely adopted in practice.