Simulating the Impact of Dynamic Rerouting on Metropolitan-scale Traffic Systems

Active traffic management strategies using changeable message signs or broadcast media are regularly used by traffic management centers to present alternate routes to drivers when abnormal events occur on the roadways. Shifting traffic onto alternate routes maximizes the efficiency and capacity of the network and increases safety by reducing secondary vehicle accidents due to unexpected congestion. With highways, parallel corridors that handle the rerouted traffic must be available with adequate capacity for rerouting to be successful. This requires traffic centers, with humans in the loop, to assess the situation, have knowledge of the parallel routes via predefined control strategies, and implement an active control plan. In the past five years, smartphone navigation apps have gained popularity and serve a similar role. These applications use their current understanding of congestion to compute new routes in which the travel time is shorter for the user of the device. The congestion information is generated by aggregating the speeds and locations of all of the devices that the app is monitoring. A road congestion estimate is created using road network information and other traffic information, such as historical traffic information, and combined with the current user’s destination. If the projected congestion of the user’s current route significantly impacts their expected travel time, the app may suggest a new route to the user with a shorter travel time. As such, these navigation applications are also serving as active traffic managers.

The introduction of these new active traffic managers makes it more difficult for researchers and government transportation agencies to understand and predict the dynamics of congested transportation systems. Traffic simulation is a key capability for these organizations to analyze different scenarios and predict the potential impacts of different infrastructure changes, policies, or control strategies. However, the complexity of transportation systems makes them extremely difficult to simulate at the urban scale. As such, little is known about how to control and actively manage large-scale road networks in the presence of significant congestion, particularly with the active management now being implemented by different agents, such as smartphone apps and traffic centers. We do know that providing information about congested states and having a variety of sources provide new travel routes will likely create unpredictable patterns and dynamic variation. Control strategies implemented through traffic management can include infrastructure modifications such as signal-phase-timing adjustments of signals to redirect traffic and expedite congestion mitigation. These control decisions are determined by estimating the impact of the rerouting action. To-date, these do not include rerouting that occurs as a result of drivers following an app’s routing information. This was made evident when during an evacuation event, drivers were venturing into dangerous situations with routes that were not informed by road closures associated with the event [11].

In the last two decades, transportation network simulation has increased in popularity for emulating driving behaviors, predicting traffic dynamics, and predicting impacts of control strategies. Applied models can broadly be categorized into both equilibrium models, often referred to as traffic assignment models, and non-equilibrium models, often referred to as simulation models [9, 43]. The simulation models complement traffic assignment, which is one of the essential tools that planners use for estimating congestion. With a reliable origin-destination demand matrix, an accurate traffic assignment generates optimized traffic-state predictions [43]. While equilibrium models have been very popular in the past due to mathematical clarity, their main drawback is the inability to capture the congestion-dependent evolution of a driver’s route [30]. This limitation is overcome with non-equilibrium models that allow for modeling route guidance dynamics and unexpected events such as incidents or evacuation strategies [9, 22].

In order to predict the emergent traffic dynamics that result from congestion-dependent rerouting and unexpected events, we have extended our parallel discrete event simulation of large-scale traffic dynamics [14] with new dynamic vehicle rerouting capabilities. To understand the context of this article, Figure 1 describes four different traffic models we have developed for large-scale network modeling, each with a different set of algorithms. Details of the baseline model can be found in our previous article [14] and the quasi-dynamic traffic assignment model will be described in a forthcoming publication [15]. In this article, we focus on the second model with dynamic routing which we believe best captures realistic emergent traffic dynamics. Note that we acknowledge that there are a percentage of routes that are neither shortest free speed routes nor dynamically routed in the real-world. We denote them as knowledge-based routes, representing routes that are arbitrarily chosen by a user, such as a route chosen to include a café stop en route to work. At this time, we do not model these types of routes; thus, all 19 M routes are either shortest free speed routes or dynamically routed.

The ultimate goal of active traffic management is to drive the system towards an optimized state, which is the focus of most traffic assignment algorithms. But in reality, reaching equilibrium is highly unlikely regardless of the attempted active management strategies. As such, a mechanism for simulating and analyzing hypothetical scenarios that will reveal emergent dynamics with prescribed dynamic routing algorithms would be extremely valuable for those designing active traffic management systems. With increased urbanization in cities today, understanding how to actively manage the dynamics on the road network is becoming increasingly urgent, not only from a loss in productivity perspective but from a fuel consumption perspective. Our research focus is to breakdown the computational barriers of simulating large-scale road network dynamics in order to investigate the impacts of active traffic management strategies and reduce the multifaceted cost of the increasing congestion in our cities.

This article presents a methodology for representing the transportation system with dynamic rerouting capabilities in a scalable parallel discrete event simulation (PDES) formalism. We include a description of the core link actor model and its constituent components that calculate vehicle traversal times, timing constraints, and storage capacity constraints. We also introduce vehicle controller actors that monitor congestion within the traffic system and handle dynamic reroute requests from simulated vehicles. We demonstrate that the resulting mesoscopic model achieves scalable computational performance for a large system (19 million vehicles over a network with 1 million links representing the San Francisco Bay Area), simulating a day of traffic with 50 percent dynamic rerouting penetration in three minutes on 256 cores of the NERSC Cori computer at Lawrence Berkeley National Laboratory [39]. We provide a parallel performance and scalability analysis, a rerouting parameter sensitivity analysis, a discussion of the simulated impacts of varying the dynamic rerouting penetration rate on various transportation system metrics, and finally, present a validation of the simulation results compared to real-world data sources.

There has been much previous work in the area of transportation system modeling and simulation. Previous simulators can be broadly classified as macroscopic, mesoscopic, and microscopic based on the level of detail of the behavior of individual agents simulated. The level of behavior detail in the model typically also correlates to the size of the area under simulation—larger area simulations tend to use macroscopic models, while simulations focusing on small areas use more detailed microscopic models. Dynamic traffic assignment (DTA) solvers and dynamic simulators such as MATSim [38], POLARIS [3], DTALite [58], DynaMIT [10], DynaSmart [35], Aimsun [13], SUMO [34], INTEGRATION [42], BEAM [46], Ugirumurera et al. [51], MANTA [56], and many others fall along the spectrum of different modeling approaches, target problem scales, modeling fidelities, and output (e.g., traffic assignment versus simulation). The level of fidelity targeted by our Mobiliti simulator is mesoscopic since it does not resolve lane-changing or vehicle following behavior, although it does resolve individual vehicle movements and rerouting decisions.

There is a large body of previous work in the area of parallel discrete event simulation (PDES, see [23] for an overview). In particular, seminal work by Chandy and Misra [16], Bryant [1], and Jefferson [28] laid the groundwork for parallel discrete event simulation, which was shown to run efficiently on supercomputers [7, 8]. Previous traffic simulators that utilize PDES include those by Perumalla [40] and Yoginath [57], who used optimistic PDES for modeling traffic grids, though they evaluated their system on smaller-scale synthetic grid networks and used a different mechanism to mediate congestion among vehicles. Thulasidasan et al. [48] also modeled road networks using queue-based parallel discrete event simulation, but they did not include adaptive vehicle rerouting behavior in their model, where vehicles can respond to congestion by rerouting in the middle of (rather than just at the beginning of) their trip. Furthermore, our article includes a description of the design and implementation of a dynamic vehicle rerouting system with a parameter sensitivity analysis for vehicles and controllers at a large-scale.

In the area of dynamic re-routing, research by Liang and Wakahara [33] showed how proactive dynamic re-routing could reduce average travel time for congested road networks using the SUMO micro simulator on a medium-sized area of London (3,002 links and 332 nodes with 954 vehicles). Zhao et al. [58] gave a theoretical analysis of the dynamics and stability of equilibrium with travelers that reroute depending on cost difference, and demonstrated their results on some small networks (up to 31 nodes and 40 links with three OD pairs). Similarly, [52] provides a theoretical treatment of distributed multi-agent route selection problem with incentives, with numerical examples of their approach on the Sioux Falls road network (24 nodes). In [49], the authors utilize SUMO and OMNeT++ to model an area of Brooklyn with 380 nodes and 474 links, with a traffic demand of 2,500 vehicles. Their approach is similar to ours (albeit on a smaller network) in that they utilize a distributed cloud-based vehicle control system, where sensors detect congestion, and routes are suggested to avoid the congestion. In [31], the authors utilize a combination of SUMO, Veins, and OMNeT++ tools to analyze the impact of rerouting on a 2 × 2 grid map (9 nodes, 24 links). In [41], the authors describe a vanpool scheduler architecture for dynamic rerouting. They evaluate their approach on a network of the Munich city center, where the focus is more on responding to stochastic vanpool requests rather than dynamic congestion effects. In [32], they investigate the information comply model to simulate how drivers can react to information about an event and evaluated their approach on a network with approximately 1,000 links. There has also been related work in the area of route selection in the context of dynamic traffic assignment (e.g., [4]). Dynamic traffic assignment approaches typically model converged dynamic driver behavior adapted to daily congestion patterns, rather than more reactive dynamic rerouting scenarios that explore how traffic can respond to unexpected events. Our approach differs from these previous works in that we leverage high-performance computing techniques to simulate the effect of dynamic rerouting on much larger road networks (on the order of hundreds of thousands to millions of links) with tens of millions of vehicle trips per day to resolve system-wide impacts on large urban regions.

Other prior work in utilizing HPC for simulating traffic systems have focused on various aspects of parallelization and performance optimization. In one early article on this subject [12], they describe two parallel implementations for microscopic simulation on a data-parallel (vector) architecture and a message passing architecture, simulating a road network with up to 20,000 miles and 200,000 vehicles up to \(2.7\times\) real-time speeds. An improved geographic recursive bisection algorithm was presented in [53] for computing the parallel domain decomposition so that the workload is better balanced across processors. They evaluated their approach on a small network with 232 roads and 670 connections between roads, and observed a parallel speedup of up to \(4.1\times\) using 336 CPUs. In [54], they focus on evaluating the impact of a dynamic load balancing strategy for reducing computational load imbalance and improving parallel scalability, showing that parallel speedup improved from \(3.1\times\) to \(4.2\times\) for a network with 80,000 links and 1.2 million trips using 32 compute cores. Following that work, they introduced a method for reducing the overheads of conservative synchronization in a nanoscopic simulation to improve performance by relaxing the causality of simulated events [55]. In that article, they used a similar sized network of 80,000 links on a compute cluster consisting of three compute nodes, each with 16 cores, and evaluated the impact of relaxing causality on the accuracy of the simulation.

In Reference [24], a parallel simulation with a dynamic route solution module is presented for a test network of 62 links, showing scaling performance up to 6 processors with a parallel speedup of up to \(2.3\times\) versus serial. Their parallel dynamic rerouting methodology, where vehicle routes are iteratively updated in a synchronous fashion, does not simulate the individual request/response communication mechanism that occurs between the vehicles (or smart navigation devices) and the vehicle controller actors, which serves to aggregate network congestion data and handle rerouting requests. Since the routes are solved iteratively and synchronously, their methodology is more appropriate for solving the traffic assignment problem, rather than studying the sensitivities in a dynamic rerouting mechanism (e.g., frequency of reroute checks, thresholds for rerouting). Furthermore, they evaluated their method on a network with 62 links for up to one hour of simulated time, which they reported takes over 15 minutes of computational time and reaches its maximum parallel speedup with 4 single-core processors (the execution time actually increases when more than 4 processors are used). This is quite different in scale compared to the simulated San Francisco Bay Area system we examine in this article, which contains over one million links with 19 million vehicles (each with its own specific origin and destination nodes), and scales to much higher core counts, achieving a \(146\times\) parallel speedup on 256 compute cores.

In Reference [6], the authors study the impact of dynamic rerouting (re-planning) in the context of VANets using SUMO and evaluate the impacts of dynamic rerouting on both mobility and ad-hoc network connectivity. In contrast to our work, their analysis did not formulate the dynamic rerouting mechanism as a distributed process that involves coordination between link and controller actors on different compute nodes in a parallel computing environment. SUMO uses a serial simulation engine, so all data is generated and available to the one thread executing at all times, whereas in a distributed memory platform, messages must be sent between compute nodes to share their simulation state. Moreover, that work did not investigate the sensitivity of additional parameters such as setting a minimum time savings threshold on the new route before taking a reroute action.

Our article goes beyond these prior works by (1) focusing on the design, implementation, and evaluation of dynamic vehicle rerouting in the context of a distributed-memory traffic simulation (which requires passing messages to share state between compute nodes), and (2) evaluation, performance analysis, and parameter sensitivity study of a large-scale HPC simulation with millions of road network links and tens of millions of vehicle trips. The problem of modeling dynamic vehicle rerouting behavior adds considerable computational load and model complexity to the simulation to coordinate link cost updates and compute new vehicle routes. Traffic simulation systems use a variety of models and algorithms based on the problem that is being addressed. Geographical scale, modeling fidelity, time horizon, and desired output data all lead to a varied landscape of simulation tools. However, to the best of our knowledge, this article presents the first simulation framework to model the impact of dynamically rerouting vehicles in a large urban region at scale with millions of nodes and links and millions of vehicles in short execution times, as well as present a rerouting parameter sensitivity discussion and detailed model validation analysis.

Mobiliti is a distributed-memory, parallel simulation framework that runs on high-performance computing platforms. It uses optimistic PDES based on the Time Warp [28] protocol to model the traffic congestion in the network and the rerouting behavior of vehicles in the system. The optimistic nature of the simulator allows events to be speculatively executed and then rolled back if needed to maintain causality in the simulation. In order to avoid excessive synchronization overheads and enable a high degree of parallelism, our optimistic PDES simulator utilizes a computational design similar to the actor model of concurrent computation [2, 5]. In the actor model, the system is composed of a set of actors, which are entities that can execute concurrently with respect to one another and can only obtain information from each other through message passing. Moreover, with the arrival of each message, an actor may do computation that modifies its local state and may send new messages to other actors. There is no shared global state, thus avoiding the need for expensive locks, mutexes, or atomics and enabling distributed-memory parallelism.

In our optimistic PDES simulator, the entire state of the simulated system is divided among entities (called logical processes in the PDES literature [23]), which correspond to the actors in the actor model. The logical processes send events to one another, which correspond to the messages in the actor model. As with the actor model, each logical process may only do computation that updates its local partition of the system state (again, no shared global state) and send new events to other logical processes. A key distinguishing feature is that our simulator also utilizes a synchronization protocol (Time Warp) to enforce the correct execution ordering of events, even if the events arrive out of order over the network. For the remainder of this article, we will refer to the logical process entities as actors since that term is more familiar to researchers outside the PDES community.

The overall workflow of the simulator is presented in Figure 2, which shows a concise subset of the inputs, stages of initialization and simulation, and outputs produced by the simulator. During the simulation phase, the links are modeled as actors, and the vehicles are represented as events that are passed from link to link as they travel through the road network. Figure 3 illustrates the representation of network links as actors passing vehicle events between them.

For background on the simulator’s implementation, please see our previous article [14]. This section describes relevant updates to the simulation that improve the accuracy of the link model and enable vehicle dynamic rerouting capabilities. Specifically, we have added mechanisms inside the link actor to enforce more accurate link timing constraints and storage capacity constraints, and we have added a new set of vehicle controller actors that can be queried by vehicles to compute better routes using the current congested state of the network.

Vehicles are rapidly gaining the ability to utilize up-to-date road congestion information to re-route their paths during their trips through smartphone navigation apps. In this article, we presented a computational methodology to model varying degrees of dynamic rerouting in a large-scale transportation system using the Mobiliti high-performance parallel discrete event simulator. We described updates to our link actor model to capture the effects of link congestion, timing constraints, and storage capacity constraints. We have detailed the implementation of new VehicleController actors and the events required to update their knowledge of the system state and service dynamic rerouting requests. Because we have designed our simulator to scale over distributed memory parallel computing platforms, we can simulate one model day of the San Francisco Bay Area with 19 million vehicle trips and 50 percent dynamic rerouting penetration over a road network with 0.5 million nodes and 1 million links in three minutes of simulation time. We conducted an analysis of system-level impacts when varying the dynamic rerouting penetration rate at 10% increments and examined the varying effects on different functional classes and geographical regions. Finally, we presented a validation of the simulation results compared to real-world data sources.