This research is cited from: Wu, Y., Tan, H., Qin, L., & Ran, B. (2020). Differential variable speed limits control for freeway recurrent bottlenecks via deep actor-critic algorithm. Transportation Research Part C: Emerging Technologies, 117, 102649.

Happy to answer any questions you have. Please email us at kaimaogege@gmail.com or Yuankai.Wu@mcgill.ca.

fcd.rou.xml --- The vehicle routes, this can be generated by the rm_vsl_co.writenewtrips function in network0.py

floating_car.sumo.cfg --- The SUMO simulation configuration

map.net.xml --- The transportation freeway networks - An 875.51 m, on- and off-ramp inclusive, northbound freeway section of I405 in California, USA. The map is downloaded from openstreetmap, and processed by netedit of SUMO.

ns.det.xml --- This is the detectors used to collect traffic state and parts of reward signals.

output_sumo.xml --- The output information of the simulation.

networks0.py --- The python script for running the code, it provides a interface with the reinforcement learning agent.

ddpg0.py --- The script for training DDPG agent for DVSL control.

The demand is defined by OD matrix. The simulation lasts for 5 hours, each hours' demand of each route are modeled as Poisson distribution. The mean of the Poisson distribution is given in defined in the script.

We use two types of vehicles with different length, and the vehicle is randomly set with two types of car following behavior.

The desire of the vehicle to driving under speed limits is defined as a normal distribution using speedFactor="normc(1,0.1,0.2,2)，the tendency to perform lane change to gain speed is defined as lcSpeedGain = "1".

If incidents = True, a random incident will happen. This is done by set vehicle speed to 0 during a period of simulation.

get_step_state() --- a function used to collect occupancy rate of state detectors. It provides the state variable for RL agent.

set_vsl(v) --- set the speed limits for different lanes.

calc_outflow() --- calculate the difference between outflow and inflow.

calc_bottlespeed() --- calculate the bottleneck speed by the detector located in the merge area.

calc_emission() --- calculate the pollution air emission of the transportation system.

start_new_simulation() --- start a new simulation. You can choose whether to rewrite a new route file by setting write_newtrips, e.g, net.start_new_simulation(write_newtrips = False) means that the simulatio environment wouldn't write a new trips.

run_step() --- run one step with the set speed limits. This function will output the simulationstep, state, reward variables and emissiong in one step.

close() --- it will closes the simulation

writenewtrips() --- You can change the attribute of the vehicle by editing the speedFactor and lcSpeedGain value

The agent is based on an actor-critic framework. In this version, we didn't use the exploration in order to make the agent converges.