A researcher from SPbGASU investigated how deep reinforcement learning and simulation modeling can be used to optimize the operation of autonomous taxis within an urban road network. The developed models account for fluctuating passenger flows, traffic volume, and vehicle speeds.
The study was conducted by Aleksey Namestnikov, DSc in Engineering and Professor at the Department of Information and Mathematical Modeling Technologies. The work focuses on integrating reinforcement learning models with simulation models, using the behavior of autonomous taxis in urban traffic conditions as a case study.
Aleksey NamestnikovThe development of driverless transport technologies opens up opportunities to integrate autonomous vehicles into urban public transport systems. A key challenge, however, is ensuring accessibility for passengers: driverless taxis need to be located where demand is highest. This demand, in turn, can fluctuate based on passenger flow intensity and traffic conditions.
To address this task, the study employed AnyLogic simulation tools. Based on these, hybrid models were developed that combine discrete-event and agent-based approaches to model a fleet of autonomous taxis as integrated intelligent systems. These models were augmented with deep reinforcement learning technologies.
This approach makes it possible to model the behavior of autonomous taxis while accounting for the changing conditions of the urban transport system and to determine the optimal vehicle positioning within the road network. Consequently, it becomes possible to simultaneously consider multiple factors—such as fluctuations in passenger flow, the number of vehicles, and urban traffic speeds.
The scientific novelty of the project lies in the development of a new method for the optimal positioning of autonomous taxis within an urban road network, accounting for stochastic demand. Unlike existing approaches, the proposed method is based on the integration of simulation models and deep reinforcement learning models.
Furthermore, the study involved the development of a new Deep Q-Network (DQN) model based on deep reinforcement learning. A distinctive feature of this model is its neural network architecture, which was adapted to the specific task at hand.
"The development of multi-agent deep reinforcement learning systems enables a shift toward more complex modeling of autonomous vehicle behavior. A promising trend is training them via simulation using geospatial data," noted Aleksey Namestnikov.
Aleksey Namestnikov has been conducting research on the application of intelligent technologies in the design of technical systems since 1997. He has published approximately 120 scientific papers on this subject. Moving forward, he plans to continue developing multi-agent deep reinforcement learning systems to train behavioral policies for autonomous vehicles, utilizing simulation modeling and geoinformation data as a foundation.
The study was conducted under a grant for research activities to be carried out by academic staff of SPbGASU in 2026.