Modeling of Bus Holding Strategy in Public Transit Systems with Multi-Agent Reinforcement Learning

Excessive fluctuations in travel time between stops and demand at bus stops during bus operations can lead to operational instability in bus systems, such as bus bunching. To tackle this issue, this paper presents a dynamic bus holding control strategy leveraging multi-agent reinforcement learning to stabilize bus system operations and prevent bus bunching. First, bus motion system is constructed, and the rules for bus operation and passenger behavior are defined. Then, agent-based transit operation management is established, the elements of the multi-agent reinforcement learning framework are outlined, and a centralized training and decentralized execution method is proposed. Additionally, an event-driven simulation environment is developed for training and testing the agents. Finally, extensive numerical simulations are conducted to evaluate the proposed method against baseline approaches using various performance metrics. The results demonstrate that the proposed method effectively captures the dynamics of the bus system and accounts for the long-term impacts of current decisions, resulting in the most balanced bus trajectories, optimal passenger load distribution, and minimal total holding time.