Beyond Sensors: Text-Augmented Reinforcement Learning for Real-Time and Robust Traffic Signal Control

Traffic signal control aims to minimize congestion and maximize efficiency in urban traffic systems. While recent reinforcement learning algorithms have shown promising performances, they are solely based on structured sensor inputs and cannot cope with unexpected disruptions described in natural language, such as accidents, construction, or extreme weather. Current RL methods experience 3% to 5% performance degradation under these severe disruptions. In this paper, we propose TARL-TSC, a framework to integrate natural language event description with traffic observations using learned fusion mechanisms. TARL-TSC uses a frozen pre-trained text encoder for semantic understanding and presents two designed fusion architectures for cross-model integration: attention-based fusion and adaptive gating fusion, and intersections are spatially coordinated by a Graph Attention Network. Experimental results on real traffic networks show that TARL-TSC performs better than baselines in several aspects. TARL-TSC reaches a throughput of 869.1 vehicles per episode on the Hangzhou 4×4 network, improving over state-of-the-art MPLight by 2.9% (p = 0.023). Cross-city comparisons also show strong generalization over unseen urban networks, without fine-tuning specifically for certain cities. TARL-TSC achieves stable gains of 1.2% to 2.9% on the Jinan and New York networks. TARL-TSC maintains a stable performance under event distribution with minimal degradation of only 1.3%, while the baselines decline about 5%. Besides, the framework achieves real-time inference within 8.3 ms, which demonstrates its practical deployment ability. TARL-TSC sets a new benchmark for event-aware traffic signal control, proving the feasibility of using text-augmented RL in achieving better robustness without sacrificing operational efficiency.