Optimizing Electric Vehicle Battery Swapping Services: A Behavior Cloning-Integrated Reinforcement Learning Framework for Charging Decisions

<div> Battery swapping stations (BSS) enable electric vehicle drivers to replace depleted batteries with fully charged ones within minutes, addressing range anxiety and charging time concerns. BSS operators must determine optimal battery charging schedules under uncertainty from stochastic demand, time-varying electricity prices, and capacity constraints to ensure profitability. </div> <div> This paper formulates the BSS charging problem as a periodic Markov Decision Process and develops a Demonstration Augmented Actor-Critic (DAAC) framework that integrates behavior cloning with deep reinforcement learning. The approach addresses the sparse reward problem where charging costs are immediate but revenues are delayed until batteries are used for swaps.&nbsp; </div> <div> Our methodology features three innovations: engineered state representations capturing current operations and forward-looking conditions, reward shaping providing denser feedback signals, and two-phase training combining offline pretraining on expert demonstrations with online fine-tuning.&nbsp;<span>Using real-world data from NIO's Shanghai operations, we validate DAAC against Proximal Policy Optimization, standalone Behavior Cloning, and threshold-based policies. Extensive simulations demonstrate that DAAC consistently achieves the highest total profit while maintaining competitive service quality across diverse operational scenarios. Under both static and dynamic demand patterns, DAAC outperforms all baselines in profitability while effectively minimizing lost sales.&nbsp;</span><span>Component analysis confirms that both offline pretraining and reward shaping significantly improve performance. The framework offers actionable insights for BSS operators on optimal charging strategies and supports informed investment decisions across various battery inventory configurations, thereby advancing the battery-as-a-service business model.</span> </div>