A chronological deep reinforcement learning environment for flexible job shop scheduling problems

The Flexible Job-shop Scheduling Problem (FJSP) is a classical combinatorial optimization problem that has a wide-range of applications in the real world. The deep reinforcement learning (DRL) scheduling methods for FJSP are mainly focused on the design of scheduling Agent, overlooking the modeling of DRL environment. This paper presents a chronological DRL environment for FJSP based on discrete event simulation and an end-to-end DRL scheduling model is proposed based on the proximal policy optimization (PPO) where a hybrid prioritized experience replay method is employed to accelerate the training process. Furthermore, a short novel state representation of FJSP is proposed based on two state variables in the scheduling environment and a new comprehensible reward function is designed based on the scheduling area of machines. Experimental results on public benchmark instances show that the performance of simple priority dispatching rules (PDR) is improved in our environment and our DRL scheduling model obtains competing performance compared with OR-Tools, meta-heuristic, DRL and PDR scheduling methods.