Experimental Study of Multi-Agent Reinforcement Learning Approaches for the Coverage Path Planning Problem

Purpose. The coverage path planning problem for groups of autonomous agents over a given target area is relevant to a wide range of applied systems. An increase in the dimensionality of the environment and in the number of interacting agents leads to higher coordination complexity and longer times to achieve complete coverage. Additional difficulty arises from agents’ limited observations, which results in a control problem formulated under partial observability and stochastic dynamics. In this context, the development of specialized intelligent control approaches that minimize the time required to cover the target area under decentralized decision-making is of significant interest. The aim of this work is to develop a framework for training control models for groups of homogeneous autonomous agents in the coverage path planning problem, ensuring minimization of the expected time to achieve full coverage under a partially observable Markov decision process. Methods. To formalize the problem, a partially observable Markov decision process model is employed, including a description of system states, agents’ action and observation spaces, probabilistic environment dynamics, and a reward function. The synthesis of a group control policy is based on deep reinforcement learning methods for multi-agent systems, oriented toward decentralized execution with centralized training using a critic. Performance evaluation is carried out using simulation-based experiments in a discrete two-dimensional grid environment. Novelty. This work introduces a unified experimental environment for comparing multi-agent architectures in the coverage path planning problem. It is shown that the use of small-scale maps limits the statistical significance of several agent coordination metrics, which motivates the transition to larger maps. Typical sources of coverage performance degradation related to boundary effects and small target area fragments are identified, and an environment modification is proposed to mitigate their impact. Results. A formal problem statement is presented, and an approach to synthesizing a group control policy is proposed that enables complete coverage of the target area within finite time. Simulation results confirm the feasibility of effective agent coordination and a reduction in coverage time compared to uncoordinated strategies, while preserving decentralized control. Experimental studies reveal differences in coverage dynamics and agent coordination across the considered architectures. The theoretical significance of this work lies in the advancement of methods for formalizing and solving multi-agent coverage problems under partial observability. The practical significance is determined by the applicability of the obtained results to the development of intelligent control systems for groups of autonomous mobile agents, including applications in monitoring, reconnaissance, and robotic systems. The results can be used in the design of multi-agent coverage systems and in the comparative analysis of agent control methods in complex discrete environments.