Multi-modal Photovoltaic Anomaly Detection via Synergistic Graph-Contrastive Regularization

Photovoltaic (PV) anomaly detection is crucial for the effective operation and maintenance of PV systems. Although anomaly detection based on computer vision offers a promising solution, existing methods that rely on a single modality of images still have inherent limitations in practical scenarios. This study introduces a new comprehensive multi-modal dataset named PV-VISIR, which contains 9 categories and a total of 2146 pairs of labeled visible light and infrared images. Based on this dataset, a novel dualbranch multi-modal fusion framework is proposed to achieve more accurate anomaly detection. Experimental results demonstrate a significant improvement in detection performance, achieving an mAP@0.5 of 0.781, which is 9.8% higher than the YOLOv8 baseline. The performance improvement can be largely attributed to two proposed modules, namely Label-driven Cross-Modal Registration (LCMR) and Synergistic Training via Prototypical Graph-Contrastive Regularization, which contribute 5.3% and 2.7% to the overall gain in mAP@0.5, respectively. The developed PV-VISIR dataset and the proposed framework are expected to jointly drive innovation in related methodologies and advance the development of PV monitoring technology.