Reframing Rain as Noise for rainfall measurement: A Lightweight Framework to Video-Based Rainfall Sensing

Video-based rainfall measurement is a promising approach for rainfall perception, but rain streaks in videos typically appear as sparse, weak-textured signals, making their accurate detection challenging, particularly for standard cameras with limited computational resources. To address the crucial trade-off between measurement accuracy and computational efficiency, we propose a lightweight machine learning method named Coupled Image Enhancement and Gradual Aggregation Network (CEGA-NET). Unlike traditional methods relying on the physical falling characteristics of raindrops, CEGA-NET treats rain streaks as image noise, reconstructing structural background information through image enhancement and extracting clean rain signals via image differencing. Additionally, it employs a simplified progressive temporal aggregation strategy that entirely removes the optical-flow alignment modules, significantly improving computational efficiency while preserving measurement accuracy. Experiments on real-world rainfall scenarios and semi-synthetic datasets demonstrate that CEGA-NET achieves a mean relative error of only 11.7% across 11 independent rainfall events with diverse intensities and both daytime and nighttime conditions, outperforming existing video-based rainfall estimation methods by more than 10%. Additionally, CEGA-NET achieves a processing speed of up to 26 frames per second at edge-device-level resolutions—representing an improvement of three orders of magnitude over conventional CNN-based methods and two orders of magnitude over physics-based methods (typically 1 frame per second). These results highlight the lightweight design and practical deployability of CEGA-NET, enabling efficient training and real-time rainfall monitoring on resource-constrained devices. Overall, this study substantially reduces both the error and computational cost associated with video-based rainfall measurement, providing a practical and low-cost solution for hydrological monitoring applications.