A deep learning multimodal method for precipitation estimation

Deep-Learning (DL) is a sub-field of Machine-Learning (ML) whose popularity has grown exponentially during the last decade thanks to its numerous successes related to artificial intelligence, e.g. visual object recognition and detection, image generation, natural language processing or speech recognition. But, like other ML algorithms, DL can be applied to a broader variety of problems, as long as the necessary data is available to train the model. In this aspect, the field of meteorology satisfy the data requirement. Particularly, DL could improve the accuracy of precipitation estimation. Accurate precipitation estimation is a very important product of weather institutes for water supply monitoring, and as input and validation for Numerical Weather Prediction models and precipitation nowcasting. Currently, precipitation are mainly estimated using both radar and rain-gauge data, but an important limitation of this method is the limited coverage of these measurements. A possible solution to this problem could come from satellite radiometers observations. Unfortunately, estimating precipitation accurately from satellite radiometer data is challenging. As a solution, we developed a DL method to merge rain gauge measurements with a ground-based radar composite and satellite radiometer imagery. The proposed convolutional neural network, composed of an encoder–decoder architecture, performs a multiscale analysis of the three input modalities to estimate simultaneously the rainfall probability and the precipitation rate. We used SEVIRI infrared channels, the OPERA radar composite and the measurements of automatic rain gauges. The training of our model and its performance evaluation are carried out on a dataset spanning 5 years from 2015 to 2019 and covering Belgium, the Netherlands, Germany and the North Sea. Our results for instantaneous precipitation detection, instantaneous precipitation rate estimation, and for daily rainfall accumulation estimation show that the best accuracy is obtained for the model combining all three modalities. We show that the combination of rain gauge measurements with radar data allows for a considerable increase in the accuracy of the precipitation estimation, and the addition of satellite imagery provides precipitation estimates where rain gauge and radar coverage are lacking. We also show that our multi-modal model significantly improves performance compared to the European radar composite product provided by OPERA and the quasi gauge-adjusted radar product RADOLAN provided by the DWD for precipitation rate estimation. Additionally, we carried a study of our method on the case of the extreme precipitation event of July 2021 that affected Belgium and Germany, which caused huge societal and economical damage.