Investigating the effects of hyperparameter sensitivity on machine learning algorithms for PV forecasting

Machine Learning (ML) models have been introduced in the past, and users have debated whether to tune the hyperparameters of the models. This study investigates the effects of tuning the hyperparameters of the ML models and summarizes the models that are most sensitive to hyperparameter tuning. This study leveraged the historic energy production data of two already operational PV plants. Four state-of-the-art ML models, namely Decision Trees (DT), Random Forest (RF), K-Nearest Neighbors (KNN), and Support Vector Regression (SVR) were investigated. All the ML models were trained with the same training features (meteorological estimates) obtained from the National Aeronautics and Space Administration’s (NASA) Power project, with the daily PV energy production selected as the target variable. Models were developed and executed with default and tuned hyperparameters using an 85-15% train-test split. The results revealed that all the models showed improved performance with the tuned hyperparameters. However, the DT and SVR models depicted significantly improved RMSE after tuning of the hyperparameters. The RMSE of DT improved from 111 kWh/d to 75 kWh/d for one plant and from 442 kWh/d to 270 kWh/d for the second plant after tuning the hyperparameters. Similarly, the RMSE of SVR improved from 59 kWh/d to 50 kWh/d in the first case, and in the second case, the improvement of RMSE from 536 kWh/d to 294 kWh/d was observed. The efficiency of the RF and KNN models also improved to some extent after tuning, but the RMSE closely agreed with the default hyperparameters in one case study, making the RF and KNN less prone to hyperparameter sensitivity. This study concluded with the finding that it is necessary to tune the hyperparameters of the DT and SVR models, specifically for energy forecasting. Moreover, the results of this study also highlight the significance of meteorological estimates from NASA’s Power project, as models successfully discerned the complex energy forecast patterns. The dataset is deemed suitable for energy forecasting for areas with sparse ground-based observatories and may serve as a baseline dataset for training the ML models.