Interactive Jupyter Notebooks for the visual analysis of critical choices in Global Sensitivity Analysis

<p>With earth system models growing ever more complex, a comprehensive, transparent and easily communicable analysis of the interacting model components is becoming increasingly difficult. Global Sensitivity Analysis (GSA) provides a structured analytical approach to tackle this problem by quantifying the relative importance of various model inputs and components on the variability of the model outputs. However, there are a number of critical choices needed to set up a GSA, such as selecting the appropriate GSA method for the intended purpose and defining the inputs to be tested and their variability space. In this work, we test the use of interactive visualization to analyze the impacts of such critical choices on GSA results and hence achieve a more robust and comprehensive understanding of the model behavior. To this end, we combine the Python version of the Sensitivity Analysis For Everybody (SAFE) toolbox, which is currently used by more than 2000 researchers worldwide, with the literate programming platform Jupyter Notebooks, and interactive visualizations. Unlike traditional static visualization, interacting visualizations allow the user to interrogate in real time the impact of user choices on the analysis result.  Due to computational constraints of most earth system models, not all impacts can be visualized in real time (e.g. only those not requiring the model to be re-run). In those cases where a model needs to be re-run (e.g. to test the impact of the definition of the inputs space of variability), interactive visualizations still offer a useful tool, which allows to highlight only specific features of interest (e.g. behavior of the input/output samples for extreme values of the inputs or outputs). Jupyter Notebooks, by combining text, code and figures, enhance the transparency, transferability and reproducibility of GSA results. Interactive visualizations strengthen the understanding of the impacts of the choices made to run GSA and the robustness of GSA result (e.g. by being able to easily assess the impact of varying output metrics or the GSA method on the analysis results). In general, this work offers an example of how the use of notebooks and interactive visualizations can increase the transparency and communication of complex modelling concepts.</p>