Hidden amongst Chaos

Weather shapes societies in various ways, from daily routines to infrastructure, but the rapid climate change caused by human activities is exposing vulnerabilities to extreme weather events. While medium-range weather forecasting has improved, long-term forecasts spanning weeks to months, known as subseasonal-to-seasonal (S2S) timescales, remain challenging. The thesis focuses on using data-driven methods to improve the skill and understanding of subseasonal to seasonal (S2S) weather forecasts, with a particular emphasis on North America. The author explores four main areas: (1) predicting temperature extremes in the eastern United States (US), (2) studying the ocean-atmosphere interaction driving predictability in the eastern US, (3) predicting harvest failure in the eastern US, and (4) identifying challenges, opportunities, and a vision for exploring S2S dynamics and predictability using data-driven methods. (1) For temperature extremes, an algorithm is developed to extract a reliable preceding sea surface temperature (SST) pattern from the North Pacific, improving forecast skill for heatwave events. The trade-off between extremity and spatial aggregation is explored, indicating compromises needed for reliable forecasts at longer lead-times. Predicting event probabilities within wider time windows enhances forecast skill for moderate hot events up to 60 days ahead. (2) To address the issue of trustworthiness in purely statistical machine learning models, the author emphasizes the importance of incorporating physical understanding into forecast models. They employ causal discovery methods to learn physical relationships from data. By applying a causal discovery algorithm, they study the interaction between the atmospheric Rossby wave and the underlying ocean, revealing that summer temperature predictability in the eastern US originates from low-frequency variability in the north Pacific. The study demonstrates that the low-frequency Pacific variability, driven by atmosphere-to-ocean forcing and two-way feedbacks in winter and spring, leads to an upward forcing from the ocean to the atmosphere in summer. The presence of a strong horseshoe-shaped SST pattern in spring enhances predictability by causing more frequent and persistent atmospheric waves, which result in a high-pressure system, higher temperatures, and reduced rainfall in the eastern US. (3) The winter-to-spring horseshoe sea surface temperature (SST) pattern holds significant importance as it suggests the potential predictability of hot and dry weather in the mid-to-eastern United States (US) at long lead-times. This predictability opens up opportunities for the agricultural sector, enabling informed decisions to be made prior to the planting season. The author employs a response-guided dimensionality reduction method and a causal inference-based selection step to extract reliable input features from observational SST and soil moisture datasets. Using this approach, the forecast model successfully predicts poor soybean harvest years as early as February 1st, several months before sowing. This provides farmers with valuable information for decision-making, such as adjusting sowing density, avoiding drought-prone areas, or selecting drought-resistant seeds. (4) The thesis suggests that S2S forecasting potential and value may have been underestimated. However, challenges remain, such as establishing best practices for data-driven forecasting. The author advocates for dedicated open-source software and a collaborative community. Furthermore, operationalizing forecasts and supporting the required infrastructure are essential for societal benefits.