Why observations matter more than ever in the age of AI forecasting

Artificial Intelligence and Machine Learning are changing weather forecasting faster than anyone anticipated. Numerical Weather Prediction (NWP) models that take hours to run are now accompanied by Machine Learning Weather Prediction (MLWP) models that deliver global forecasts in minutes. Yet in the age of AI forecasting, observations matter more than ever, because even the most advanced models are limited by the data they see.  

It is easy to assume that smarter algorithms reduce the need for measurements. In reality, AI does not eliminate the need for high-quality observations. It amplifies it.

High-quality forecasts need high-quality observations

In addition to MLWP models that rely on initial states from NWP model-based analysis, Direct Observation Prediction (DOP) models are a field of intense research. These DOP models learn the atmosphere’s expected future observations without any data assimilation or physics, only based on features in the observation data used to train these models.

The old statement “garbage in – garbage out” might be replaced with a statement: “if your training target is garbage, your model’s output is garbage”. In order to train an ML-based weather model (MLWP or DOP), the training target (reanalysis field or observations, respectively) must be meteorologically meaningful and of the quality expected from the forecasts.

The shifting role of observations in AI forecasting

The rise of machine learning-based weather prediction shifts how observations are used. They now serve three critical functions:

• Target data for training. The quality and representativeness of the target data directly determine the maximum achievable quality of the machine learning forecast. No amount of clever architecture can compensate for poor training targets. In MLWP, observations are the key ingredient of reanalysis datasets, whereas in DOP, the observations are used directly as targets.
• Predictor and inference data. During training and operational runs of DOP models, observations must comprehensively describe the atmospheric state. No machine learning model can compensate for missing atmospheric features in the input data. Meteorological expertise remains essential in selecting the right data to maximize descriptive power.
• Validation. Model outputs must be compared against relevant meteorological observations, such as surface station data or radiosonde profiles, to assess the performance of the forecast system.

Speed makes real-time observations critical

Machine learning models generate forecasts much faster than traditional numerical weather prediction. NOAA's new AI-driven Global Forecast System AIGFS completes a 16-day forecast in approximately 40 minutes using only 0.3 percent of the computing resources of the traditional model. ECMWF’s (European Centre for Medium-Range Weather Forecasts) AIFS model’s computing time is dominated by NWP-based data assimilation. These models are MLWP models.

As the DOP models don’t include any NWP-based data assimilation that might take 40 minutes or an hour to run on a supercomputer, the complete end-to-end forecast cycle can be accomplished in a matter of minutes. This means that the latency of the forecast availability is not dictated by supercomputer processing time, but by the delay in getting the observations to the DOP computing process. The speed of observation data delivery becomes more important than before.

Are our observation networks ready?

As machine learning model development accelerates, regular retraining with new datasets and new architectures will become standard practice. This raises a critical question: Are current meteorological observation networks sufficient for the needs of future modelers?

The trustworthiness of machine learning-based weather prediction depends directly on the quality and comprehensiveness of both training and verification datasets. To surpass the capabilities of traditional numerical weather prediction, these datasets must offer reliable, comprehensive targets for model training and robust benchmarks for performance evaluation.

Better observations, smarter AI

The message is clear: AI does not eliminate the need for world-class observations. It amplifies it. While maintaining the traceability of physical observations remains fundamental, well-supervised AI methods can maximize the utility of observational data in both traditional and machine learning-based weather prediction models.

The transformation of weather forecasting through artificial intelligence is real and accelerating. But the foundation of that transformation is observational excellence. The better our observations, the better AI learns the atmosphere. And the better AI learns the atmosphere, the better it protects lives and property through more accurate, timely forecasts.

Learn more from my poster at AMS: Maximizing the Value of Meteorological Observations in the Age of Artificial Intelligence for Weather Prediction

Dive deeper into the topic from our latest article in Meteorological Technology International: