CMIP5 Model-analog Seasonal Forecast Skill: A Metric for ENSO simulations

Abstract:

Ding et al. (2018) showed that tropical Indo-Pacific SST forecast skill from the North American Multi-Model Ensemble (NMME) can be matched or even exceeded using a "model-analog” method applied to existing control runs from each NMME model. States taken directly from a long control run are chosen as model-analogs of each observed initial state during 1982-2015; then, their subsequent evolution in the control run provides a model-analog forecast. Subsequently, Ding et al. (2019) applied the model-analog method to preindustrial simulations from 28 different CMIP5 CGCMs. For most of the CMIP5 models, model-analogs provide skillful SST and precipitation hindcasts, with some as skillful as operational CGCMs.

Here, we show that model-analog hindcast skill can also be used as a metric for ENSO simulations, since it evaluates how each CMIP5 model reproduces the observed evolution of ENSO. As before, model-analogs are determined in the tropical Indo-Pacific domain, using observed monthly SST and SSH anomalies. Analog ensembles corresponding to the observed anomalies are then identified in each of the 28 CMIP5 simulations. Hindcasts of SST (1961-2015) and precipitation (1979-2015) are then made for leads of 1-12 months in the tropical Indo-Pacific. The long-term skill of the seasonal forecasts of tropical Pacific SST and precipitation represent how well each CMIP5 model’s attractor corresponds to nature’s attractor, in contrast with the more static aspects of ENSO measured by most existing metrics.

While precipitation is not used to select the analogs, the model-analog precipitation forecast skill emerges as a key identifier of the better tropical Indo-Pacific simulations. Several models with the best representation of observed SST anomalies actually display very poor precipitation forecast skill. The models with the highest precipitation forecast skill show the most realistic mean states in the tropical Pacific. In addition, the most skillful models at predicting precipitation also better simulate the interannual variability in the equatorial Pacific. These results show a direct relationship between the mean model error and seasonal forecast error, which has been difficult to determine using more traditional forecast approaches. The potential for using model-analogs to evaluate CMIP6 models is also examined.