Climate-Invariant Nets: Using Physical Rescalings to Help Neural Networks Generalize to Out-of-Sample Climates

Abstract:

Data-driven algorithms, in particular neural networks, can emulate the effect of sub-grid scale processes in coarse-resolution climate models if trained on realistic, high-resolution atmospheric simulations. However, they make large errors when evaluated outside of their training set, limiting their usefulness for the climate community. Furthermore, they rely on high volumes of convective heating and moistening data, preventing the use of sparse field campaign observations of convection and confining data-driven algorithms to synthetic training data from imperfect models. Here, we propose a framework to incorporate physical rescalings within neural networks: By aligning the distributions of both input and output variables across climates, we transform an extrapolation problem into an interpolation one and significantly improve the ability of neural networks to generalize to unseen climates. These improved neural networks, referred to as “climate-invariant nets”, generalize well to out-of-sample climates even when only exposed to small portions of training data from a different climate, which suggests that they learn more general representations of the interaction between convection and large-scale thermodynamics. Additionally, climate-invariant nets facilitate transfer learning between idealized and realistic atmospheric data, with the potential to assist modern data assimilation of field campaign observations into global climate models. Finally, our results suggest the potential existence of a “climate-invariant” mapping from the large-scale climate to cloud-scale thermodynamics, opening the door to improving the performance and stability of data-driven climate simulations.