Opening the black box of LSTM models using XAI

Abstract:

Dramatic success in machine learning methods, especially deep neural networks, has led to an exponential increase in their applications to Earth system science domains. At the same time, there has been persistent skepticism in the effectiveness of these methods due to our inability to explain how the information is propagated through layers of neural networks. Explainable AI methods are essential to open the black boxes of neural networks and understand the rationale behind neural networks, and eventually increase researchers’ trust in neural network-based models representing underlying mechanisms that drive system dynamics in space and time. We explored the use of an XAI technique called layer-wise relevance propagation (LRP) on a bi-directional long short-term memory (LSTM) model we built to predict soil respiration from soil temperature, soil moisture and precipitation in semi-arid ecosystems. LRP quantifies the contribution from each input features to support its predictions, i.e., explainability, by propagating the prediction backward in the neural network using a set of pre-designed propagation rules. Our study demonstrates that LRP is able to capture the shift in relative importance among soil temperature, soil moisture and precipitation in controlling soil respiration under different soil wetness regimes and precipitation scenarios. In particular, we found two distinct patterns before and after the rewetting: soil temperature is the dominant driver in providing diurnal signals in dry period, while soil moisture quickly comes into play after each rainfall event; during the wet period, soil moisture competes with soil temperature to control soil respiration. The relative contribution from past and future soil temperatures to soil respiration predictions also successfully explains the effect of diel hysteresis. The successful application of XAI we demonstrate in this study shows huge potential in opening the black box of neural networks to enable data-driven discovery to accelerate our predictive understanding of complex Earth system.