Interpretation and evaluation of machine learning-based earthquake monitoring

Abstract:

Geoscientists are increasingly adopting machine learning (ML) to analyze large sensor data sets for natural hazards monitoring. In contrast with traditional modeling approaches that perform calculations based on physical laws or scientific knowledge, ML methods build data-driven models that learn representations of and relationships within data. Many ML models are “black boxes” that are difficult to inspect due to the large number of model parameters and complex relationships among them. The black-box nature of these models can be a limitation in geoscience applications, as researchers are often leveraging ML algorithms to extract scientific insights from data, characterize hazards, or provide decision support information. To build trust in ML-based systems for these applications, scientists require a framework for evaluating model reliability and ensuring that data-driven models capture meaningful relationships rather than spurious correlations.

This presentation will explore trustworthy machine learning for geoscience applications. Building trust in ML-based systems requires evaluating model performance using metrics beyond accuracy. This includes a broader set of criteria such as sensitivity to noise or perturbations, robustness to data set shift, consistency with expert knowledge, estimates of uncertainty, and performance limits and model failures. Trust can also be build using tools that enable interpretation of complex ML models, such as the ability obtain explanations for specific predictions or insights into the features/criteria used by the model to make decisions. In this work, I will discuss strategies to build trust in ML-based systems, including specific approaches for visualizing and evaluating ML models. This work will focus on ML for signal processing applications, using earthquake monitoring as a case study.