Towards a community-wide effort for clean benchmarking in geostatistical inversion

Abstract:

Past decades have brought vast methodological advances in geostatistical inversion, e.g., quasi-linear inversion, (Ensemble) Kalman Filters with various localizations and hybridizations, or recent Monte-Carlo Markov Chain algorithms. However, when presenting their new methods, most studies compare these new methods against a few predecessor versions, typically using synthetic test cases that differ from publication to publication. This hinders effective and global comparison of all advances under homogeneous test conditions. Also, the community lacks accurate reference solutions for conclusive tests. For example, comparing the best estimate field to a “synthetic truth” is inappropriate in sparse-data regimes. Moreover, comparing estimation variances between methods is difficult, because none of them is known to be really accurate; some suffer from filter collapse while others suffer from excess dispersion.

In an ongoing project, we develop a set of synthetic, open-source benchmarking scenarios for inversion of hydraulic conductivity from pressure data. We also develop high-fidelity algorithms and compute accurate reference solutions by means of high-performance computing. These reference solutions will be publicly available as well. We will present the algorithm, the benchmark cases, the computed reference solutions and suggested benchmarking metrics.

We invite the community to use our benchmarks and reference solutions, and to engage in a community-wide effort towards clean and conclusive benchmarking. We aim at a special issue in an appropriate journal, where such clean comparison studies can be submitted together with a key paper that provides an overview and presents detailed information on how to use the benchmarking cases, reference solutions, and proposed benchmarking metrics.