Spatial Ensembling of Land Cover Models

Abstract:

Spatial location is a useful feature in models that use remotely sensed data. Satellite imagery, for example, can be used to estimate land cover on the surface of the earth, however both the distribution of satellite imagery and distribution of land cover will vary with location. At a macro scale, there are well defined areas over the world that contain distinct features in satellite imagery and land cover distributions such as different eco-regions. At a micro scale, spatially coherent features like ``lakes'' and ``cities'' provide a strong signal about the underlying land cover distribution. The distribution of pixels in images collected over space can differ for less obvious reasons such as the sensors used.

Models that incorporate spatial location when modeling land cover can directly account for these modes of variation and therefore produce better estimates, however incorporating spatial information into large models such as convolutional neural networks (CNNs) is non-trivial. Random forests (RFs), on the other hand, can easily use spatial location as input features, however are less effective in modeling with high dimensional data (such as large patches of multi-spectral imagery).

In this work, we propose a spatial ensemble method that allows us to incorporate spatial location information that can without necessarily changing the manner in which the component models train or operate, meaning that it is compatible with state of the art Deep Learning models. Our algorithm involves 1) using a diversified set of pre-trained base models (on existing sets of satellite images) that output feature embeddings, 2) training an artificial neural network (ANN) model that uses coordinate features to learn ensemble parameters that govern the transformation (e.g. re-weighting) of the embeddings and a secondary ANN responsible for mapping the combined embedding vector to a final class prediction. We evaluate our methodology on the task of land cover segmentation from NAIP imagery on tiles of remote sensing images that exhibit input and/or distribution shift. Given a limited number of labeled pixels in each specific tile, our Spatial Ensemble models can outperform using the same labeled data to directly fine tune existing deep learning land cover models.