Error Scaling with Confusion Matrices for Global Optical Remote Sensing of Building and Road Detection

Abstract:

Remote sensing of human-built systems such as roads and buildings can provide important data for quantitatively studying patterns of anthropogenic activity. High-resolution earth imagery is becoming increasingly available that allows for building-scale observation and classification. As the use of high-resolution imagery expands for a number of applications, it is critical to develop scalable error quantification methods that can be used for characterizing the quality of image classification and segmentation. In this work, recently produced PlanetScope imagery with global segmentation of buildings and roads was analyzed. Data from systematically selected and diverse places such as dense urban cities, suburban neighborhoods, and rural areas, was sampled and confusion matrices were created and normalized to describe classification performance. A first assessment shows accuracy and sensitivity in the 0.8-0.9 range and a range of 0.58 to 0.77 in the Kappa coefficients. Preliminary results also show that classification error varies not only between regions–depending on factors such as terrain contrast, etc.– but within different clusters of the regions themselves. One of the issues on a pixel-by-pixel basis is the false positive rate (FPR) for tightly clustered buildings, which are, in reality, separate structures. Ground resolution is an important factor as well as color contrast in the RGB bands. As a result, a universal underestimation of building number count and an overestimation in total built area was determined and quantified. The results indicate that, through a systematic selection of data, this error classification approach can be scaled to characterize and bound errors that are potentially globally applicable.