A Comparison of In-Sample and Out-of-Sample Model Selection Approaches for ANN Streamflow Simulation

Abstract:

Artificial neural networks (ANNs) have been widely applied in hydrological modelling in the past three decades, and many studies have demonstrated ANNs’ capability of successfully estimating daily streamflow from meteorological data on the watershed level. One major challenge of ANN streamflow modelling is finding the optimal network structure that produces good simulation performance while ameliorating model overfitting. This study examines two types of model selection approaches for simulating streamflow time series; the out-of-sample approach using blocked cross-validation, and an in-sample approach that relies on the Akaike’s information criterion (AIC) and Bayesian information criterion (BIC). Two adjacent small watersheds in San Antonio region in south central Texas are used in this study to create the rainfall-streamflow models, in which the discharge at the watershed outlets are the model objectives. The model selection results of the two approaches are compared, and some commonly used goodness-of-fit indices on the stand-alone testing datasets are computed to evaluate the effectiveness of the two approaches. This study showed that in general the out-of-sample and in-sample approaches do not converge to the same model selection results, and the out-of-sample approach is preferable for modelling streamflow time series.