502512 Set-Membership Nonlinear Regression Approach to Parameter Estimation

Nikola D. Peric and Benoit Chachuat, Centre for Process Systems Engineering, Department of Chemical Engineering, Imperial College London, London, United Kingdom

Most commonly, regression problems in nonlinear parameter estimation are posed as optimization problems, where selected parameter values are tuned so that the model predictions match the available measurements as close as possible, for instance in the maximum-likelihood sense. Incorporation of measurement uncertainty into the regression exercise has spawned several different schools of thought. Statistical approaches can be broadly classified as Frequentist or Bayesian. The former seek to determine confidence regions around the regressed parameter values, considered as the `true' parameter values [1]. They are capable of dealing with large-scale models, but confidence analysis based on model linearization can be misleading, especially for highly nonlinear models or when the available measurements are scarce. Bayesian approaches meanwhile construct a probability distribution of the parameters based on the available observations, from which credibility regions can be inferred [2]. These approaches are appealing, but they rely on a so-called prior for the parameter estimates, and existing algorithms such as Markov-chain Monte-Carlo (MCMC) can be time consuming for problems having upwards of 5-10 parameters. An alternative to these statistical approaches is set-membership estimation, which seeks to determine the set of all possible parameter values that are consistent with bounded-error measurements [3]. Such problems have been shown to be tractable using complete-search methods with ca. 10 parameters, e.g., based on interval analysis or higher-order inclusion techniques [4,5]. Issues may arise however, as model mismatch or outlying measurements can lead to situations where the resulting parameter set is empty.

We propose a new approach to nonlinear regression, called set-membership nonlinear regression (SMNLR). Given a residual function $ \mathcal{R}({\bf p}\vert{\bf e})$ , where $ {\bf p}\in\mathbb{R}^{n}$ stands for the (unknown) model parameters and $ {\bf e}\in\mathbb{R}^{m}$ the measurement errors, the problem addressed involves characterizing the set P given by

$\displaystyle P := \left\{{\bf p}\in P_0 \:\middle\vert\: \exists {\bf e}\in E: {\bf p}\in\arg\min_{\bf p}\mathcal{R}({\bf p}\vert{\bf e}) \right\},$

(1)

where $ P_0\subset\mathbb{R}^{n}$ is a (non-empty) prior set for the model parameters; and $ E\subset\mathbb{R}^{m}$ a given measurement-error set (e.g., bounds on the measurement errors, level-sets of a joint probability distribution, etc). We study the properties of SMNLR, with a view to relating problem (1) with existing parameter estimation approaches. In the context of bounded-error measurement estimation, SMNLR presents the advantage that the set P is always non-empty, regardless of the model mismatch or measurement error. In the context of statistical inference, P may be interpreted as parameter estimability sets for a given confidence level-set E on the measurement errors [6]. Moreover, we derive conditions under which the SMNLR regions are asymptotically equivalent to the classical Wald confidence regions.

Notwithstanding, SMNLR is a particular arduous problem as it involves finding all global minimizers to all parameter estimation problems with $ {\bf e}\in E$ . We describe how this problem can be tackled using complete-search methods (branch-and-prune), and we discuss several simplifications for reducing the computational burden. Comparisons are made with the classical set-membership estimation approach as well as classical confidence analysis methods for selected parameter estimation problems.

Bibliography

[1] S.P. Asprey, S. Macchietto, 2000. Statistical tools for optimal dynamic model building. Computers & Chemical Engineering 24:1261-1267.
[2] D. Alspach, H. Sorenson, 1972. Nonlinear Bayesian estimation using Gaussian sum approximations. IEEE Transactions on Automatic Control 17(4):439-448.
[3] E. Walter, 1990. Special Issue on Parameter Identification with Error Bounds. Mathematics & Computers in Simulation 32:447-607.
[4] L. Jaulin and E. Walter, 1993. Set inversion via interval analysis for nonlinear bounded-error estimation. Automatica 29(4):1053-1064.
[5] B. Chachuat, B. Houska, R. Paulen R, N.D. Peric, J. Rajyaguru, M.E. Villanueva, 2015. Set-theoretic approaches in analysis, estimation and control of nonlinear systems. IFAC-PapersOnLine 48(8):981-995.
[6] K.A. McLean and K.B. McAuley, 2012. Mathematical modelling of chemical processes - obtaining the best model predictions and parameter estimates using identifiability and estimability procedures. The Canadian Journal of Chemical Engineering 90(2):351-366.

Extended Abstract: File Not Uploaded
See more of this Session: Estimation and Control of Uncertain Systems
See more of this Group/Topical: Computing and Systems Technology Division