This paper examines the problem of estimating linear time-invariant state-space system models. In particular it addresses the parametrization and numerical robustness concerns that arise in the multivariable case. These difficulties are well recognised in the literature, resulting (for example) in extensive study of subspace based techniques, as well as recent interest in “data driven” local co-ordinate approaches to gradient search solutions. The paper here proposes a different strategy that employs the Expectation Maximisation (EM) technique. The consequence is an algorithm that is iterative, and locally convergent to stationary points of the (Gaussian) Likelihood function. Furthermore, theoretical and empirical evidence presented here establishes additional attractive properties such as numerical robustness, avoidance of difficult parametrization choices, the ability to estimate unstable systems, the ability to naturally and easily estimate non-zero initial conditions, and moderate computational cost. Moreover, since the methods here are Maximum-Likelihood based, they have associated known and asymptotically optimal statistical properties. 1

Abstract—This paper addresses the problem of estimating the parameters in a multivariable bilinear model on the basis of observed input-output data. The main contribution is to develop, analyze, and empirically study new techniques for computing a maximum-likelihood based solution. In particular, the emphasis here is on developing practical methods that are illustrated to be numerically reliable, robust to choice of initialization point, and numerically efficient in terms of how computation and memory requirements scale relative to problem size. This results in new methods that can be reliably deployed on systems of nontrivial state, input and output dimension. Underlying these developments is a new approach (in this context) of employing the expectation-maximization method as a means for robust and gradient free computation of the maximum-likelihood solution. Index Terms—Bilinear systems, maximum likelihood (ML), parameter estimation, system identification. I.

In this paper, we describe an efficient decomposition algorithm for parameter estimation of linear dynamical systems with the state-space formulation which contain a “drive ” term as a free, unknown system parameter. The dynamical system can be viewed as a natural extension from the discrete-state hidden Markov model to its continuous-state counterpart. The focus of this paper is on unified techniques for efficient estimation of the p~eters of such a model. The Expectation-Maximization (EM) algorithm is developed, in conjunction with the conventional Kalman smoothing estimators, for estimating the system parameters by maximum likelihood. The algorithm developed is applicable to either stationary or non-stationary version of the dynamic system. In particular, a decomposition technique is described in detail and is shown to reduce effectively the compu~tional load in parameter estimation for hip-dimensioM1 systems. Simulation results are presented which demonstrate the accuracy of the proposed parameter estimation technique. @ 1997 Elsevier Science B.V.

Abstract — The expectation maximisation (EM) algorithm has proven to be effective for a range of identification problems. Unfortunately, the way in which the EM algorithm has previously been applied has proven unsuitable for the commonly employed innovations form model structure. This paper addresses this problem, and presents a previously unexamined method of EM algorithm employment. The results are profiled, which indicate that a hybrid EM/gradient-search technique may in some cases outperform either a pure EM or a pure gradient-based search approach. I.

Abstract: The field of control-oriented system identification is mature. Nevertheless, it is still very active. This is because there are many important unsolved challenges. Of these, this paper considers a selection. This involves considering the estimation of general nonlinear model structures, together with accurate error bounds, using methods that scale well to models of high dimension. A particular strength of the system identification field is that it has always actively sought to understand, embrace and develop ideas from other fields, such as statistics, mathematics and econometrics. This paper proposes a continuation of this successful strategy by proposing and profiling the adoption of new ideas originating in statistics, signal processing and statistical mechanics.