Results 1  10
of
17
KullbackLeibler approximation of spectral density functions
 IEEE Trans. Inform. Theory
, 2003
"... Abstract—We introduce a Kullback–Leiblertype distance between spectral density functions of stationary stochastic processes and solve the problem of optimal approximation of a given spectral density 9 by one that is consistent with prescribed secondorder statistics. In general, such statistics are ..."
Abstract

Cited by 29 (15 self)
 Add to MetaCart
(Show Context)
Abstract—We introduce a Kullback–Leiblertype distance between spectral density functions of stationary stochastic processes and solve the problem of optimal approximation of a given spectral density 9 by one that is consistent with prescribed secondorder statistics. In general, such statistics are expressed as the state covariance of a linear filter driven by a stochastic process whose spectral density is sought. In this context, we show i) that there is a unique spectral density 8 which minimizes this Kullback–Leibler distance, ii) that this optimal approximate is of the form 9 where the “correction term ” is a rational spectral density function, and iii) that the coefficients of can be obtained numerically by solving a suitable convex optimization problem. In the special case where 9=1, the convex functional becomes quadratic and the solution is then specified by linear equations. Index Terms—Approximation of power spectra, crossentropy minimization, Kullback–Leibler distance, mutual information, optimization, spectral estimation. I.
Generalized interpolation in H∞ with a complexity constraint
 TRANS. AMER. MATH. SOC
, 2006
"... In a seminal paper, Sarason generalized some classical interpolation problems for H∞ functions on the unit disc to problems concerning lifting onto H² of an operator T that is defined on K = H² ⊖φH² (φ is an inner function) and commutes with the (compressed) shift S. In particular, he showed that in ..."
Abstract

Cited by 25 (12 self)
 Add to MetaCart
(Show Context)
In a seminal paper, Sarason generalized some classical interpolation problems for H∞ functions on the unit disc to problems concerning lifting onto H² of an operator T that is defined on K = H² ⊖φH² (φ is an inner function) and commutes with the (compressed) shift S. In particular, he showed that interpolants (i.e., f ∈ H ∞ such that f(S)=T) having norm equal to �T � exist, and that in certain cases such an f is unique and can be expressed as a fraction f = b/a with a, b ∈ K. In this paper, we study interpolants that are such fractions of K functions and are bounded in norm by 1 (assuming that �T � < 1, in which case they always exist). We parameterize the collection of all such pairs (a, b) ∈ K × K and show that each interpolant of this type can be determined as the unique minimum of a convex functional. Our motivation stems from the relevance of classical interpolation to circuit theory, systems theory, and signal processing, where φ is typically a finite Blaschke product, and where the quotient representation is a physically meaningful complexity constraint.
Matrixvalued NevanlinnaPick interpolation with complexity constraint: An optimization approach
 IEEE Trans. Automatic Control
, 2003
"... Abstract. Over the last several years a new theory of NevanlinnaPick interpolation with complexity constraint has been developed for scalar interpolants. In this paper we generalize this theory to the matrixvalued case, also allowing for multiple interpolation points. We parameterize a class of in ..."
Abstract

Cited by 18 (5 self)
 Add to MetaCart
(Show Context)
Abstract. Over the last several years a new theory of NevanlinnaPick interpolation with complexity constraint has been developed for scalar interpolants. In this paper we generalize this theory to the matrixvalued case, also allowing for multiple interpolation points. We parameterize a class of interpolants consisting of “most interpolants ” of no higher degree than the central solution in terms of spectral zeros. This is a complete parameterization, and for each choice of interpolant we provide a convex optimization problem for determining it. This is derived in the context of duality theory of mathematical programming. To solve the convex optimization problem, we employ a homotopy continuation technique previously developed for the scalar case. These results can be applied to many classes of engineering problems, and, to illustrate this, we provide some examples. In particular, we apply our method to a benchmark problem in multivariate robust control. By constructing a controller satisfying all design specifications but having only half the McMillan degree of conventional H ∞ controllers, we demonstrate the efficiency of our method.
Interior point solutions of variational problems and global inverse function theorems
 INT. J. ROBUST NONLINEAR CONTROL (IN PRESS)
, 2006
"... ..."
Relative Entropy and the multivariable multidimensional Moment Problem
 IEEE Trans. on Information Theory
"... Entropylike functionals on operator algebras have been studied since the pioneering work of von Neumann, Umegaki, Lindblad, and Lieb. The most wellknown are the von Neumann entropy I(ρ): = −trace(ρ log ρ) and a generalization of the KullbackLeibler distance S(ρσ): = trace(ρ log ρ − ρ log σ), re ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
(Show Context)
Entropylike functionals on operator algebras have been studied since the pioneering work of von Neumann, Umegaki, Lindblad, and Lieb. The most wellknown are the von Neumann entropy I(ρ): = −trace(ρ log ρ) and a generalization of the KullbackLeibler distance S(ρσ): = trace(ρ log ρ − ρ log σ), refered to as quantum relative entropy and used to quantify distance between states of a quantum system. The purpose of this paper is to explore I and S as regularizing functionals in seeking solutions to multivariable and multidimensional moment problems. It will be shown that extrema can be effectively constructed via a suitable homotopy. The homotopy approach leads naturally to a further generalization and a description of all the solutions to such moment problems. This is accomplished by a renormalization of a Riemannian metric induced by entropy functionals. As an application we discuss the inverse problem of describing power spectra which are consistent with secondorder statistics, which has been the main motivation behind the present work.
IMPORTANT MOMENTS IN SYSTEMS AND CONTROL
, 2008
"... The moment problem matured from its various special forms in the late 19th and early 20th centuries to a general class of problems that continues to exert profound influence on the development of analysis and its applications to a wide variety of fields. In particular, the theory of systems and cont ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
(Show Context)
The moment problem matured from its various special forms in the late 19th and early 20th centuries to a general class of problems that continues to exert profound influence on the development of analysis and its applications to a wide variety of fields. In particular, the theory of systems and control is no exception, where the applications have historically been to circuit theory, optimal control, robust control, signal processing, spectral estimation, stochastic realization theory, and the use of the moments of a probability density. Many of these applications are also still works in progress. In this paper, we consider the generalized moment problem, expressed in terms of a basis of a finitedimensional subspace P of the Banach space C[a, b] and a “positive ” sequence c, but with a new wrinkle inspired by the applications to systems and control. We seek to parameterize solutions which are positive “rational” measures in a suitably generalized sense. Our parameterization is given in terms of smooth objects. In particular, the desired solution space arises naturally as a manifold which can be shown to be diffeomorphic to a Euclidean space and which is the domain of some canonically defined functions. The analysis of these functions, and related maps, yields interesting corollaries for the moment problem and its applications, which we compare to those in the recent literature and which play a crucial role in part of our proof.
Predictionerror approximation by convex optimization
, 2007
"... This paper is dedicated to Giorgio Picci on the occasion of his 65th birthday. I have come to appreciate Giorgio not only as a great friend but also as a great scholar. When we first met at Brown University in 1973, he introduced me to his seminal paper [29] on splitting subspaces, which became the ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
This paper is dedicated to Giorgio Picci on the occasion of his 65th birthday. I have come to appreciate Giorgio not only as a great friend but also as a great scholar. When we first met at Brown University in 1973, he introduced me to his seminal paper [29] on splitting subspaces, which became the impetus for our joint work on the geometric theory of linear stochastic systems [23–26]. This led to a lifelong friendship and a book project that never seemed to converge, but now is close to being finished [27]. I have learned a lot from Giorgio. The present paper grew out of a discussion in our book project, when Giorgio taught me about the connections between predictionerror identification and the KullbackLeibler criterion. These concepts led directly into the recent theory of analytic interpolation with complexity constraint, with which I have been deeply involved in recent times. I shall try to explain these connections in the following paper.
The moment problem for rational measures: convexity in the spirit of Krein
 in Modern Analysis and Application: To the centenary of Mark Krein, Vol. I: Operator Theory and Related Topics, Birkhäuser
, 2009
"... In memory of Mark Grigoryevich Krein on the occasion of the 100th anniversary of his birth Abstract. The moment problem as formulated by Krein and Nudel’man is a beautiful generalization of several important classical moment problems, including the power moment problem, the trigonometric moment prob ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
In memory of Mark Grigoryevich Krein on the occasion of the 100th anniversary of his birth Abstract. The moment problem as formulated by Krein and Nudel’man is a beautiful generalization of several important classical moment problems, including the power moment problem, the trigonometric moment problem and the moment problem arising in NevanlinnaPick interpolation. Motivated by classical applications and examples, in both finite and infinite dimensions, we recently formulated a new version of this problem that we call the moment problem for positive rational measures. The formulation reflects the importance of rational functions in signals, systems and control. While this version of the problem is decidedly nonlinear, the basic tools still rely on convexity. In particular, we present a solution to this problem in terms of a nonlinear convex optimization problem that generalizes the maximum entropy approach used in several classical special cases.
Important Moments in Systems, Control and Optimization
"... Abstract — The moment problem matured from its various special forms in the late 19th and early 20th Centuries to a general class of problems that continues to exert profound influence on the development of analysis and its applications to a wide variety of fields. In particular, the theory of syste ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract — The moment problem matured from its various special forms in the late 19th and early 20th Centuries to a general class of problems that continues to exert profound influence on the development of analysis and its applications to a wide variety of fields. In particular, the theory of systems and control is no exception, where the applications have historically been to circuit theory, optimal control, robust control, signal processing, spectral estimation, stochastic realization theory and the use of the moments of a probability density. Many of these applications are also still works in progress. In this paper, we consider the generalized moment problem, expressed in terms of a basis of a finitedimensional subspace P of the Banach space C[a, b] and a “positive ” sequences c, but with a new wrinkle inspired by the applications to systems and control. We seek to parameterize solutions which are positive “rational” measures, in a suitably generalized sense. Our parameterization is given in terms of smooth objects. In particular, the desired solution space arises naturally as a manifold which can be shown to be diffeomorphic to a Euclidean space and which is the domain of some canonically defined functions. Moreover, on these spaces one can derive natural convex optimization criteria which characterize solutions to this new class of moment problems. I.
The Uncertain Generalized Moment Problem With Complexity Constraint
 INTEGRAL EQUATIONS AND OPERATOR THEORY 56 (2006) 163–180. MOMENTS IN SYSTEMS AND CONTROL 17
, 2003
"... This paper is dedicated to Arthur Krener  a great researcher, a great teacher and a great friend  on the occasion of his 60th birthday. In this work we study the generalized moment problem with complexity constraints in the case where the actual values of the moments are uncertain. For example, ..."
Abstract
 Add to MetaCart
This paper is dedicated to Arthur Krener  a great researcher, a great teacher and a great friend  on the occasion of his 60th birthday. In this work we study the generalized moment problem with complexity constraints in the case where the actual values of the moments are uncertain. For example, in spectral estimation the moments correspond to estimates of covariance lags computed from a finite observation record, which inevitably leads to statistical errors, a problem studied earlier by Shankwitz and Georgiou. Our approach is a combination of methods drawn from optimization and the di#erentiable approach to geometry and topology. In particular, we give an intrinsic geometric derivation of the Legendre transform and use it to describe convexity properties of the solution to the generalized moment problems as the moments vary over an arbitrary compact convex set of possible values. This is also interpreted in terms of minimizing the KullbackLeibler divergence for the generalized moment problem