## A convex optimization approach to generalized moment problems, Control and Modeling of Complex Systems (2003)

Venue: | Cybernetics in the 21st Century: Festschrift in Honor of Hidenori Kimura on the Occasion of his 60th |

Citations: | 16 - 10 self |

### BibTeX

@INPROCEEDINGS{Byrnes03aconvex,

author = {Christopher I. Byrnes and Anders Lindquist},

title = {A convex optimization approach to generalized moment problems, Control and Modeling of Complex Systems},

booktitle = {Cybernetics in the 21st Century: Festschrift in Honor of Hidenori Kimura on the Occasion of his 60th},

year = {2003},

pages = {3--21}

}

### Years of Citing Articles

### OpenURL

### Abstract

ABSTRACT In this paper we present a universal solution to the generalized moment problem, with a nonclassical complexity constraint. We show that this solution can be obtained by minimizing a strictly convex nonlinear functional. This optimization problem is derived in two different ways. We first derive this intrinsically, in a geometric way, by path integration of a one-form which defines the generalized moment problem. It is observed that this one-form is closed and defined on a convex set, and thus exact with, perhaps surprisingly, a strictly convex primitive function. We also derive this convex functional as the dual problem of a problem to maximize a cross entropy functional. In particular, these approaches give a constructive parameterization of all solutions to the Nevanlinna-Pick interpolation problem, with possible higher-order interpolation at certain points in the complex plane, with a degree constraint as well as all soutions to the rational covariance extension problem- two areas which have been advanced by the work of Hidenori Kimura. Illustrations of these results in system identifiaction and probablity are also mentioned. Key words. Moment problems, convex optimization, Nevanlinna-Pick interpolation, covariance extension, systems identification, Kullback-Leibler distance. 1

### Citations

1154 | Information theory and statistics
- Kullback
(Show Context)
Citation Context ... cn ⎢ c1 c2 ⎢ · · · cn+1 ⎥ ⎢ ⎣ . . . .. ⎥ . ⎦ cn cn+1 · · · c2n is positive definite. Remarkably, the function I(Φ, Ψ) = IΨ(Φ) is the cross-entropy [26], gain of information [46], directed divergency =-=[39]-=-, or the Kullback-Leibler distance between Ψ and Φ. Then the optimization problem of Theorem 3.5 is equivalent to minimizing I(Φ, Ψ) subject to the moment conditions (3.3). This gives an interesting i... |

258 | I-divergence geometry of probability distributions and minimization problems. The Annals of Probability - Csiszár - 1975 |

218 |
The classical moment problem and some related questions
- Akhiezer
- 1965
(Show Context)
Citation Context ...t.sA Convex Optimization Approach to Generalized Moment Problems 5 3 The generalized moment problem with complexity constraint There is a vast literature on the generalized moment problem (see, e.g., =-=[1, 2, 27, 38, 49]-=-), in part because so many problems and theorems in pure and applied mathematics, physics and engineering can be formulated as moment problems. The classical problem can be formulated in the following... |

217 |
Probability Theory
- Rényi
- 1970
(Show Context)
Citation Context ...kel matrix ⎡ ⎤ c0 c1 · · · cn ⎢ c1 c2 ⎢ · · · cn+1 ⎥ ⎢ ⎣ . . . .. ⎥ . ⎦ cn cn+1 · · · c2n is positive definite. Remarkably, the function I(Φ, Ψ) = IΨ(Φ) is the cross-entropy [26], gain of information =-=[46]-=-, directed divergency [39], or the Kullback-Leibler distance between Ψ and Φ. Then the optimization problem of Theorem 3.5 is equivalent to minimizing I(Φ, Ψ) subject to the moment conditions (3.3). T... |

153 |
The Problem of Moments
- Shohat, Tamarkin
- 1943
(Show Context)
Citation Context ...t.sA Convex Optimization Approach to Generalized Moment Problems 5 3 The generalized moment problem with complexity constraint There is a vast literature on the generalized moment problem (see, e.g., =-=[1, 2, 27, 38, 49]-=-), in part because so many problems and theorems in pure and applied mathematics, physics and engineering can be formulated as moment problems. The classical problem can be formulated in the following... |

84 |
System identification using Laguerre models
- Wahlberg
- 1991
(Show Context)
Citation Context ....3. A popular method in systems identification amounts to estimating the first n + 1 coefficients in an orthogonal basis function expansion G(z) = 1 2 c0f0(z) ∞� + ckfk(z) of a transfer function G(z) =-=[51, 52]-=-. The functions f0, f1, f2, . . . are orthonormal on the unit circle, i.e., 1 2π k=1 � π fj(e −π iθ ) ∗ fk(e iθ )dθ = δjk. A general class of such functions is given by � 1 − |ξk| fk(z) = 2 k−1 � 1 − ... |

50 | A convex optimization approach to the rational covariance extension problem
- Byrnes, Gusev, et al.
- 1999
(Show Context)
Citation Context ...leaves are defined by fixing the interpolation values [8]. The question of actually finding, or computing solutions to either problem can be solved in the context of nonlinear convex optimization. In =-=[7]-=-, we presented a convex optimization approach for determining an arbitrary soA Convex Optimization Approach to Generalized Moment Problems 3 lution to the rational covariance extension problem with d... |

50 |
Realization of power spectra from partial covariance sequences
- Georgiou
- 1987
(Show Context)
Citation Context ...filters of a bounded degree and with a given window of Laurent coefficients, in terms of Szegö polynomials of the first and second kind. This parameterization was independently discovered by Georgiou =-=[23]-=-. From the Kimura-Georgiou parameterization one can see that the space of shaping filters with a fixed window of covariance lags is a smooth manifold [3]. Georgiou [23] used degree theory for function... |

49 | A Generalized Entropy Criterion for Nevanlinna–Pick Interpolation with Degree Constraint
- Byrnes, Georgiou, et al.
- 2001
(Show Context)
Citation Context ...ation at distinct points in the complex plane. In this way, one obtains both an algorithm for solving the covariance extension problem and a constructive proof of Georgiou’s conjecture. Similarly, in =-=[9]-=- a generalized entropy criterion is developed for solving the rational Nevanlinna-Pick problem with degree constraints. In both problems, the primal problem of maximizing this entropy gain has a very ... |

49 |
System identification using Kautz models
- Wahlberg
- 1994
(Show Context)
Citation Context ....3. A popular method in systems identification amounts to estimating the first n + 1 coefficients in an orthogonal basis function expansion G(z) = 1 2 c0f0(z) ∞� + ckfk(z) of a transfer function G(z) =-=[51, 52]-=-. The functions f0, f1, f2, . . . are orthonormal on the unit circle, i.e., 1 2π k=1 � π fj(e −π iθ ) ∗ fk(e iθ )dθ = δjk. A general class of such functions is given by � 1 − |ξk| fk(z) = 2 k−1 � 1 − ... |

47 |
Maximum entropy for hypothesis formulation especially in multidimensional contingency tables
- Good
- 1965
(Show Context)
Citation Context ...ive if and only if the Hankel matrix ⎡ ⎤ c0 c1 · · · cn ⎢ c1 c2 ⎢ · · · cn+1 ⎥ ⎢ ⎣ . . . .. ⎥ . ⎦ cn cn+1 · · · c2n is positive definite. Remarkably, the function I(Φ, Ψ) = IΨ(Φ) is the cross-entropy =-=[26]-=-, gain of information [46], directed divergency [39], or the Kullback-Leibler distance between Ψ and Φ. Then the optimization problem of Theorem 3.5 is equivalent to minimizing I(Φ, Ψ) subject to the ... |

39 |
Some questions in the theory of moments
- Akhiezer, Krein
- 1962
(Show Context)
Citation Context ...t.sA Convex Optimization Approach to Generalized Moment Problems 5 3 The generalized moment problem with complexity constraint There is a vast literature on the generalized moment problem (see, e.g., =-=[1, 2, 27, 38, 49]-=-), in part because so many problems and theorems in pure and applied mathematics, physics and engineering can be formulated as moment problems. The classical problem can be formulated in the following... |

29 | A New Approach to Spectral Estimation: A Tunable High-Resolution Spectral Estimator - Byrnes, Georgiou, et al. - 2000 |

29 | Realization of covariance sequences - Kalman - 1981 |

27 | From finite covariance windows to modeling filters: A convex optimization approach
- Byrnes, Gusev, et al.
(Show Context)
Citation Context ...determined by the unique minimum of a strictly convex functional. In fact, the following theorem is a version of a result proved in [16], generalizing similar results in analytic interpolation theory =-=[7, 9, 10, 11, 15]-=-.sA Convex Optimization Approach to Generalized Moment Problems 9 Theorem 3.4. Let P be spanned by C 2 functions α0, α1, . . . , αn, whose nonzero real and imaginary parts form a linearly independent ... |

26 |
Dooren, Speech modelling and the trigonometric moment problem
- Delsarte, Kamp, et al.
- 1982
(Show Context)
Citation Context ...only if Φ(θ) := Re{f(e iθ )} is positive for all θ ∈ [−π, π]. Note that In particular, Φ(θ) = c0 + 2 ∞� ck cos kθ. k=0 � π 1 Φ(θ) cos kθ dθ = ck 2π −π (2.1) for k = 0, 1, 2, . . . . As pointed out in =-=[20]-=-, the rational covariance extension problem is a trigonometric moment problem: Given a positive sequence c0, c1, · · · , cn, find all positive Φ such that (2.1) holds for k = 0, 1, . . . , n. However,... |

26 | The interpolation problem with a degree constraint
- Georgiou
(Show Context)
Citation Context ...e. This leads naturally to the NevanlinnaPick problem with degree constraint. A complete parameterization of the class of such interpolants was conjectured by Georgiou in [24] and recently settled in =-=[25]-=- for interpolation at distinct points in the complex plane. This can again be enhanced using the geometry of foliations on a space of rational, positive real functions, with the foliation defined by c... |

25 |
A complete parametrization of all positive rational extensions of a covariance sequence
- Byrnes, Lindquist, et al.
- 1995
(Show Context)
Citation Context ...stence result and conjectured a refinement of his result that would give a complete parameterization of all solutions, of no more than a given degree, to the rational covariance extension problem. In =-=[6]-=- Georgiou’s conjecture was proved by first noting that the Kimura-Georgiou parameterization defined the leaves of a foliation of a space of positive real rational functions having a bounded degree. A ... |

20 | A topological approach to Nevanlinna–Pick interpolation
- Georgiou
- 1987
(Show Context)
Citation Context ...has an a priori bounded degree. This leads naturally to the NevanlinnaPick problem with degree constraint. A complete parameterization of the class of such interpolants was conjectured by Georgiou in =-=[24]-=- and recently settled in [25] for interpolation at distinct points in the complex plane. This can again be enhanced using the geometry of foliations on a space of rational, positive real functions, wi... |

19 | Cepstral coefficients, covariance lags and pole-zero models for finite data strings
- Byrnes, Enqvist, et al.
- 2001
(Show Context)
Citation Context ... of mathematical programming. We have thus provided a unified framework that generalizes previous work on interpolation of the Carathéodory and of the Nevanlinna-Pick type [7, 9, 10, 15]. We refer to =-=[9, 13, 14]-=- for applications to signal processing and to [42, 43, 44, 45] for applications to robust control. Algorithms using homotopy continuation methods, based on our convex optimization approach, have been ... |

19 |
Positive partial realization of covariance sequences
- Kimura
- 1987
(Show Context)
Citation Context ...ch Council, the Göran Gustafsson Foundation, and Southwestern Bell. This is page 1 Printer: Opaque thiss2 C. I. Byrnes and A. Lindquist noise into a process with a given window of covariance lags. In =-=[31]-=-, Kimura was able to give a neat parameterization of rational filters of a bounded degree and with a given window of Laurent coefficients, in terms of Szegö polynomials of the first and second kind. T... |

19 |
Robust stabilizability for a class of transfer functions
- Kimura
- 1984
(Show Context)
Citation Context ... Georgiou’s conjecture was obtained as a corollary of a theorem about the geometry of these two foliations, including the fact that leaves of one intersect leaves of the other transversely. The paper =-=[32]-=- on robust stabilization of plants with a fixed number of unstable poles and Nyquist plot in a neighborhood of the Nyquist plot of a nominal plant was one of the key contributions which ushered in the... |

17 | On the duality between filtering and Nevanlinna–Pick interpolation
- Byrnes, Lindquist
(Show Context)
Citation Context ... foliations on a space of rational, positive real functions, with the foliation defined by covariance windows being replaced by a foliation whose leaves are defined by fixing the interpolation values =-=[8]-=-. The question of actually finding, or computing solutions to either problem can be solved in the context of nonlinear convex optimization. In [7], we presented a convex optimization approach for dete... |

15 | On the nonlinear dynamics of fast filtering algorithms
- Byrnes, Lindquist, et al.
- 1994
(Show Context)
Citation Context ...m of [40, 41], viewed as a dynamical system on the same space of positive real functions, also defines a foliation of this space, with its leaves being the stable manifolds through various equilibria =-=[4, 5]-=-. The proof of Georgiou’s conjecture was obtained as a corollary of a theorem about the geometry of these two foliations, including the fact that leaves of one intersect leaves of the other transverse... |

14 | A new algorithm for optimal filtering of discrete-time stationary processes
- Lindquist
- 1974
(Show Context)
Citation Context ...ou parameterization defined the leaves of a foliation of a space of positive real rational functions having a bounded degree. A second observation used in [6] was that the fast filtering algorithm of =-=[40, 41]-=-, viewed as a dynamical system on the same space of positive real functions, also defines a foliation of this space, with its leaves being the stable manifolds through various equilibria [4, 5]. The p... |

13 | Identifiability and wellposedness of shaping-filter parameterizations: A global analysis approach
- Byrnes, Enqvist, et al.
(Show Context)
Citation Context ... of mathematical programming. We have thus provided a unified framework that generalizes previous work on interpolation of the Carathéodory and of the Nevanlinna-Pick type [7, 9, 10, 15]. We refer to =-=[9, 13, 14]-=- for applications to signal processing and to [42, 43, 44, 45] for applications to robust control. Algorithms using homotopy continuation methods, based on our convex optimization approach, have been ... |

13 | Interior Point Solutions of Variational Problems and Global Inverse Function Theorems
- Byrnes, Lindquist
- 2001
(Show Context)
Citation Context ...ex optimization problem. We settle this question in a geometric way by path integration of a one-form which defines the generalized moment problem. This exposition follows the original calculation in =-=[16]-=- where it is observed that this one-form is closed and defined on a convex set, and thus exact. Since its integral is therefore path-independent, it is intrinsic and is, perhaps surprisingly, a strict... |

13 |
Conjugation, interpolation and model-matching in H
- Kimura
- 1989
(Show Context)
Citation Context ...ssary and sufficient conditions for the solution of this problem in terms of the classical Nevanlinna-Pick interpolation problem, a methodology he would continue to use and develop for robust control =-=[33, 34, 35, 36, 37]-=-. We also refer the reader to the books [21, 53] and the references therein. Nevanlinna-Pick interpolation for bounded-real rational functions is now one of the tools commonly used in robust control. ... |

12 |
On the geometry of the Kimura-Georgiou parameterization of modelling filter
- Bymes, Lindquist
- 1989
(Show Context)
Citation Context ...tion was independently discovered by Georgiou [23]. From the Kimura-Georgiou parameterization one can see that the space of shaping filters with a fixed window of covariance lags is a smooth manifold =-=[3]-=-. Georgiou [23] used degree theory for functions on manifolds to give a very basic existence result and conjectured a refinement of his result that would give a complete parameterization of all soluti... |

12 |
Forms and their applications, (Univ
- Grenader, Szegö, et al.
- 1958
(Show Context)
Citation Context |

12 |
Feedback stabilization of linear dynamical plants with uncertainty in the gain factor
- Tannenbaum
- 1980
(Show Context)
Citation Context ...ree constraint on the interpolant f has been introduced, a restriction motivated by applications [9, 10]. In fact, many problems in systems and control can be reduced to Nevanlinna-Pick interpolation =-=[19, 21, 28, 32, 53, 50]-=-, and, as the interpolant generally can be interpreted as a transfer function, the bound on the degree is a natural complexity constraint. To reformulate this interpolation problem as a generalized mo... |

11 |
A robust solver using a continuation method for Nevanlinna-Pick interpolation with degree constraint
- Nagamune
- 2003
(Show Context)
Citation Context ...fied framework that generalizes previous work on interpolation of the Carathéodory and of the Nevanlinna-Pick type [7, 9, 10, 15]. We refer to [9, 13, 14] for applications to signal processing and to =-=[42, 43, 44, 45]-=- for applications to robust control. Algorithms using homotopy continuation methods, based on our convex optimization approach, have been developed for Carathéodory extension in [22] and for Nevanlinn... |

7 |
Predictability and unpredictability in Kalman filtering
- Byrnes, Lindquist, et al.
- 1991
(Show Context)
Citation Context ...m of [40, 41], viewed as a dynamical system on the same space of positive real functions, also defines a foliation of this space, with its leaves being the stable manifolds through various equilibria =-=[4, 5]-=-. The proof of Georgiou’s conjecture was obtained as a corollary of a theorem about the geometry of these two foliations, including the fact that leaves of one intersect leaves of the other transverse... |

7 |
Spectral estimation by Geometric, Topological and Optimization
- Enqvist
- 2001
(Show Context)
Citation Context ...nd to [42, 43, 44, 45] for applications to robust control. Algorithms using homotopy continuation methods, based on our convex optimization approach, have been developed for Carathéodory extension in =-=[22]-=- and for Nevanlinna-Pick interpolation in [43, 45]. The results presented in this paper have interesting interpretations also in probability and statistics, where moment problems are prevalent. Indeed... |

7 |
Non-Euclidean analysis and electronics
- Helton
- 1982
(Show Context)
Citation Context ...ree constraint on the interpolant f has been introduced, a restriction motivated by applications [9, 10]. In fact, many problems in systems and control can be reduced to Nevanlinna-Pick interpolation =-=[19, 21, 28, 32, 53, 50]-=-, and, as the interpolant generally can be interpreted as a transfer function, the bound on the degree is a natural complexity constraint. To reformulate this interpolation problem as a generalized mo... |

7 |
Some reduced-order non-Riccati equations for linear least-squares estimation: the stationary, single-output case
- Lindquist
- 1976
(Show Context)
Citation Context ...ou parameterization defined the leaves of a foliation of a space of positive real rational functions having a bounded degree. A second observation used in [6] was that the fast filtering algorithm of =-=[40, 41]-=-, viewed as a dynamical system on the same space of positive real functions, also defines a foliation of this space, with its leaves being the stable manifolds through various equilibria [4, 5]. The p... |

6 |
On the role of the Nevanlinna-Pick problem
- Delsarte, Kamp
- 1981
(Show Context)
Citation Context ...ree constraint on the interpolant f has been introduced, a restriction motivated by applications [9, 10]. In fact, many problems in systems and control can be reduced to Nevanlinna-Pick interpolation =-=[19, 21, 28, 32, 53, 50]-=-, and, as the interpolant generally can be interpreted as a transfer function, the bound on the degree is a natural complexity constraint. To reformulate this interpolation problem as a generalized mo... |

5 |
Closed-loop shaping based on Nevanlinna-Pick interpolation with degree constraint
- Nagamune
(Show Context)
Citation Context ...fied framework that generalizes previous work on interpolation of the Carathéodory and of the Nevanlinna-Pick type [7, 9, 10, 15]. We refer to [9, 13, 14] for applications to signal processing and to =-=[42, 43, 44, 45]-=- for applications to robust control. Algorithms using homotopy continuation methods, based on our convex optimization approach, have been developed for Carathéodory extension in [22] and for Nevanlinn... |

4 |
interpolation approach to -optimization and robust stabilization
- Kimura
- 1987
(Show Context)
Citation Context ...ssary and sufficient conditions for the solution of this problem in terms of the classical Nevanlinna-Pick interpolation problem, a methodology he would continue to use and develop for robust control =-=[33, 34, 35, 36, 37]-=-. We also refer the reader to the books [21, 53] and the references therein. Nevanlinna-Pick interpolation for bounded-real rational functions is now one of the tools commonly used in robust control. ... |

4 |
Sensitivity shaping in feedback control and analytic interpolation theory
- Nagamune, Lindquist
- 2001
(Show Context)
Citation Context ...fied framework that generalizes previous work on interpolation of the Carathéodory and of the Nevanlinna-Pick type [7, 9, 10, 15]. We refer to [9, 13, 14] for applications to signal processing and to =-=[42, 43, 44, 45]-=- for applications to robust control. Algorithms using homotopy continuation methods, based on our convex optimization approach, have been developed for Carathéodory extension in [22] and for Nevanlinn... |

4 |
Robust control with complexity constraint: A NevanlinnaPick interpolation approach
- Nagamune
- 2002
(Show Context)
Citation Context |

3 |
Essentials of Robust Control, Prentice-Hall
- Zhou, Doyle
- 1997
(Show Context)
Citation Context ...blem in terms of the classical Nevanlinna-Pick interpolation problem, a methodology he would continue to use and develop for robust control [33, 34, 35, 36, 37]. We also refer the reader to the books =-=[21, 53]-=- and the references therein. Nevanlinna-Pick interpolation for bounded-real rational functions is now one of the tools commonly used in robust control. Indeed, it is widely known that the sensitivity ... |

2 |
Generalized interpolation in H∞: Solutions of bounded complexity, in preparation
- Byrnes, Georgiou, et al.
(Show Context)
Citation Context ...determined by the unique minimum of a strictly convex functional. In fact, the following theorem is a version of a result proved in [16], generalizing similar results in analytic interpolation theory =-=[7, 9, 10, 11, 15]-=-.sA Convex Optimization Approach to Generalized Moment Problems 9 Theorem 3.4. Let P be spanned by C 2 functions α0, α1, . . . , αn, whose nonzero real and imaginary parts form a linearly independent ... |

2 |
Minimal partial realization from orthonormal basis function expansions
- Hoog, Heuberger, et al.
- 2001
(Show Context)
Citation Context ...� π fj(e −π iθ ) ∗ fk(e iθ )dθ = δjk. A general class of such functions is given by � 1 − |ξk| fk(z) = 2 k−1 � 1 − ξ z − ξk ∗ j z , z − ξj where ξ0, ξ1, ξ2, . . . are poles to be selected by the user =-=[29]-=-. Given the estimated coefficients c0, c1, · · · , cn, the usual problem considered in the literature [47] is to find a rational function G of smallest degree which match these coefficients. Here, how... |

2 |
An den Hof, Extended Ho-Kalman algorithm for systems represented in generalized orthonormal bases, Automatica 36
- Szabó, Heuberger, et al.
- 2000
(Show Context)
Citation Context ...−1 � 1 − ξ z − ξk ∗ j z , z − ξj where ξ0, ξ1, ξ2, . . . are poles to be selected by the user [29]. Given the estimated coefficients c0, c1, · · · , cn, the usual problem considered in the literature =-=[47]-=- is to find a rational function G of smallest degree which match these coefficients. Here, however, we consider the corresponding problem where G is a Carathéodory function of degree at most n, a prob... |

1 |
forthcoming paper
- Byrnes, Georgiou, et al.
(Show Context)
Citation Context ... between Ψ and Φ. Then the optimization problem of Theorem 3.5 is equivalent to minimizing I(Φ, Ψ) subject to the moment conditions (3.3). This gives an interesting interpretation, further pursued in =-=[12]-=-, to the present problem: Given an a priori probability density Ψ, we want to find another probability density Φ that has prescribed moments up to order 2n and that minimizes the Kullback-Leibler dist... |

1 |
Information projections revisited, prepint
- Csiszár, Matúˇs
- 2001
(Show Context)
Citation Context ...minimizes the Kullback-Leibler distance to Ψ. Similar optimization problems have been considered in the statistical literature, where, however, minimization is generally with respect to Ψ; see, e.g., =-=[17, 18]-=-. 4 A geometric derivation of the convex optimization problem Following [16], we prove Theorem 3.4 by first constructing the dual functional from the moment equations (3.14). To this end, for any Ψ ∈ ... |

1 |
State space approach to the classical interpolation problem and its applications, Three decades of mathematical system theory
- Kimura
- 1989
(Show Context)
Citation Context ...ssary and sufficient conditions for the solution of this problem in terms of the classical Nevanlinna-Pick interpolation problem, a methodology he would continue to use and develop for robust control =-=[33, 34, 35, 36, 37]-=-. We also refer the reader to the books [21, 53] and the references therein. Nevanlinna-Pick interpolation for bounded-real rational functions is now one of the tools commonly used in robust control. ... |

1 |
On directional interpolation in H ∞ . Linear circuits, systems and signal processing: theory and application
- Kimura, Iwase
- 1987
(Show Context)
Citation Context |

1 |
Directional interpolation in the state space
- Kimura
- 1988
(Show Context)
Citation Context |