Results 1  10
of
321
Convex Analysis
, 1970
"... In this book we aim to present, in a unified framework, a broad spectrum of mathematical theory that has grown in connection with the study of problems of optimization, equilibrium, control, and stability of linear and nonlinear systems. The title Variational Analysis reflects this breadth. For a lo ..."
Abstract

Cited by 3279 (61 self)
 Add to MetaCart
In this book we aim to present, in a unified framework, a broad spectrum of mathematical theory that has grown in connection with the study of problems of optimization, equilibrium, control, and stability of linear and nonlinear systems. The title Variational Analysis reflects this breadth. For a long time, ‘variational ’ problems have been identified mostly with the ‘calculus of variations’. In that venerable subject, built around the minimization of integral functionals, constraints were relatively simple and much of the focus was on infinitedimensional function spaces. A major theme was the exploration of variations around a point, within the bounds imposed by the constraints, in order to help characterize solutions and portray them in terms of ‘variational principles’. Notions of perturbation, approximation and even generalized differentiability were extensively investigated. Variational theory progressed also to the study of socalled stationary points, critical points, and other indications of singularity that a point might have relative to its neighbors, especially in association with existence theorems for differential equations.
An equivalence between sparse approximation and Support Vector Machines
 A.I. Memo 1606, MIT Arti cial Intelligence Laboratory
, 1997
"... This publication can be retrieved by anonymous ftp to publications.ai.mit.edu. The pathname for this publication is: aipublications/15001999/AIM1606.ps.Z This paper shows a relationship between two di erent approximation techniques: the Support Vector Machines (SVM), proposed by V.Vapnik (1995), ..."
Abstract

Cited by 205 (7 self)
 Add to MetaCart
This publication can be retrieved by anonymous ftp to publications.ai.mit.edu. The pathname for this publication is: aipublications/15001999/AIM1606.ps.Z This paper shows a relationship between two di erent approximation techniques: the Support Vector Machines (SVM), proposed by V.Vapnik (1995), and a sparse approximation scheme that resembles the Basis Pursuit DeNoising algorithm (Chen, 1995 � Chen, Donoho and Saunders, 1995). SVM is a technique which can be derived from the Structural Risk Minimization Principle (Vapnik, 1982) and can be used to estimate the parameters of several di erent approximation schemes, including Radial Basis Functions, algebraic/trigonometric polynomials, Bsplines, and some forms of Multilayer Perceptrons. Basis Pursuit DeNoising is a sparse approximation technique, in which a function is reconstructed by using a small number of basis functions chosen from a large set (the dictionary). We show that, if the data are noiseless, the modi ed version of Basis Pursuit DeNoising proposed in this paper is equivalent to SVM in the following sense: if applied to the same data set the two techniques give the same solution, which is obtained by solving the same quadratic programming problem. In the appendix we also present a derivation of the SVM technique in the framework of regularization theory, rather than statistical learning theory, establishing a connection between SVM, sparse approximation and regularization theory.
HOMOGENIZATION AND TWOSCALE CONVERGENCE
, 1992
"... Following an idea of G. Nguetseng, the author defines a notion of "twoscale" convergence, which is aimed at a better description of sequences of oscillating functions. Bounded sequences in L2(f) are proven to be relatively compact with respect to this new type of convergence. A correctortype theor ..."
Abstract

Cited by 176 (11 self)
 Add to MetaCart
Following an idea of G. Nguetseng, the author defines a notion of "twoscale" convergence, which is aimed at a better description of sequences of oscillating functions. Bounded sequences in L2(f) are proven to be relatively compact with respect to this new type of convergence. A correctortype theorem (i.e., which permits, in some cases, replacing a sequence by its "twoscale " limit, up to a strongly convergent remainder in L2(12)) is also established. These results are especially useful for the homogenization of partial differential equations with periodically oscillating coefficients. In particular, a new method for proving the convergence of homogenization processes is proposed, which is an alternative to the socalled energy method of Tartar. The power and simplicity of the twoscale convergence method is demonstrated on several examples, including the homogenization of both linear and nonlinear secondorder elliptic equations.
Networks and the Best Approximation Property
 Biological Cybernetics
, 1989
"... Networks can be considered as approximation schemes. Multilayer networks of the backpropagation type can approximate arbitrarily well continuous functions (Cybenko, 1989# Funahashi, 1989# Stinchcombe and White, 1989). Weprovethatnetworks derived from regularization theory and including Radial Bas ..."
Abstract

Cited by 95 (7 self)
 Add to MetaCart
Networks can be considered as approximation schemes. Multilayer networks of the backpropagation type can approximate arbitrarily well continuous functions (Cybenko, 1989# Funahashi, 1989# Stinchcombe and White, 1989). Weprovethatnetworks derived from regularization theory and including Radial Basis Functions (Poggio and Girosi, 1989), have a similar property.From the point of view of approximation theory, however, the property of approximating continuous functions arbitrarily well is not sufficientforcharacterizing good approximation schemes. More critical is the property of best approximation. The main result of this paper is that multilayer networks, of the type used in backpropagation, are not best approximation. For regularization networks (in particular Radial Basis Function networks) we prove existence and uniqueness of best approximation.
On the Approximation of Complicated Dynamical Behavior
 SIAM Journal on Numerical Analysis
, 1998
"... We present efficient techniques for the numerical approximation of complicated dynamical behavior. In particular, we develop numerical methods which allow to approximate SBRmeasures as well as (almost) cyclic behavior of a dynamical system. The methods are based on an appropriate discretization of ..."
Abstract

Cited by 73 (25 self)
 Add to MetaCart
We present efficient techniques for the numerical approximation of complicated dynamical behavior. In particular, we develop numerical methods which allow to approximate SBRmeasures as well as (almost) cyclic behavior of a dynamical system. The methods are based on an appropriate discretization of the FrobeniusPerron operator, and two essentially different mathematical concepts are used: the idea is to combine classical convergence results for finite dimensional approximations of compact operators with results from Ergodic Theory concerning the approximation of SBRmeasures by invariant measures of stochastically perturbed systems. The efficiency of the methods is illustrated by several numerical examples. Key words. computation of invariant measures, approximation of the FrobeniusPerron operator, approximation of (almost) invariant sets, approximation of (almost) cyclic behavior AMS subject classification. 58F11, 65L60, 58F12 Research of the authors is partly supported by the D...
Some Perturbation Theory for Linear Programming
 Mathematical Programming
, 1992
"... This paper examines a few relations between solution characteristics of an LP and the amount by which the LP must be perturbed to obtain either a primal infeasible LP or a dual infeasible LP. We consider such solution characteristics as the size of the optimal solution and the sensitivity of the opt ..."
Abstract

Cited by 72 (2 self)
 Add to MetaCart
This paper examines a few relations between solution characteristics of an LP and the amount by which the LP must be perturbed to obtain either a primal infeasible LP or a dual infeasible LP. We consider such solution characteristics as the size of the optimal solution and the sensitivity of the optimal value to data perturbations. We show, for example, that an LP has a large optimal solution, or has a sensitive optimal value, only if the instance is nearly primal infeasible or dual infeasible. The results are not particularly surprising but they do formalize an interesting viewpoint which apparently has not been made explicit in the linear programming literature. The results are rather general. Several of the results are valid for linear programs defined in arbitrary real normed spaces. A HahnBanach Theorem is the main tool employed in the analysis; given a closed convex set in a normed vector space and a point in the space but not in the set, there exists a continuous linear functional strictly separating the set from the point. We introduce notation, then the results. Let X;Y denote real vector spaces, each with a norm. We use the same notation (i.e. k k) for all norms, it being clear from context which norm is referred to. Let X
NonEquilibrium Statistical Mechanics of Anharmonic Chains Coupled to Two Heat Baths at Different Temperatures
, 1999
"... . We study the statistical mechanics of a finitedimensional nonlinear Hamiltonian system (a chain of anharmonic oscillators) coupled to two heat baths (described by wave equations). Assuming that the initial conditions of the heat baths are distributed according to the Gibbs measures at two differ ..."
Abstract

Cited by 53 (14 self)
 Add to MetaCart
. We study the statistical mechanics of a finitedimensional nonlinear Hamiltonian system (a chain of anharmonic oscillators) coupled to two heat baths (described by wave equations). Assuming that the initial conditions of the heat baths are distributed according to the Gibbs measures at two different temperatures we study the dynamics of the oscillators. Under suitable assumptions on the potential and on the coupling between the chain and the heat baths, we prove the existence of an invariant measure for any temperature difference, i.e., we prove the existence of steady states. Furthermore, if the temperature difference is sufficiently small, we prove that the invariant measure is unique and mixing. In particular, we develop new techniques for proving the existence of invariant measures for random processes on a noncompact phase space. These techniques are based on an extension of the commutator method of H ormander used in the study of hypoelliptic differential operators. 1. Intr...
Weak and Measurevalued Solutions to Evolutionary PDEs
, 1996
"... L p estimates for the Cauchy problem with applications to the NavierStokes equations in exterior domains. J. Funct. Anal. 102, no. 1, 7294. Girault, V. and Raviart, P.A. (1986) Finite Element Methods for NavierStokes Equations. SCM 5, SpringerVerlag, Berlin. Giusti, E. (1984) Minimal Surfac ..."
Abstract

Cited by 51 (3 self)
 Add to MetaCart
L p estimates for the Cauchy problem with applications to the NavierStokes equations in exterior domains. J. Funct. Anal. 102, no. 1, 7294. Girault, V. and Raviart, P.A. (1986) Finite Element Methods for NavierStokes Equations. SCM 5, SpringerVerlag, Berlin. Giusti, E. (1984) Minimal Surfaces and Functions of Bounded Variation. Monographs in Mathematics, Vol. 80, Birkhauser, BaselBoston Stuttgart. Gobert, J. (1962) Une in'equation fondamentale de la th'eorie de l"elasticit'e (A fundamental inequality in elasticity theory). Bull. Soc. Roy. Sci. Li`ege 34, 182191. Gobert, J. (1971) Sur une in'egalit'e de coercivit'e (On an inequality related to coercivity). J. Math. Anal. Appl. 36, 518528. Godlewski, E. and Raviart, P.A. (1991) Hyperbolic Systems of Conservation Laws. Mathematiques & Applications, S.M.A.I., Ellipses, Paris (in English). Goldstein, S. (1963) Modern Developments in Fluid Dynamics. Oxford University Press, Oxford. Guillop'e, C. and Saut, J.C. (1990) Glo...
Practical Aspects of the MoreauYosida Regularization I: Theoretical Properties
, 1994
"... When computing the infimal convolution of a convex function f with the squared norm, one obtains the socalled MoreauYosida regularization of f . Among other things, this function has a Lipschitzian gradient. We investigate some more of its properties, relevant for optimization. Our main result co ..."
Abstract

Cited by 49 (2 self)
 Add to MetaCart
When computing the infimal convolution of a convex function f with the squared norm, one obtains the socalled MoreauYosida regularization of f . Among other things, this function has a Lipschitzian gradient. We investigate some more of its properties, relevant for optimization. Our main result concerns secondorder differentiability and is as follows. Under assumptions that are quite reasonable in optimization, the MoreauYosida is twice diffferentiable if and only if f is twice differentiable as well. In the course of our development, we give some results of general interest in convex analysis. In particular, we establish primaldual relationship between the remainder terms in the firstorder development of a convex function and its conjugate.