”This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.”

Abstract: Developments in computational models of evolutionary processes have led to the realisation of powerful, robust and general optimization and adaptive systems collectively called evolutionary algorithms. In this paper we provide an overview of evolutionary algorithms and consider the features and characteristics that are particularly appropriate for control engineering applications. The versatile and robust qualities of these algorithms are considered and a number of application areas described.

Abstract not found

The paper considers the problem of detecting the sparsity pattern of a k-sparse vector in R n from m random noisy measurements. A new necessary condition on the number of measurements for asymptotically reliable detection with maximum likelihood (ML) estimation and Gaussian measurement matrices is derived. This necessary condition for ML detection is compared against a sufficient condition for simple maximum correlation (MC) or thresholding algorithms. The analysis shows that the gap between thresholding and ML can be described by a simple expression in terms of the total signal-to-noise ratio (SNR), with the gap growing with increasing SNR. Thresholding is also compared against the more sophisticated lasso and orthogonal matching pursuit (OMP) methods. At high SNRs, it is shown that the gap between lasso and OMP over thresholding is described by the range of powers of the nonzero component values of the unknown signals. Specifically, the key benefit of lasso and OMP over thresholding is the ability of lasso and OMP to detect signals with relatively small components.

Sparse and redundant representations – an emerging and powerful model for signals – suggests that a data source could be described as a linear combination of few atoms from a pre-specified and over-complete dictionary. This model has drawn a considerable attention in the past decade, due to its appealing theoretical foundations, and promising practical results it leads to. Many of the applications that use this model are formulated as a mixture of ℓ2-ℓp (p ≤ 1) optimization expressions. Iterated Shrinkage algorithms are a new family of highly effective numerical techniques for handling these optimization tasks, surpassing traditional optimization techniques. In this paper we aim to give a broad view of this group of methods, motivate their need, present their derivation, show their comparative performance, and most important of all, discuss their potential in various applications.

Genetic algorithms (GAs) are stochastic search techniques inspired by the principles of natural selection and natural genetics which have revealed a number of characteristics particularly useful for applications in optimization, engineering, and computer science, among other fields. In control engineering, they have found application mainly in problems involving functions difficult to characterize mathematically or known to present difficulties to more conventional numerical optimizers, as well as problems involving non-numeric and mixed-type variables. In addition, they exhibit a large degree of parallelism, making it possible to effectively exploit the computing power made available through parallel processing. Despite their early recognized potential for multiobjective optimization (almost all engineering problems involve multiple, often conflicting objectives), genetic algorithms have, for the most part, been applied to aggregations of the objectives in a single-objective fashion, like conventional optimizers. Although alternative approaches based on the notion of Pareto-dominance have been suggested, multiobjective optimization with genetic algorithms has received comparatively

The paper introduces an efficient construction algorithm for obtaining sparse linear-in-the-weights regression models based on an approach of directly optimizing model generalization capability. This is achieved by utilizing the delete-1 cross validation concept and the associated leave-one-out test error also known as the predicted residual sums of squares (PRESS) statistic, without resorting to any other validation data set for model evaluation in the model construction process. Computational efficiency is ensured using an orthogonal forward regression, but the algorithm incrementally minimizes the PRESS statistic instead of the usual sum of the squared training errors. A local regularization method can naturally be incorporated into the model selection procedure to further enforce model sparsity. The proposed algorithm is fully automatic, and the user is not required to specify any criterion to terminate the model construction procedure. Comparisons with some of the existing state-of-art modeling methods are given, and several examples are included to demonstrate the ability of the proposed algorithm to effectively construct sparse models that generalize well.

Linear expansions of images nd many applications in image processing and computer vision.

Abstract—To date, clustering techniques have always been oriented to solve classification and pattern recognition problems. However, some authors have applied them unchanged to construct initial models for function approximators. Nevertheless, classification and function approximation problems present quite different objectives. Therefore it is necessary to design new clustering algorithms specialized in the problem of function approximation. This paper presents a new clustering technique, specially designed for function approximation problems, which improves the performance of the approximator system obtained, compared with other models derived from traditional classification oriented clustering algorithms and input–output clustering techniques. Index Terms—Clustering techniques, function approximation, model initialization. I.

A locally regularized orthogonal least squares (LROLS) algorithm is proposed for constructing parsimonious or sparse regression models that generalize well. By associating each orthogonal weight in the regression model with an individual regularization parameter, the ability for the orthogonal least squares (OLS) model selection to produce a very sparse model with good generalization performance is greatly enhanced. Furthermore, with the assistance of local regularization, when to terminate the model selection procedure becomes much clearer. This LROLS algorithm has computational advantages over the recently introduced relevance vector machine (RVM) method. Keywords Orthogonal least squares algorithm, regularization, regression, support vector machines, relevance vector machines. I.