Results 11  20
of
238
Global Optimization for Neural Network Training
 IEEE Computer
, 1996
"... In this paper, we study various supervised learning methods for training feedforward neural networks. In general, such learning can be considered as a nonlinear global optimization problem in which the goal is to minimize a nonlinear error function that spans the space of weights using heuristic st ..."
Abstract

Cited by 45 (12 self)
 Add to MetaCart
(Show Context)
In this paper, we study various supervised learning methods for training feedforward neural networks. In general, such learning can be considered as a nonlinear global optimization problem in which the goal is to minimize a nonlinear error function that spans the space of weights using heuristic strategies that look for global optima (in contrast to local optima). We survey various global optimization methods suitable for neuralnetwork learning, and propose the NOVEL method, a novel global optimization method for nonlinear optimization and neural network learning. By combining global and local searches, we show how NOVEL can be used to find a good local minimum in the error space. Our key idea is to use a userdefined trace that pulls a search out of a local minimum without having to restart it from a new starting point. Using five benchmark problems, we compare NOVEL against some of the best global optimization algorithms and demonstrate its superior improvement in performance. 1 In...
A PrimalRelaxed Dual Global Optimization Approach
, 1993
"... A deterministic global optimization approach is proposed for nonconvex constrained nonlinear programming problems. Partitioning of the variables, along with the introduction of transformation variables, if necessary, convert the original problem into primal and relaxed dual subproblems that provide ..."
Abstract

Cited by 44 (20 self)
 Add to MetaCart
A deterministic global optimization approach is proposed for nonconvex constrained nonlinear programming problems. Partitioning of the variables, along with the introduction of transformation variables, if necessary, convert the original problem into primal and relaxed dual subproblems that provide valid upper and lower bounds respectively on the global optimum. Theoretical properties are presented which allow for a rigorous solution of the relaxed dual problem. Proofs of fflfinite convergence and fflglobal optimality are provided. The approach is shown to be particularly suited to (a) quadratic programming problems, (b) quadratically constrained problems, and (c) unconstrained and constrained optimization of polynomial and rational polynomial functions. The theoretical approach is illustrated through a few example problems. Finally, some further developments in the approach are briefly discussed.
Flexibility and Efficiency Enhancements for Constrained Global Design Optimization with Kriging Approximations
, 2002
"... ..."
Classification of adaptive memetic algorithms: a comparative study
 IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
, 2006
"... Abstract—Adaptation of parameters and operators represents one of the recent most important and promising areas of research in evolutionary computations; it is a form of designing selfconfiguring algorithms that acclimatize to suit the problem in hand. Here, our interests are on a recent breed of ..."
Abstract

Cited by 39 (5 self)
 Add to MetaCart
(Show Context)
Abstract—Adaptation of parameters and operators represents one of the recent most important and promising areas of research in evolutionary computations; it is a form of designing selfconfiguring algorithms that acclimatize to suit the problem in hand. Here, our interests are on a recent breed of hybrid evolutionary algorithms typically known as adaptive memetic algorithms (MAs). One unique feature of adaptive MAs is the choice of local search methods or memes and recent studies have shown that this choice significantly affects the performances of problem searches. In this paper, we present a classification of memes adaptation in adaptive MAs on the basis of the mechanism used and the level of historical knowledge on the memes employed. Then the asymptotic convergence properties of the adaptive MAs considered are analyzed according to the classification. Subsequently, empirical studies on representatives of adaptive MAs for different typelevel meme adaptations using continuous benchmark problems indicate that globallevel adaptive MAs exhibit better search performances. Finally we conclude with some promising research directions in the area. Index Terms—Adaptation, evolutionary algorithm, memetic algorithm, optimization. I.
Fuzzy Connectives Based Crossover Operators to Model Genetic Algorithms Population Diversity
, 1995
"... Genetic algorithms are adaptive methods which may be used to solve search and optimization problems. Genetic algorithms process a population of search space solutions with three operations: selection, crossover and mutation. An important problem in the use of genetic algorithms is the premature conv ..."
Abstract

Cited by 39 (24 self)
 Add to MetaCart
Genetic algorithms are adaptive methods which may be used to solve search and optimization problems. Genetic algorithms process a population of search space solutions with three operations: selection, crossover and mutation. An important problem in the use of genetic algorithms is the premature convergence in a local optimum. Their main causes are the lack of diversity in the population and the disproportionate relationship between exploitation and exploration. The crossover operator is considered one of the most determinant elements for solving this problem. In this paper, we present new crossover operators based on fuzzy connectives for realcoded genetic algorithms. These operators are designed to avoid the premature convergence problem. To do so, they should keep the right exploitation/exploration balance to suitably model the diversity of the population.
A tutorial on Bayesian optimization of expensive cost functions, withapplicationtoactiveusermodeling andhierarchical reinforcement learning
, 2009
"... We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions. Bayesian optimization employs the Bayesian technique of setting a prior over the objective function and combining it with evidence to get a posterior function. This permits a utilitybased se ..."
Abstract

Cited by 32 (3 self)
 Add to MetaCart
We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions. Bayesian optimization employs the Bayesian technique of setting a prior over the objective function and combining it with evidence to get a posterior function. This permits a utilitybased selection of the next observation to make on the objective function, which must take into account both exploration (sampling from areas of high uncertainty) and exploitation (sampling areas likely to offer improvement over the current best observation). We also present two detailed extensions of Bayesian optimization, with experiments—active user modelling with preferences, and hierarchical reinforcement learning— and a discussion of the pros and cons of Bayesian optimization based on our experiences. 1
Maximizing Speedup Through SelfTuning of Processor Allocation
 IN PROCEEDINGS OF THE 10TH INTERNATIONAL PARALLEL PROCESSING SYMPOSIUM
, 1996
"... We address the problem of maximizing the speedup of an individual parallel job through the selection of an appropriate number of processors on which to run it. If a parallel job exhibits speedup that increases monotonically in the number of processors, the solution is clearly to make use of all av ..."
Abstract

Cited by 31 (2 self)
 Add to MetaCart
We address the problem of maximizing the speedup of an individual parallel job through the selection of an appropriate number of processors on which to run it. If a parallel job exhibits speedup that increases monotonically in the number of processors, the solution is clearly to make use of all available processors. However, many applications do not have this characteristic: they reach a point beyond which the use of additional processors degrades performance. For these applications, it is important to choose a processor allocation carefully. Our approach to this problem is to provide a runtime system that adjusts the number of processors used by the application based on dynamic measurements of performance gathered during its execution. Our runtime system has a number of advantages over user specified fixed allocations, the currently most common approach to this problem: (1) we are resilient to changes in an application's speedup behavior due to the input data; and (2) we are...
Adaptation of Genetic Algorithm Parameters Based on Fuzzy Logic Controllers
 Genetic Algorithms and Soft Computing
"... . The genetic algorithm behaviour is determined by the exploitation and exploration relationship kept throughout the run. Adaptive genetic algorithms have been built for inducing exploitation/exploration relationships that avoid the premature convergence problem and improve the final results. One of ..."
Abstract

Cited by 31 (9 self)
 Add to MetaCart
(Show Context)
. The genetic algorithm behaviour is determined by the exploitation and exploration relationship kept throughout the run. Adaptive genetic algorithms have been built for inducing exploitation/exploration relationships that avoid the premature convergence problem and improve the final results. One of the most widely studied adaptive approaches are the adaptive parameter setting techniques. In this paper, we study these techniques in depth, based on the use of fuzzy logic controllers. Furthermore, we design and discuss an adaptive realcoded genetic algorithm based on the use of fuzzy logic controllers. Although suitable results have been obtained by using this type of adaptive technique, we report some reflections on open problems that still remain. Keywords. Exploitation/exploration relationship, adaptive genetic algorithms, fuzzy logic controllers. 1 Introduction GA behaviour is strongly determined by the balance between exploiting what already works best and exploring possibilities t...
On the Selection of Subdivision Directions in Interval BranchandBound Methods for Global Optimization
 J. Global Optimization
, 1995
"... . This paper investigates the influence of the interval subdivision selection rule on the convergence of interval branchandbound algorithms for global optimization. For the class of rules that allows convergence, we study the effects of the rules on a model algorithm with special list ordering. Fo ..."
Abstract

Cited by 31 (13 self)
 Add to MetaCart
. This paper investigates the influence of the interval subdivision selection rule on the convergence of interval branchandbound algorithms for global optimization. For the class of rules that allows convergence, we study the effects of the rules on a model algorithm with special list ordering. Four different rules are investigated in theory and in practice. A wide spectrum of test problems is used for numerical tests indicating that there are substantial differences between the rules with respect to the required CPU time, the number of function and derivative evaluations, and the necessary storage space. Two rules can provide considerable improvements in efficiency for our model algorithm. Keywords: Global optimization, interval arithmetic, branchandbound, interval subdivision 1. Introduction The investigated class of interval branchandbound methods for global optimization [7], [8], [19] addresses the problem of finding guaranteed and reliable solutions of global optimization...
Novel Estimation Methods for Unsupervised Discovery of Latent Structure in Natural Language Text
, 2006
"... This thesis is about estimating probabilistic models to uncover useful hidden structure in data; specifically, we address the problem of discovering syntactic structure in natural language text. We present three new parameter estimation techniques that generalize the standard approach, maximum likel ..."
Abstract

Cited by 30 (8 self)
 Add to MetaCart
(Show Context)
This thesis is about estimating probabilistic models to uncover useful hidden structure in data; specifically, we address the problem of discovering syntactic structure in natural language text. We present three new parameter estimation techniques that generalize the standard approach, maximum likelihood estimation, in different ways. Contrastive estimation maximizes the conditional probability of the observed data given a “neighborhood” of implicit negative examples. Skewed deterministic annealing locally maximizes likelihood using a cautious parameter search strategy that starts with an easier optimization problem than likelihood, and iteratively moves to harder problems, culminating in likelihood. Structural annealing is similar, but starts with a heavy bias toward simple syntactic structures and gradually relaxes the bias. Our estimation methods do not make use of annotated examples. We consider their performance in both an unsupervised model selection setting, where models trained under different initialization and regularization settings are compared by evaluating the training objective on a small set of unseen, unannotated development data, and supervised model selection, where the most accurate model on the development set (now with annotations)