Results 1  10
of
88
Simple statistical gradientfollowing algorithms for connectionist reinforcement learning
 Machine Learning
, 1992
"... Abstract. This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units. These algorithms, called REINFORCE algorithms, are shown to make weight adjustments in a direction that lies along the gradient of expected reinfor ..."
Abstract

Cited by 327 (0 self)
 Add to MetaCart
Abstract. This article presents a general class of associative reinforcement learning algorithms for connectionist networks containing stochastic units. These algorithms, called REINFORCE algorithms, are shown to make weight adjustments in a direction that lies along the gradient of expected reinforcement in both immediatereinforcement tasks and certain limited forms of delayedreinforcement tasks, and they do this without explicitly computing gradient estimates or even storing information from which such estimates could be computed. Specific examples of such algorithms are presented, some of which bear a close relationship to certain existing algorithms while others are novel but potentially interesting in their own right. Also given are results that show how such algorithms can be naturally integrated with backpropagation. We close with a brief discussion of a number of additional issues surrounding the use of such algorithms, including what is known about their limiting behaviors as well as further considerations that might be used to help develop similar but potentially more powerful reinforcement learning algorithms.
Local search algorithms for SAT: An empirical evaluation
 JOURNAL OF AUTOMATED REASONING
, 2000
"... Local search algorithms are among the standard methods for solving hard combinatorial problems from various areas of Artificial Intelligence and Operations Research. For SAT, some of the most successful and powerful algorithms are based on stochastic local search and in the past 10 years a large num ..."
Abstract

Cited by 62 (18 self)
 Add to MetaCart
Local search algorithms are among the standard methods for solving hard combinatorial problems from various areas of Artificial Intelligence and Operations Research. For SAT, some of the most successful and powerful algorithms are based on stochastic local search and in the past 10 years a large number of such algorithms have been proposed and investigated. In this article, we focus on two particularly wellknown families of local search algorithms for SAT, the GSAT and WalkSAT architectures. We present a detailed comparative analysis of these algorithms' performance using a benchmark set which contains instances from randomised distributions as well as SATencoded problems from various domains. We also investigate the robustness of the observed performance characteristics as algorithmdependent and problemdependent parameters are changed. Our empirical analysis gives a very detailed picture of the algorithms' performance for various domains of SAT problems; it also reveals a fundamental weakness in some of the bestperforming algorithms and shows how this can be overcome.
Evaluating Las Vegas algorithms  pitfalls and remedies
 IN PROCEEDINGS OF THE FOURTEENTH CONFERENCE ON UNCERTAINTY IN ARTIFICIAL INTELLIGENCE (UAI98
, 1998
"... Stochastic search algorithms are among the most sucessful approaches for solving hard combinatorial problems. A large class of stochastic search approaches can be cast into the framework of Las Vegas Algorithms (LVAs). As the runtime behavior of LVAs is characterized by random variables, the detail ..."
Abstract

Cited by 52 (21 self)
 Add to MetaCart
Stochastic search algorithms are among the most sucessful approaches for solving hard combinatorial problems. A large class of stochastic search approaches can be cast into the framework of Las Vegas Algorithms (LVAs). As the runtime behavior of LVAs is characterized by random variables, the detailed knowledge of runtime distributions provides important information for the analysis of these algorithms. In this paper we propose a novel methodology for evaluating the performance of LVAs, based on the identification of empirical runtime distributions. We exemplify our approach by applying it to Stochastic Local Search (SLS) algorithms for the satisfiability problem (SAT) in propositional logic. We point out pitfalls arising from the use of improper empirical methods and discuss the benefits of the proposed methodology for evaluating and comparing LVAs.
Towards a characterisation of the behaviour of stochastic local search algorithms for SAT
 ARTIFICIAL INTELLIGENCE
, 1999
"... Stochastic local search (SLS) algorithms have been successfully applied to hard combinatorial problems from different domains. Due to their inherent randomness, the runtime behaviour of these algorithms is characterised by a random variable. The detailed knowledge of the runtime distribution provi ..."
Abstract

Cited by 45 (16 self)
 Add to MetaCart
Stochastic local search (SLS) algorithms have been successfully applied to hard combinatorial problems from different domains. Due to their inherent randomness, the runtime behaviour of these algorithms is characterised by a random variable. The detailed knowledge of the runtime distribution provides important information about the behaviour of SLS algorithms. In this paper we investigate the empirical runtime distributions for Walksat, one of the most powerful SLS algorithms for the Propositional Satisfiability Problem (SAT). Using statistical analysis techniques, we show that on hard Random3SAT problems, Walksat's runtime behaviour can be characterised by exponential distributions. This characterisation can be generalised to various SLS algorithms for SAT and to encoded problems from other domains. This result also has a number of consequences which are of theoretical as well as practical interest. One of these is the fact that these algorithms can be easily parallelised such that optimal speedup is achieved for hard problem instances.
Characterizing the Runtime Behavior of Stochastic Local Search
 IN PROCEEDINGS AAAI99
, 1998
"... Stochastic local search (SLS) algorithms have been successfully applied to hard combinatorial problems from different domains. One important feature of SLS algorithms is the fact that their runtime behavior is characterized by a random variable. Consequently, the detailed knowledge of the runtime ..."
Abstract

Cited by 25 (4 self)
 Add to MetaCart
Stochastic local search (SLS) algorithms have been successfully applied to hard combinatorial problems from different domains. One important feature of SLS algorithms is the fact that their runtime behavior is characterized by a random variable. Consequently, the detailed knowledge of the runtime distribution provides important information for the analysis of SLS algorithms. In this paper we investigate the empirical runtime distributions for several stateoftheart stochastic local search algorithms for SAT and CSP. Using statistical analysis techniques, we show that on a variety of problems from both randomized distributions and encodings of the blocks world planning and graph coloring domains, the observed runtime behavior can be characterized by exponential distributions. As a first direct consequence of this result, we establish that these algorithms can be easily parallelized with optimal speedup.
Computing the Distribution of the Product of Two Continuous Random Variables
, 2003
"... We present an algorithm for computing the probability density function of the product of two independent random variables, along with an an implementation of the algorithm in a computer algebra system. We combine this algorithm with earlier work on transformations of random variables to create an au ..."
Abstract

Cited by 16 (2 self)
 Add to MetaCart
We present an algorithm for computing the probability density function of the product of two independent random variables, along with an an implementation of the algorithm in a computer algebra system. We combine this algorithm with earlier work on transformations of random variables to create an automated algorithm for convolutions of random variables. Some examples demonstrate the algorithm's application.
Integrating behavioral trust in web service compositions
 In Proceedings of the 7th IEEE International Conference on Web Services (ICWS
, 2009
"... Algorithms for composing Web services (WS) traditionally utilize the functional and qualityofservice parameters of candidate services to decide which services to include in the composition. Users often have differing experiences with a WS. While trust in a WS is multifaceted and consists of secur ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
Algorithms for composing Web services (WS) traditionally utilize the functional and qualityofservice parameters of candidate services to decide which services to include in the composition. Users often have differing experiences with a WS. While trust in a WS is multifaceted and consists of security and behavioral aspects, our focus in this paper is on the latter. We adopt a formal model for trust in a WS, which meets many of our intuitions about trustworthy WSs. We hypothesize predictors of a positive experience with a WS and conduct a small pilot study to explore correlations between subjects ’ experiences with WSs in a composition and the predictor values for those WSs. Furthermore, we show how we may derive trust for compositions from trust models of individual services. We conclude by presenting and evaluating a novel framework, called Wisp, that utilizes the trust models and, in combination with any WS composition tool, chooses compositions to deploy that are deemed most trustworthy. 1
A Generalized Univariate ChangeofVariable Transformation Technique
, 1997
"... o appear in introductory probability and statistics texts. Casella and Berger [3; page 51] discuss transforming random variables using the changeofvariable technique when the entire transformation is manyto1, except for a finite number of points, that is, the cardinality of the set g \Gamma ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
o appear in introductory probability and statistics texts. Casella and Berger [3; page 51] discuss transforming random variables using the changeofvariable technique when the entire transformation is manyto1, except for a finite number of points, that is, the cardinality of the set g \Gamma1 (y) is the same for almost all y in the support of Y . Hogg and Craig [5; page 190] extend this manyto1 technique to ndimensional random variables. We are concerned with a more general univariate case in which the transformations are "piecewise manyto1," where "many" may vary based on the subinterval of the support of Y under consideration. We state and prove a theorem for this case and present code in a computer algebra system to implement the result. Although the theorem is a straightforward generalization of
A.: Computing Probabilistic Least Common Subsumers in Description Logics
 LNCS (LNAI
, 1999
"... Abstract. Computing least common subsumers in description logics is an important reasoning service useful for a number of applications. As shown in the literature, it can, for instance, be used for similaritybased information retrieval where information retrieval is performed on the basis of the si ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
Abstract. Computing least common subsumers in description logics is an important reasoning service useful for a number of applications. As shown in the literature, it can, for instance, be used for similaritybased information retrieval where information retrieval is performed on the basis of the similarities of userspecified examples. In this article, we first show that, for crisp DLs, in certain cases the set of retrieved information items can be too large. Then we propose a probabilistic least common subsumer operation based on a probabilistic extension of the description logic language ALN. We show that by this operator the amount of retrieved data can be reduced avoiding information flood. 1
Genotypic and phenotypic assortative mating in genetic algorithm
 Inf. Sci
, 1998
"... Three new methods of selection of mating pairs for Genetic Algorithms (GAs) are introduced where the partners are chosen based on either their genotypic similarity (called genotypic assortative mating) or their phenotypic similarity (called phenotypic assortative mating). These methods not only help ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
Three new methods of selection of mating pairs for Genetic Algorithms (GAs) are introduced where the partners are chosen based on either their genotypic similarity (called genotypic assortative mating) or their phenotypic similarity (called phenotypic assortative mating). These methods not only help in exploiting the current search space properly before exploring the new one but also enable one to mimic inbreeding of natural genetics. A comparative study in terms of disruption of schema due to crossover is made between these methods and conventional genetic algorithm (CGA). The superiority of this new methodology over the CGA and the incest prevention algorithm is established on some problems of optimizing complex functions and selecting optimal neural