Results 1  10
of
57
Evolving Networks: Using the Genetic Algorithm with Connectionist Learning
 In
, 1990
"... It is appealing to consider hybrids of neuralnetwork learning algorithms with evolutionary search procedures, simply because Nature has so successfully done so. In fact, computational models of learning and evolution offer theoretical biology new tools for addressing questions about Nature that hav ..."
Abstract

Cited by 179 (2 self)
 Add to MetaCart
It is appealing to consider hybrids of neuralnetwork learning algorithms with evolutionary search procedures, simply because Nature has so successfully done so. In fact, computational models of learning and evolution offer theoretical biology new tools for addressing questions about Nature that have dogged that field since Darwin [Belew, 1990]. The concern of this paper, however, is strictly artificial: Can hybrids of connectionist learning algorithms and genetic algorithms produce more efficient and effective algorithms than either technique applied in isolation? The paper begins with a survey of recent work (by us and others) that combines Holland's Genetic Algorithm (GA) with connectionist techniques and delineates some of the basic design problems these hybrids share. This analysis suggests the dangers of overly literal representations of the network on the genome (e.g., encoding each weight explicitly). A preliminary set of experiments that use the GA to find unusual but successf...
Symbolic and neural learning algorithms: an experimental comparison
 Machine Learning
, 1991
"... Abstract Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with ..."
Abstract

Cited by 99 (6 self)
 Add to MetaCart
Abstract Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perception and backpropagation neural learning algorithms have been performed using five large, realworld data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a "distributed " output encoding.
Biological constraints on connectionist modelling
 Connectionism in Perspective
, 1989
"... Many researchers interested in connectionist models accept that such models are "neurally inspired " but do not worry too much about whether their models are biologically realistic. While such a position may be perfectly justifiable, the present paper attempts to illustrate how biological ..."
Abstract

Cited by 75 (8 self)
 Add to MetaCart
Many researchers interested in connectionist models accept that such models are "neurally inspired " but do not worry too much about whether their models are biologically realistic. While such a position may be perfectly justifiable, the present paper attempts to illustrate how biological information can be used to constrain connectionist models. Two particular areas are discussed. The first section deals with visual information processing in the primate and human visual system. It is argued that speed with which visual information is processed imposes major constraints on the architecture and operation of the visual system. In particular, it seems that a great deal of processing must depend on a single bottumup pass. The second section deals with biological aspects of learning algorithms. It is argued that although there is good evidence for certain coactivation related synaptic modification schemes, other learning mechanisms, including backpropagation, are not currently supported by experimental data.
On The Problem Of Local Minima In Backpropagation
 IEEE Transactions on Pattern Analysis and Machine Intelligence
, 1992
"... Supervised Learning in MultiLayered Neural Networks (MLNs) has been recently proposed through the wellknown Backpropagation algorithm. This is a gradient method which can get stuck in local minima, as simple examples can show. In this paper, some conditions on the network architecture and the lear ..."
Abstract

Cited by 72 (17 self)
 Add to MetaCart
Supervised Learning in MultiLayered Neural Networks (MLNs) has been recently proposed through the wellknown Backpropagation algorithm. This is a gradient method which can get stuck in local minima, as simple examples can show. In this paper, some conditions on the network architecture and the learning environment are proposed which ensure the convergence of the Backpropagation algorithm. It is proven in particular that the convergence holds if the classes are linearlyseparable. In this case, the experience gained in several experiments shows that MLNs exceed perceptrons in generalization to new examples. Index Terms MultiLayered Networks, learning environment, Backpropagation, pattern recognition, linearlyseparable classes. I. Introduction Supervised learning in MultiLayered Networks can be accomplished thanks to Backpropagation (BP ) ([19, 25, 31]). Its application to several different subjects [25], and, particularly, to pattern recognition ([3, 6, 8, 20, 27, 29]), has bee...
A Survey of Fuzzy Clustering Algorithms for Pattern Recognition  Part 11
"... the concepts of fuzzy clustering and soft competitive learning in clustering algorithms is proposed on the basis of the existing literature. Moreover, a set of functional attributes is selected for use as dictionary entries in the comparison of clustering algorithms. In this paper, five clustering a ..."
Abstract

Cited by 50 (2 self)
 Add to MetaCart
the concepts of fuzzy clustering and soft competitive learning in clustering algorithms is proposed on the basis of the existing literature. Moreover, a set of functional attributes is selected for use as dictionary entries in the comparison of clustering algorithms. In this paper, five clustering algorithms taken from the literature are reviewed, assessed and compared on the basis of the selected properties of interest. These clustering models are 1) selforganizing map (SOM); 2) fuzzy learning vector quantization (FLVQ); 3) fuzzy adaptive resonance theory (fuzzy ART); 4) growing neural gas (GNG); 5) fully selforganizing simplified adaptive resonance theory (FOSART). Although our theoretical comparison is fairly simple, it yields observations that may appear parodoxical. First, only FLVQ, fuzzy ART, and FOSART exploit concepts derived from fuzzy set theory (e.g., relative and/or absolute fuzzy membership functions). Secondly, only SOM, FLVQ, GNG, and FOSART employ soft competitive learning mechanisms, which are affected by asymptotic misbehaviors in the case of FLVQ, i.e., only SOM, GNG, and FOSART are considered effective fuzzy clustering algorithms. Index Terms—Ecological net, fuzzy clustering, modular architecture, relative and absolute membership function, soft and hard competitive learning, topologically correct mapping. I.
PP is Closed Under TruthTable Reductions
 Information and Computation
, 1991
"... Beigel, Reingold and Spielman [BRS] showed that PP is closed under intersection and a variety of special cases of truthtable closure. We extend the techniques in [BRS] to show PP is closed under general polynomialtime truthtable reductions. 1 Introduction In the seminal paper on probabilistic co ..."
Abstract

Cited by 41 (2 self)
 Add to MetaCart
Beigel, Reingold and Spielman [BRS] showed that PP is closed under intersection and a variety of special cases of truthtable closure. We extend the techniques in [BRS] to show PP is closed under general polynomialtime truthtable reductions. 1 Introduction In the seminal paper on probabilistic computation, Gill [G] defined the class PP, the class of problems decidable by a probabilistic polynomialtime Turing machine that need only accept a string with probability at least onehalf. Gill left open the question as to whether PP is closed under intersection. Recently Beigel, Reingold and Spielman [BRS] showed that in fact PP is closed under intersection. They also showed PP is closed under a variety of other reductions including polynomialtime conjunctive and disjunctive reductions, boundeddepth Boolean formula reductions, O(logn) Turing reductions, threshold reductions, symmetric reductions, and multilinear reductions. However they left open the question as to whether PP is closed ...
Representation of Finite State Automata in Recurrent Radial Basis Function Networks
, 1996
"... to :hs paper we propose some techniques ft>r injccling linite Stale automata rate l.ec:rr,zn Radial Basis Functlt>n networks (R2BF). When providing proper hints and constraining the v,oght space prlpe'ly. we show that thc,e nelworks behave as automata. A teebraque is snggcsted /"t ebrorag the lemmn ..."
Abstract

Cited by 36 (5 self)
 Add to MetaCart
to :hs paper we propose some techniques ft>r injccling linite Stale automata rate l.ec:rr,zn Radial Basis Functlt>n networks (R2BF). When providing proper hints and constraining the v,oght space prlpe'ly. we show that thc,e nelworks behave as automata. A teebraque is snggcsted /"t ebrorag the lemmng process re develop aulomata representationq that is based on adding a pro)per penalty tunelton to the mdinary cost. Successful experinental results are shown for tuducttvc mcrenc.' 1 regular gramrnar Keywords: Attemala, backpropagation t[rough trine, high(rder neural networks, induclix. c reference. learning item hints. radial basis ftlnctions, rectarent radial basra tnnclmns. recurrent netw(>rks 1. introduction The ability (>f learning fiom examples is certainly lhe most appealing l'eature c)f neu ral networks. In the last lw years, several researchers have used conncctontst models for solving different kinds ol probfoms ranging from robot control to pattern recogmtioa Coping wilh optimization of [unctions with several thousands of x, ariablcs s quite common Surprisingly, in many practical cases, global or near global r)ptimization is attained also wth non sophistteated numertcal methods. For example, successlul applications of neural nets fi)r recognition of handwritten characters (le Cun, 189) md for phoncmc discrimination (Waibcl c al., 1989) ave bccn proposed which d() n<,t report serious convergence problems Some attempts to understand the theoretical reasons )r lhc successes and atlures of supervised }earrang schemes have been carried oat which explain when such schemes are likely to succeed in discovering oplmal solutions (Bmnchini cl al.. 1994; Gori & Tesi, 1992; Yu, 192), and to gencrali7c to new examples (Baum & Haussler. 1989L These results give st>me ...
Separating AC 0 from depth2 majority circuits
 In Proc. of the 39th Symposium on Theory of Computing (STOC
, 2007
"... Abstract. We construct a function in AC 0 that cannot be computed by a depth2 majority circuit of size less than exp(Θ(n 1/5)). This solves an open problem due to Krause and Pudlák (1994) and matches Allender’s classic result (1989) that AC 0 can be efficiently simulated by depth3 majority circuit ..."
Abstract

Cited by 36 (17 self)
 Add to MetaCart
Abstract. We construct a function in AC 0 that cannot be computed by a depth2 majority circuit of size less than exp(Θ(n 1/5)). This solves an open problem due to Krause and Pudlák (1994) and matches Allender’s classic result (1989) that AC 0 can be efficiently simulated by depth3 majority circuits. To obtain our result, we develop a novel technique for proving lower bounds on communication complexity. This technique, the Degree/Discrepancy Theorem, is of independent interest. It translates lower bounds on the threshold degree of a Boolean function into upper bounds on the discrepancy of a related function. Upper bounds on the discrepancy, in turn, immediately imply lower bounds on communication and circuit size. In particular, our work yields the first known function in AC 0 with exponentially small discrepancy, exp(−Ω(n 1/5)). Key words. Majority circuits, constantdepth AND/OR/NOT circuits, communication complexity, discrepancy, threshold degree of Boolean functions. AMS subject classifications. 03D15, 68Q15, 68Q17
The pattern matrix method for lower bounds on quantum communication
 In Proc. of the 40th Symposium on Theory of Computing (STOC
, 2007
"... In a breakthrough result, Razborov (2003) gave optimal lower bounds on the communication complexity of every function f of the form f (x, y) = D(x ∧ y) for some D: {0, 1,..., n} → {0, 1}, in the boundederror quantum model with and without prior entanglement. This was proved by the multidimension ..."
Abstract

Cited by 35 (11 self)
 Add to MetaCart
In a breakthrough result, Razborov (2003) gave optimal lower bounds on the communication complexity of every function f of the form f (x, y) = D(x ∧ y) for some D: {0, 1,..., n} → {0, 1}, in the boundederror quantum model with and without prior entanglement. This was proved by the multidimensional discrepancy method. We give an entirely different proof of Razborov’s result, using the original, onedimensional discrepancy method. This refutes the commonly held intuition (Razborov 2003) that the original discrepancy method fails for functions such as disjointness. More importantly, our communication lower bounds hold for a much broader class of functions for which no methods were available. Namely, fix an arbitrary function f: {0, 1} n/4 → {0, 1} and let A be the Boolean matrix whose columns are each an application of f to some subset of the variables x1, x2,..., xn. We prove that the communication complexity of A in the boundederror quantum model with and without prior entanglement is Ω(d), where d is the approximate degree of f. From this result, Razborov’s lower bounds follow easily. Our result also establishes a large new class of total Boolean functions whose quantum communication complexity (regardless of prior entanglement) is at best polynomially smaller than their classical complexity. Our proof method is a novel combination of two ingredients. The first is a certain equivalence of approximation and orthogonality in Euclidean nspace, which follows by linearprogramming duality. The second is a new construction of suitably structured matrices with low spectral norm, the pattern matrices, which we realize using matrix analysis and the Fourier transform over Z n 2. The method of this paper has recently inspired important progress in multiparty communication complexity. 1
Communication lower bounds using dual polynomials
 Bulletin of the EATCS
"... Representations of Boolean functions by real polynomials play an important role in complexity theory. Typically, one is interested in the least degree of a polynomial p(x1,..., xn) that approximates or signrepresents a given Boolean function f (x1,..., xn). This article surveys a new and growing bo ..."
Abstract

Cited by 19 (8 self)
 Add to MetaCart
Representations of Boolean functions by real polynomials play an important role in complexity theory. Typically, one is interested in the least degree of a polynomial p(x1,..., xn) that approximates or signrepresents a given Boolean function f (x1,..., xn). This article surveys a new and growing body of work in communication complexity that centers around the dual objects, i.e., polynomials that certify the difficulty of approximating or signrepresenting a given function. We provide a unified guide to the following results, complete with all the key proofs: • Sherstov’s Degree/Discrepancy Theorem, which translates lower bounds on the threshold degree of a Boolean function into upper bounds on the discrepancy of a related function; • Two different methods for proving lower bounds on boundederror communication based on the approximate degree: Sherstov’s pattern matrix method and Shi and Zhu’s block composition method; • Extension of the pattern matrix method to the multiparty model, obtained by Lee and Shraibman and by Chattopadhyay and Ada, and the resulting improved lower bounds for disjointness; • David and Pitassi’s separation of NP and BPP in multiparty communication complexity for k � (1 − ɛ) log n players.