Results 1  10
of
278
Monotone Complexity
, 1990
"... We give a general complexity classification scheme for monotone computation, including monotone spacebounded and Turing machine models not previously considered. We propose monotone complexity classes including mAC i , mNC i , mLOGCFL, mBWBP , mL, mNL, mP , mBPP and mNP . We define a simple ..."
Abstract

Cited by 2837 (11 self)
 Add to MetaCart
We give a general complexity classification scheme for monotone computation, including monotone spacebounded and Turing machine models not previously considered. We propose monotone complexity classes including mAC i , mNC i , mLOGCFL, mBWBP , mL, mNL, mP , mBPP and mNP . We define a simple notion of monotone reducibility and exhibit complete problems. This provides a framework for stating existing results and asking new questions. We show that mNL (monotone nondeterministic logspace) is not closed under complementation, in contrast to Immerman's and Szelepcs 'enyi's nonmonotone result [Imm88, Sze87] that NL = coNL; this is a simple extension of the monotone circuit depth lower bound of Karchmer and Wigderson [KW90] for stconnectivity. We also consider mBWBP (monotone bounded width branching programs) and study the question of whether mBWBP is properly contained in mNC 1 , motivated by Barrington's result [Bar89] that BWBP = NC 1 . Although we cannot answer t...
Boundedwidth polynomialsize branching programs recognize exactly those languages
 in NC’, in “Proceedings, 18th ACM STOC
, 1986
"... We show that any language recognized by an NC ’ circuit (fanin 2, depth O(log n)) can be recognized by a width5 polynomialsize branching program. As any boundedwidth polynomialsize branching program can be simulated by an NC ’ circuit, we have that the class of languages recognized by such prog ..."
Abstract

Cited by 270 (14 self)
 Add to MetaCart
(Show Context)
We show that any language recognized by an NC ’ circuit (fanin 2, depth O(log n)) can be recognized by a width5 polynomialsize branching program. As any boundedwidth polynomialsize branching program can be simulated by an NC ’ circuit, we have that the class of languages recognized by such programs is exactly nonuniform NC’. Further, following
Which Problems Have Strongly Exponential Complexity?
 Journal of Computer and System Sciences
, 1998
"... For several NPcomplete problems, there have been a progression of better but still exponential algorithms. In this paper, we address the relative likelihood of subexponential algorithms for these problems. We introduce a generalized reduction which we call SubExponential Reduction Family (SERF) t ..."
Abstract

Cited by 249 (9 self)
 Add to MetaCart
(Show Context)
For several NPcomplete problems, there have been a progression of better but still exponential algorithms. In this paper, we address the relative likelihood of subexponential algorithms for these problems. We introduce a generalized reduction which we call SubExponential Reduction Family (SERF) that preserves subexponential complexity. We show that CircuitSAT is SERFcomplete for all NPsearch problems, and that for any fixed k, kSAT, kColorability, kSet Cover, Independent Set, Clique, Vertex Cover, are SERFcomplete for the class SNP of search problems expressible by second order existential formulas whose first order part is universal. In particular, subexponential complexity for any one of the above problems implies the same for all others. We also look at the issue of proving strongly exponential lower bounds for AC 0 ; that is, bounds of the form 2 \Omega\Gamma n) . This problem is even open for depth3 circuits. In fact, such a bound for depth3 circuits with even l...
Learning Deep Architectures for AI
"... Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as i ..."
Abstract

Cited by 182 (32 self)
 Add to MetaCart
Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as in neural nets with many hidden layers or in complicated propositional formulae reusing many subformulae. Searching the parameter space of deep architectures is a difficult task, but learning algorithms such as those for Deep Belief Networks have recently been proposed to tackle this problem with notable success, beating the stateoftheart in certain areas. This paper discusses the motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of singlelayer models such as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks.
Every monotone graph property has a sharp threshold
 PROC. AMER. MATH. SOC
, 1996
"... In their seminal work which initiated random graph theory Erdös and Rényi discovered that many graph properties have sharp thresholds as the number of vertices tends to infinity. We prove a conjecture of Linial that every monotone graph property has a sharp threshold. This follows from the followin ..."
Abstract

Cited by 176 (16 self)
 Add to MetaCart
(Show Context)
In their seminal work which initiated random graph theory Erdös and Rényi discovered that many graph properties have sharp thresholds as the number of vertices tends to infinity. We prove a conjecture of Linial that every monotone graph property has a sharp threshold. This follows from the following theorem. Let Vn(p) ={0,1} n denote the Hamming space endowed with the probability measure µp defined by µp(ɛ1,ɛ2,...,ɛn) = pk ·(1 − p) n−k,where k = ɛ1+ ɛ2+ ···+ ɛn. Let A be a monotone subset of Vn. We say that A is symmetric if there is a transitive permutation group Γ on {1, 2,...,n} such that A is invariant under Γ. Theorem. For every symmetric monotone A,ifµp(A)>ɛthen µq(A)> 1−ɛ for q = p + c1 log(1/2ɛ) / log n. (c1isan absolute constant.) 1. Graph properties A graph property is a property of graphs which depends only on their isomorphism class. Let P be a monotone graph property; that is, if a graph G satisfies P
Representation Learning: A Review and New Perspectives
, 2012
"... The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to ..."
Abstract

Cited by 152 (4 self)
 Add to MetaCart
The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representationlearning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and joint training of deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep architectures. This motivates longerterm unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning.
Why does unsupervised pretraining help deep learning?
, 2010
"... Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Networks and stacks of autoencoder variants with impressive results being obtained in several areas, mostly on vision and language datasets. The best results obtained on supervised learning tasks ..."
Abstract

Cited by 143 (20 self)
 Add to MetaCart
Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Networks and stacks of autoencoder variants with impressive results being obtained in several areas, mostly on vision and language datasets. The best results obtained on supervised learning tasks often involve an unsupervised learning component, usually in an unsupervised pretraining phase. The main question investigated here is the following: why does unsupervised pretraining work so well? Through extensive experimentation, we explore several possible explanations discussed in the literature including its action as a regularizer (Erhan et al., 2009b) and as an aid to optimization (Bengio et al., 2007). Our results build on the work of Erhan et al. (2009b), showing that unsupervised pretraining appears to play predominantly a regularization role in subsequent supervised training. However our results in an online setting, with a virtually unlimited data stream, point to a somewhat more nuanced interpretation of the roles of optimization and regularization in the unsupervised pretraining effect.
A new approach to the minimum cut problem
 Journal of the ACM
, 1996
"... Abstract. This paper presents a new approach to finding minimum cuts in undirected graphs. The fundamental principle is simple: the edges in a graph’s minimum cut form an extremely small fraction of the graph’s edges. Using this idea, we give a randomized, strongly polynomial algorithm that finds th ..."
Abstract

Cited by 126 (9 self)
 Add to MetaCart
(Show Context)
Abstract. This paper presents a new approach to finding minimum cuts in undirected graphs. The fundamental principle is simple: the edges in a graph’s minimum cut form an extremely small fraction of the graph’s edges. Using this idea, we give a randomized, strongly polynomial algorithm that finds the minimum cut in an arbitrarily weighted undirected graph with high probability. The algorithm runs in O(n 2 log 3 n) time, a significant improvement over the previous Õ(mn) time bounds based on maximum flows. It is simple and intuitive and uses no complex data structures. Our algorithm can be parallelized to run in �� � with n 2 processors; this gives the first proof that the minimum cut problem can be solved in ���. The algorithm does more than find a single minimum cut; it finds all of them. With minor modifications, our algorithm solves two other problems of interest. Our algorithm finds all cuts with value within a multiplicative factor of � of the minimum cut’s in expected Õ(n 2 � ) time, or in �� � with n 2 � processors. The problem of finding a minimum multiway cut of a graph into r pieces is solved in expected Õ(n 2(r�1) ) time, or in �� � with n 2(r�1) processors. The “trace ” of the algorithm’s execution on these two problems forms a new compact data structure for representing all small cuts and all multiway cuts in a graph. This data structure can be efficiently transformed into the
Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion
, 2010
"... ..."
An Improved Exponentialtime Algorithm for kSAT
, 1998
"... We propose and analyze a simple new randomized algorithm, called ResolveSat, for finding satisfying assignments of Boolean formulas in conjunctive normal form. The algorithm consists of two stages: a preprocessing stage in which resolution is applied to enlarge the set of clauses of the formula, ..."
Abstract

Cited by 116 (7 self)
 Add to MetaCart
We propose and analyze a simple new randomized algorithm, called ResolveSat, for finding satisfying assignments of Boolean formulas in conjunctive normal form. The algorithm consists of two stages: a preprocessing stage in which resolution is applied to enlarge the set of clauses of the formula, followed by a search stage that uses a simple randomized greedy procedure to look for a satisfying assignment. We show that, for each k, the running time of ResolveSat on a kCNF formula is significantly better than 2 n , even in the worst case. In particular, we show that the algorithm finds a satisfying assignment of a general satisfiable 3CNF in time O(2 :448n ) with high probability; where the best previous algorithm [13] has running time O(2 :562n ). We obtain a better upper bound of 2 (2 ln 2\Gamma1)n+o(n) = O(2 0:387n ) for 3CNF that have exactly one satisfying assignment (unique kSAT). For each k, the bounds for general kCNF are the best currently known for ...