Results 1 
6 of
6
Learning Deep Architectures for AI
"... Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as i ..."
Abstract

Cited by 182 (32 self)
 Add to MetaCart
Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as in neural nets with many hidden layers or in complicated propositional formulae reusing many subformulae. Searching the parameter space of deep architectures is a difficult task, but learning algorithms such as those for Deep Belief Networks have recently been proposed to tackle this problem with notable success, beating the stateoftheart in certain areas. This paper discusses the motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of singlelayer models such as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks.
Toward accurate polynomial evaluation in rounded arithmetic
 In Foundations of computational mathematics, Santander 2005, volume 331 of London Math. Soc. Lecture Note Ser
, 2006
"... Given a multivariate real (or complex) polynomial p and a domain D, we would like to decide whether an algorithm exists to evaluate p(x) accurately for all x ∈ D using rounded real (or complex) arithmetic. Here “accurately ” means with relative error less than 1, i.e., with some correct leading digi ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Given a multivariate real (or complex) polynomial p and a domain D, we would like to decide whether an algorithm exists to evaluate p(x) accurately for all x ∈ D using rounded real (or complex) arithmetic. Here “accurately ” means with relative error less than 1, i.e., with some correct leading digits. The answer depends on the model of rounded arithmetic: We assume that for any arithmetic operator op(a, b), for example a+b or a·b, its computed value is op(a, b)·(1+δ), where δ  is bounded by some constant ǫ where 0 < ǫ ≪ 1, but δ is otherwise arbitrary. This model is the traditional one used to analyze the accuracy of floating point algorithms. Our ultimate goal is to establish a decision procedure that, for any p and D, either exhibits an accurate algorithm or proves that none exists. In contrast to the case where numbers are stored and manipulated as finite bit strings (e.g., as floating point numbers or rational numbers) we show that some polynomials p are impossible to evaluate accurately. The existence of an accurate algorithm will depend not just on p and D, but on which arithmetic operators and constants are available to the algorithm and whether branching is permitted in the algorithm. Toward this goal, we present necessary conditions on p for it to be accurately evaluable on open real or complex domains D. We also give sufficient conditions, and describe progress toward a complete decision procedure. We do present a complete decision procedure for homogeneous polynomials p with integer coefficients, D = C n, using only arithmetic operations +, − and ·. 1
Approximation and complexity: Liouvillean type theorems for linear differential equations on an interval
, 2001
"... We prove a lower bound on approximations on an interval of a function v by means of another function u satisfying linear differential equations P (u) = Q(v) = 0. Informally speaking, the result says that if complexities of P and Q are sufficiently small then the difference jv \Gamma uj could not be ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
We prove a lower bound on approximations on an interval of a function v by means of another function u satisfying linear differential equations P (u) = Q(v) = 0. Informally speaking, the result says that if complexities of P and Q are sufficiently small then the difference jv \Gamma uj could not be too small on the whole interval. One could view this result as a differential analog of the Liouvillean theorem which tells that two different algebraic numbers are well separated if they satisfy algebraic equations with small enough integer coefficients. Introduction The wellknown Liouvillean theorem tells that if f(a) = g(b) = 0 where f = P 0in f i X i ; g = P 0im g i X i 2 Z[X] and the algebraic numbers a 6= b then one can bound from below the difference ja\Gammabj. For the sake of simplicity assume that f; g have no common roots. One possible approach to its proof is to consider the resultant R = f m n g n m Q (a i \Gamma b j ) 2 Z where the product is taken over...
1 Learning Deep Architectures for AI
"... Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as i ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as in neural nets with many hidden layers or in complicated propositional formulae reusing many subformulae. Searching the parameter space of deep architectures is a difficult task, but learning algorithms such as those for Deep Belief Networks have recently been proposed to tackle this problem with notable success, beating the stateoftheart in certain areas. This paper discusses the motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of singlelayer models such as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks. 1
Approximation and complexity II: iterated integration
"... We introduce two classes of real analytic functions W U on an interval. Starting with rational functions, for constructing functions from W we allow applying two types of operations: integrating and multiplying by a polynomial with rational coecients. In a similar way, for constructing function ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
We introduce two classes of real analytic functions W U on an interval. Starting with rational functions, for constructing functions from W we allow applying two types of operations: integrating and multiplying by a polynomial with rational coecients. In a similar way, for constructing functions from U we allow integrating and additions and multiplications of already constructed functions from U and multiplications by rational numbers. Thus, U is a subring of the ring of Pfaan functions [Kh].