Results 1  10
of
222
Learning Deep Architectures for AI
"... Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as i ..."
Abstract

Cited by 182 (32 self)
 Add to MetaCart
Theoretical results suggest that in order to learn the kind of complicated functions that can represent highlevel abstractions (e.g. in vision, language, and other AIlevel tasks), one may need deep architectures. Deep architectures are composed of multiple levels of nonlinear operations, such as in neural nets with many hidden layers or in complicated propositional formulae reusing many subformulae. Searching the parameter space of deep architectures is a difficult task, but learning algorithms such as those for Deep Belief Networks have recently been proposed to tackle this problem with notable success, beating the stateoftheart in certain areas. This paper discusses the motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of singlelayer models such as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks.
Curriculum Learning
"... Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them “curriculum ..."
Abstract

Cited by 118 (12 self)
 Add to MetaCart
(Show Context)
Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them “curriculum learning”. In the context of recent research studying the difficulty of training in the presence of nonconvex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various setups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of nonconvex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of nonconvex functions). 1.
Continuation and Path Following
, 1992
"... CONTENTS 1 Introduction 1 2 The Basics of PredictorCorrector Path Following 3 3 Aspects of Implementations 7 4 Applications 15 5 PiecewiseLinear Methods 34 6 Complexity 41 7 Available Software 44 References 48 1. Introduction Continuation, embedding or homotopy methods have long served as useful ..."
Abstract

Cited by 95 (6 self)
 Add to MetaCart
CONTENTS 1 Introduction 1 2 The Basics of PredictorCorrector Path Following 3 3 Aspects of Implementations 7 4 Applications 15 5 PiecewiseLinear Methods 34 6 Complexity 41 7 Available Software 44 References 48 1. Introduction Continuation, embedding or homotopy methods have long served as useful theoretical tools in modern mathematics. Their use can be traced back at least to such venerated works as those of Poincar'e (18811886), Klein (1882 1883) and Bernstein (1910). Leray and Schauder (1934) refined the tool and presented it as a global result in topology, viz., the homotopy invariance of degree. The use of deformations to solve nonlinear systems of equations Partially supported by the National Science Foundation via grant # DMS9104058 y Preprint, Colorado State University, August 2 E. Allgower and K. Georg may be traced back at least to Lahaye (1934). The classical embedding methods were the
An evaluation of implicit surface tilers
 Computer Graphics and Applications, IEEE
, 1993
"... ..."
A Unified Framework for Modelbased Clustering
 Journal of Machine Learning Research
, 2003
"... Modelbased clustering techniques have been widely used and have shown promising results in many applications involving complex data. This paper presents a unified framework for probabilistic modelbased clustering based on a bipartite graph view of data and models that highlights the commonaliti ..."
Abstract

Cited by 74 (7 self)
 Add to MetaCart
(Show Context)
Modelbased clustering techniques have been widely used and have shown promising results in many applications involving complex data. This paper presents a unified framework for probabilistic modelbased clustering based on a bipartite graph view of data and models that highlights the commonalities and differences among existing modelbased clustering algorithms. In this view, clusters are represented as probabilistic models in a model space that is conceptually separate from the data space. For partitional clustering, the view is conceptually similar to the ExpectationMaximization (EM) algorithm. For hierarchical clustering, the graphbased view helps to visualize critical/important distinctions between similaritybased approaches and modelbased approaches.
Numerical Homotopies to compute generic Points on positive dimensional Algebraic Sets
 Journal of Complexity
, 1999
"... Many applications modeled by polynomial systems have positive dimensional solution components (e.g., the path synthesis problems for fourbar mechanisms) that are challenging to compute numerically by homotopy continuation methods. A procedure of A. Sommese and C. Wampler consists in slicing the com ..."
Abstract

Cited by 68 (30 self)
 Add to MetaCart
(Show Context)
Many applications modeled by polynomial systems have positive dimensional solution components (e.g., the path synthesis problems for fourbar mechanisms) that are challenging to compute numerically by homotopy continuation methods. A procedure of A. Sommese and C. Wampler consists in slicing the components with linear subspaces in general position to obtain generic points of the components as the isolated solutions of an auxiliary system. Since this requires the solution of a number of larger overdetermined systems, the procedure is computationally expensive and also wasteful because many solution paths diverge. In this article an embedding of the original polynomial system is presented, which leads to a sequence of homotopies, with solution paths leading to generic points of all components as the isolated solutions of an auxiliary system. The new procedure significantly reduces the number of paths to solutions that need to be followed. This approach has been implemented and applied to...
Estimating Static Models of Strategic Interactions
, 2006
"... We propose a method for estimating static games of incomplete information. A static game is a generalization of a discrete choice model, such as a multinomial logit or probit, which allows the actions of a group of agents to be interdependent. Unlike most earlier work, the method we propose is semip ..."
Abstract

Cited by 65 (12 self)
 Add to MetaCart
We propose a method for estimating static games of incomplete information. A static game is a generalization of a discrete choice model, such as a multinomial logit or probit, which allows the actions of a group of agents to be interdependent. Unlike most earlier work, the method we propose is semiparametric and does not require the covariates to lie in a discrete set. While the estimator we propose is quite flexible, we demonstrate that in most cases it can be easily implemented using standard statistical packages such as STATA. We also propose an algorithm for simulating the model which finds all equilibria to the game. As an application of our estimator, we study recommendations for high technology stocks between 19982003. We find that strategic motives, typically ignored in the empirical literature, appear to be an important consideration in the recommendations submitted by equity analysts.
TimeStepping for ThreeDimensional Rigid Body Dynamics
, 1998
"... This paper considers a wide number of timestepping methods, and discusses their implications for convergence theory and the nature of the limiting solutions. ..."
Abstract

Cited by 59 (18 self)
 Add to MetaCart
(Show Context)
This paper considers a wide number of timestepping methods, and discusses their implications for convergence theory and the nature of the limiting solutions.
Polygonization of NonManifold Implicit Surfaces
, 1995
"... A method is presented to broaden implicit surface modeling. The implicit surfaces usually employed in computer graphics are two dimensional manifolds because they are defined by realvalued functions that impose a binary regionalization of space (i.e., an inside and an outside). When tiled, these su ..."
Abstract

Cited by 50 (0 self)
 Add to MetaCart
A method is presented to broaden implicit surface modeling. The implicit surfaces usually employed in computer graphics are two dimensional manifolds because they are defined by realvalued functions that impose a binary regionalization of space (i.e., an inside and an outside). When tiled, these surfaces yield edges of degree two. The new method allows the definition of implicit surfaces with boundaries (i.e., edges of degree one) and intersections (i.e., edges of degree three or more). These nonmanifold implicit surfaces are defined by a multiple regionalization of space. The definition includes a list of those pairs of regions whose separating surface is of interest. Also presented is an implementation that converts a nonmanifold implicit surface definition into a collection of polygons. Although following conventional implicit surface polygonization, there are significant differences that are described in detail. Several example surfaces are defined and polygonized. CR Categories and Subject Descriptors: I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling  Curve, Surface, Solid, and Object Representations. Additional Keywords and Phrases: Implicit Surface, NonManifold, Polygonization. 1