Results 1 
4 of
4
A General Framework for Adaptive Processing of Data Structures
 IEEE TRANSACTIONS ON NEURAL NETWORKS
, 1998
"... A structured organization of information is typically required by symbolic processing. On the other hand, most connectionist models assume that data are organized according to relatively poor structures, like arrays or sequences. The framework described in this paper is an attempt to unify adaptive ..."
Abstract

Cited by 117 (46 self)
 Add to MetaCart
A structured organization of information is typically required by symbolic processing. On the other hand, most connectionist models assume that data are organized according to relatively poor structures, like arrays or sequences. The framework described in this paper is an attempt to unify adaptive models like artificial neural nets and belief nets for the problem of processing structured information. In particular, relations between data variables are expressed by directed acyclic graphs, where both numerical and categorical values coexist. The general framework proposed in this paper can be regarded as an extension of both recurrent neural networks and hidden Markov models to the case of acyclic graphs. In particular we study the supervised learning problem as the problem of learning transductions from an input structured space to an output structured space, where transductions are assumed to admit a recursive hidden statespace representation. We introduce a graphical formalism for r...
A Position Paper on Statistical Inference Techniques Which Integrate Neural Network and Bayesian Network Models
 In International Conference on Neural Networks (ICNN97
, 1997
"... Some statistical methods which have been shown to have direct neural network analogs are surveyed here; we discuss sampling, optimization, and representation methods which make them feasible when applied in conjunction with, or in place of, neural networks. We present the foremost of these, the Gibb ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
Some statistical methods which have been shown to have direct neural network analogs are surveyed here; we discuss sampling, optimization, and representation methods which make them feasible when applied in conjunction with, or in place of, neural networks. We present the foremost of these, the Gibbs sampler, both in its successful role as a convergence heuristic derived from statistical physics and under its probabilistic learning interpretation. We then review various manifestations of Gibbs sampling in Bayesian learning; its relation to "traditional" simulated annealing; specializations and instances such as EM; and its application as a model construction technique for the Bayesian network formalism. Next, we examine the ramifications of recent advances in Markov chain Monte Carlo methods for learning by backpropagation. Finally, we consider how the Bayesian network formalism informs the causal reasoning interpretation of some neural networks, and how it prescribes optimizations for efficient random sampling in Bayesian learning applications.
Experiments of fast learning with High Order Boltzmann Machines
"... : This work reports the results obtained with the application of High Order Boltzmann Machines without hidden units to construct classifiers for some problems that represent different learning paradigms. The Boltzmann Machine weight updating algorithm remains the same even when some of the units can ..."
Abstract
 Add to MetaCart
: This work reports the results obtained with the application of High Order Boltzmann Machines without hidden units to construct classifiers for some problems that represent different learning paradigms. The Boltzmann Machine weight updating algorithm remains the same even when some of the units can take values in a discrete set or in a continuous interval. The absence of hidden units and the restriction to classification problems allows for the estimation of the connection statistics, without the computational cost involved in the application of simulated annealing. In this setting, the learning process can be sped up several orders of magnitude with no appreciable loss of quality of the results obtained. Keywords: Neural Networks, Boltzmann Machines, High Order networks, classification problems. 0 Introduction The Boltzmann Machine is a classical neural network architecture [1, 2] that has been relegated from practical application due to its computational cost and the difficulty to ...
The Synchronous Boltzmann Machine for Learning and Hardcombinatoric Search
"... In neural network modelling asynchronous Boltzmann machines are widely regarded as a slow learners because their operative mode is not parallel. This is because that part of the theory related to energy minimisation only permits one unit at a time to update its state. Learning with synchronous Boltz ..."
Abstract
 Add to MetaCart
In neural network modelling asynchronous Boltzmann machines are widely regarded as a slow learners because their operative mode is not parallel. This is because that part of the theory related to energy minimisation only permits one unit at a time to update its state. Learning with synchronous Boltzmann machines provides an attractive alternative provided that one can offer a suitable theoretical framework. The dynamics of the synchronous Boltzmann machine were first studied by W. A. Little [Little 1974]. In contrast to the extensive literature on asynchronous Boltzmann machines and simulated annealing, surprisingly little has been written on the synchronous model. There are some attempts to use the synchronous model [Azencott 1990], [Apolloni 1991], unfortunately they do not offer formal proofs of their proposals. The principal contribution of the present study is to provide a theoretical framework, leading to a learning algorithm able to learn an environmentally observed Markov process using a synchronous Boltzmann machine. Thus substantial effort is devoted to an explanation of how the mathematical theory of first order Markov processes provides a theoretical guarantee for the existence of an equilibrium distribution for any Boltzmann machine and to illustrating methods whereby such distributions may be calculated, even in the case of nonsymmetric weight matrices. A second potential use of a synchronous Boltzmann machine is in hard combinatoric search. The main difficulty is that, for a given search problem, one has to provide a mapping from the problem specification to the weights and thresholds of the target machine, in such a way that minimisation of the associated network Lyapunov function corresponds to optimisation in the search problem. This has often been do...