Results 1  10
of
11
A General Framework for Adaptive Processing of Data Structures
 IEEE TRANSACTIONS ON NEURAL NETWORKS
, 1998
"... A structured organization of information is typically required by symbolic processing. On the other hand, most connectionist models assume that data are organized according to relatively poor structures, like arrays or sequences. The framework described in this paper is an attempt to unify adaptive ..."
Abstract

Cited by 119 (48 self)
 Add to MetaCart
A structured organization of information is typically required by symbolic processing. On the other hand, most connectionist models assume that data are organized according to relatively poor structures, like arrays or sequences. The framework described in this paper is an attempt to unify adaptive models like artificial neural nets and belief nets for the problem of processing structured information. In particular, relations between data variables are expressed by directed acyclic graphs, where both numerical and categorical values coexist. The general framework proposed in this paper can be regarded as an extension of both recurrent neural networks and hidden Markov models to the case of acyclic graphs. In particular we study the supervised learning problem as the problem of learning transductions from an input structured space to an output structured space, where transductions are assumed to admit a recursive hidden statespace representation. We introduce a graphical formalism for r...
Approximating the Semantics of Logic Programs by Recurrent Neural Networks
"... In [18] we have shown how to construct a 3layered recurrent neural network that computes the fixed point of the meaning function TP of a given propositional logic program P, which corresponds to the computation of the semantics of P. In this article we consider the first order case. We define a no ..."
Abstract

Cited by 55 (9 self)
 Add to MetaCart
In [18] we have shown how to construct a 3layered recurrent neural network that computes the fixed point of the meaning function TP of a given propositional logic program P, which corresponds to the computation of the semantics of P. In this article we consider the first order case. We define a notion of approximation for interpretations and prove that there exists a 3layered feed forward neural network that approximates the calculation of TP for a given first order acyclic logic program P with an injective level mapping arbitrarily well. Extending the feed forward network by recurrent connections we obtain a recurrent neural network whose iteration approximates the fixed point of TP. This result is proven by taking advantage of the fact that for acyclic logic programs the function TP is a contraction mapping on a complete metric space defined by the interpretations of the program. Mapping this space to the metric space IR with Euclidean distance, a real valued function fP can be defined which corresponds to TP and is continuous as well as a contraction. Consequently it can be approximated by an appropriately chosen class of feed forward neural networks.
The Neural Network Pushdown Automaton: Model, Stack and Learning Simulations
, 1993
"... In order for neural networks to learn complex languages or grammars, they must have sufficient computational power or resources to recognize or generate such languages. Though many approaches to effectively utilizing the computational power of neural networks have been discussed, an obvious one is t ..."
Abstract

Cited by 17 (2 self)
 Add to MetaCart
In order for neural networks to learn complex languages or grammars, they must have sufficient computational power or resources to recognize or generate such languages. Though many approaches to effectively utilizing the computational power of neural networks have been discussed, an obvious one is to couple a recurrent neural network with an external stack memory in effect creating a neural network pushdown automata (NNPDA). This NNPDA generalizes the concept of a recurrent network so that the network becomes a more complex computing structure. This paper discusses in detail a NNPDA its construction, how it can be trained and how useful symbolic information can be extracted from the trained network. To effectively couple the external stack to the neural network, an optimization method is developed which uses an error function that connects the learning of the state automaton of the neural network to the learning of the operation of the external stack: push, pop, and nooperation. To minimize the error function using gradient descent learning, an analog stack is designed such that the action and storage of information in the stack are continuous. One interpretation of a continuous stack is the probabilistic storage of and action on data. After training on sample strings of an unknown source grammar, a quantization procedure extracts from the analog stack and neural network a discrete pushdown automata (PDA). Simulations show that in learning deterministic contextfree grammars the balanced parenthesis language, 1 n 0 n, and the deterministic Palindrome the extracted PDA is correct in the sense that it can correctly recognize unseen strings of arbitrary length. In addition, the extracted PDAs can be shown to be identical or equivalent to the PDAs of the source grammars which were used to generate the training strings.
On the implementation of frontiertoroot tree automata in recursive neural networks
 IEEE Transactions on Neural Networks
, 1999
"... Abstract — In this paper we explore the node complexity of recursive neural network implementations of frontiertoroot tree automata (FRA). Specifically, we show that an FRAO (Mealy version) with � states, � input–output labels, and maximum rank x can be implemented by a recursive neural network w ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Abstract — In this paper we explore the node complexity of recursive neural network implementations of frontiertoroot tree automata (FRA). Specifically, we show that an FRAO (Mealy version) with � states, � input–output labels, and maximum rank x can be implemented by a recursive neural network with
Recurrent Autoassociative Networks: Developing Distributed Representations Of Hierarchically Structured Sequences By Autoassociation
, 261
"... this reportedly improved the learning. And still another important contribution in this work was a method for representing recursive structures  by means of symbolic transformation of any tree structure into a binary tree, which can easily be transformed to a sequence. Those two operations are rev ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
this reportedly improved the learning. And still another important contribution in this work was a method for representing recursive structures  by means of symbolic transformation of any tree structure into a binary tree, which can easily be transformed to a sequence. Those two operations are reversible,
A Hierarchical Classification of FirstOrder Recurrent Neural Networks
 4TH INTERNATIONAL CONFERENCE ON LANGUAGE AND AUTOMATA THEORY AND APPLICATIONS, TRIER: GERMANY
, 2011
"... We provide a refined hierarchical classification of firstorder recurrent neural networks made up of McCulloch and Pitts cells. The classification is achieved by first proving the equivalence between the expressive powers of such neural networks and Muller automata, and then translating the Wadge cl ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
We provide a refined hierarchical classification of firstorder recurrent neural networks made up of McCulloch and Pitts cells. The classification is achieved by first proving the equivalence between the expressive powers of such neural networks and Muller automata, and then translating the Wadge classification theory from the automatatheoretic to the neural network context. The obtained hierarchical classification of neural networks consists of a decidable prewell ordering of width 2 and height ω ω, and a decidability procedure of this hierarchy is provided. Notably, this classification is shown to be intimately related to the attractive properties of the networks, and hence provides a new refined measurement of the computational power of these networks in terms of their attractive behaviours.
Equivalence in Knowledge Representation: Automata, Recurrent Neural Networks, and Dynamical Fuzzy Systems
 PROCEEDINGS OF THE IEEE
, 1999
"... Neurofuzzy systemsthe combination of artificial neural networks with fuzzy logichave become useful in many application domains. However, conventional neurofuzzy models usually need enhanced representation power for applications that require context and state (e.g., speech, time series prediction, ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Neurofuzzy systemsthe combination of artificial neural networks with fuzzy logichave become useful in many application domains. However, conventional neurofuzzy models usually need enhanced representation power for applications that require context and state (e.g., speech, time series prediction, control). Some of these applications can be readily modeled as finite state automata. Previously, it was proved that deterministic finite state automata (DFA) can be synthesized by or mapped into recurrent neural networks by directly programming the DFA structure into the weights of the neural network. Based on those results, a synthesis method is proposed for mapping fuzzy finite state automata (FFA) into recurrent neural networks. Furthermore, this mapping is suitable for direct implementation in very large scale integration (VLSI), i.e., the encoding of FFA as a generalization of the encoding of DFA in VLSI systems. The synthesis method requires FFA to undergo a transformation prior to being mapped into recurrent networks. The neurons are provided with an enriched functionality in order to accommodate a fuzzy representation of FFA states. This enriched neuron functionality also permits fuzzy parameters of FFA to be directly represented as parameters of the neural network. We also prove the stability of fuzzy finite state dynamics of the constructed neural networks for finite values of network weight and, through simulations, give empirical validation of the proofs. Hence, we prove various knowledge equivalence representations between neural and fuzzy systems and models of automata.
Adaptive Contextual Processing of Structured Data by Recursive Neural Networks: A Survey of Computational Properties
, 2007
"... ..."
Neural Networks Classifying Symbolic Data
, 2000
"... Introduction A very successful approach in machine learning are neural networks. They learn an unknown regularity between real vector spaces, given a finite set of examples. Hence they can be applied in different areas such as signal processing, speech recognition, or time series prediction, to nam ..."
Abstract
 Add to MetaCart
Introduction A very successful approach in machine learning are neural networks. They learn an unknown regularity between real vector spaces, given a finite set of examples. Hence they can be applied in different areas such as signal processing, speech recognition, or time series prediction, to name just a few. Furthermore, a mathematical foundation, more precisely, their universal approximation ability, the information theoretical learnability, and the complexity of learning are investigated. However, they suffer mainly in two aspects compared to symbolic approaches: The functions implemented by neural networks are not understandable for humans. Different approaches deal with rule extraction or rule insertion in order to integrate prior knowledge  but the methods are not yet satisfactory and inprinciple theoretical limitations exist [1, 7]. Furthermore, applications of networks for symbolic data require the encoding of the data in a finite dimensional vector space; often, str