Results 1  10
of
12
Analog VLSI Stochastic Perturbative Learning Architectures
 J. Analog Integrated Circuits and Signal Processing
, 1997
"... We present analog VLSI neuromorphic architectures for a general class of learning tasks, which include supervised learning, reinforcement learning, and temporal di erence learning. The presented architectures are parallel, cellular, sparse in global interconnects, distributed in representation, and ..."
Abstract

Cited by 15 (7 self)
 Add to MetaCart
We present analog VLSI neuromorphic architectures for a general class of learning tasks, which include supervised learning, reinforcement learning, and temporal di erence learning. The presented architectures are parallel, cellular, sparse in global interconnects, distributed in representation, and robust to noise and mismatches in the implementation. They use a parallel stochastic perturbation technique to estimate the e ect of weight changes on network outputs, rather than calculating derivatives based on a model of the network. This \modelfree " technique avoids errors due to mismatchesinthephysical implementation of the network, and more generally allows to train networks of which the exact characteristics and structure are not known. With additional mechanisms of reinforcement learning, networks of fairly general structure are trained e ectively from an arbitrarily supplied reward signal. No prior assumptions are required on the structure of the network nor on the speci cs of the desired network response.
Learning Curves for Stochastic Gradient Descent in Linear Feedforward Networks
, 2004
"... Gradientfollowing learning methods can encounter problems of implementation in many applications, and stochastic variants are sometimes used to overcome these difficulties. We analyze three online training methods used with a linear perceptron: direct gradient descent, node perturbation, and we ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
Gradientfollowing learning methods can encounter problems of implementation in many applications, and stochastic variants are sometimes used to overcome these difficulties. We analyze three online training methods used with a linear perceptron: direct gradient descent, node perturbation, and weight perturbation. Learning speed is defined as the rate of exponential decay in the learning curves. When the scalar parameter that controls the size of weight updates is chosen to maximize learning speed, node perturbation is slower than direct gradient descent by a factor equal to the number of output units; weight perturbation is slower still by an additional factor equal to the number of input units. Parallel perturbation allows faster learning than sequential perturbation, by a factor that does not depend on network size. We also characterize how uncertainty in quantities used in the stochastic updates affects the learning curves. This study suggests that in practice, weight perturbation may be slow for large networks, and node perturbation can have performance comparable to that of direct gradient descent when there are few output units. However, these statements depend on the specifics of the learning problem, such as the input distribution and the target function, and are not universally applicable.
Analog VLSI neural network with digital perturbative learning
 IEEE Transactions on Circuits and Systems II : Analog and Digital Signal Processing
, 2002
"... [2] H. Fan, “A structural view of asymptotic convergence speed of adaptive ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
[2] H. Fan, “A structural view of asymptotic convergence speed of adaptive
HighSpeed, ModelFree Adaptive Control Using Parallel Synchronous Detection ABSTRACT
"... A VLSI implementation of an adaptive controller performing gradient descent optimization of external performance metrics using parallel synchronous detection is presented. Realtime modelfree gradient estimation is done by perturbation of the metrics ’ control parameters with narrowband determinis ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
A VLSI implementation of an adaptive controller performing gradient descent optimization of external performance metrics using parallel synchronous detection is presented. Realtime modelfree gradient estimation is done by perturbation of the metrics ’ control parameters with narrowband deterministic dithers resulting in fast adaptation and robust performance. A fully translinear design has been employed for the architecture, making the controller operation scalable within a very wide range of frequencies and control bandwidths, and, therefore customizable for a variety of systems and applications. Experimental results from a SiGe BiCMOS implementation are provided demonstrating the broadband and highspeed performance of the controller.
Simultaneous Perturbation Learning Rule for Recurrent Neural Networks and Its FPGA
"... Implementation ..."
Overview of Neural Hardware
, 1995
"... Neural hardware has undergone rapid development during the last few years. This paper presents an overview of neural hardware projects within industries and academia. It describes digital, analog, and hybrid neurochips and accelerator boards as well as largescale neurocomputers built from general p ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Neural hardware has undergone rapid development during the last few years. This paper presents an overview of neural hardware projects within industries and academia. It describes digital, analog, and hybrid neurochips and accelerator boards as well as largescale neurocomputers built from general purpose processors and communication elements. Special attention is given to multiprocessor projects that focus on scalability, flexibility, and adaptivity of the design and thus seem suitable for brainstyle (cognitive) processing. The sources used for this overview are taken from journal papers, conference proceedings, data sheets, and ftpsites and present an uptodate overview of current stateoftheart neural hardware implementations. 1 Categorization of neural hardware This paper presents an overview of timemultiplexed hardware designs, some of which are already commercially available, others representing design studies being carried out by research groups. A large number of design ...
Palmo: a novel pulsed based signal processing technique for programmable mixedsignal VLSI
, 1998
"... In this thesis a new signal processing technique is presented. This technique exploits the use of pulses as the signalling mechanism. This Palmo 1 signalling method applied to signal processing is novel, combining the advantages of both digital and analogue techniques. Pulsed signals are robust, i ..."
Abstract
 Add to MetaCart
In this thesis a new signal processing technique is presented. This technique exploits the use of pulses as the signalling mechanism. This Palmo 1 signalling method applied to signal processing is novel, combining the advantages of both digital and analogue techniques. Pulsed signals are robust, inherently lowpower, easily regenerated, and easily distributed across and between chips. The Palmo cells used to perform analogue operations on the pulsed signals are compact, fast, simple and programmable.
unknown title
, 1999
"... Adaptive optics based on analog parallel stochastic optimization: analysis and experimental demonstration ..."
Abstract
 Add to MetaCart
Adaptive optics based on analog parallel stochastic optimization: analysis and experimental demonstration
Ability Using Simultaneous Perturbation
"... Abstract—Hardware realization is very important when considering wider applications of neural networks (NNs). In particular, hardware NNs with a learning ability are intriguing. In these networks, the learning scheme is of much interest, with the backpropagation method being widely used. A gradient ..."
Abstract
 Add to MetaCart
Abstract—Hardware realization is very important when considering wider applications of neural networks (NNs). In particular, hardware NNs with a learning ability are intriguing. In these networks, the learning scheme is of much interest, with the backpropagation method being widely used. A gradient type of learning rule is not easy to realize in an electronic system, since calculation of the gradients for all weights in the network is very difficult. More suitable is the simultaneous perturbation method, since the learning rule requires only forward operations of the network to modify weights unlike the backpropagation method. In addition, pulse density NN systems have some promising properties, as they are robust to noisy situations and can handle analog quantities based on the digital circuits. In this paper, we describe a fieldprogrammable gate array realization of a pulse density NN using the simultaneous perturbation method as the learning scheme. We confirm the viability of the design and the operation of the actual NN system through some examples. Index Terms—Fieldprogrammable gate array (FPGA), learning ability, neural networks (NNs), pulse density, simultaneous perturbation. I.
VLSI DELTASIGMA CELLULAR NEURAL NETWORK FOR ANALOG RANDOM VECTOR GENERATION
"... We present a cellular neural network architecture for parallel analog random vector generation, including experimental results from an analog VLSI prototype with 64 channels. Nearestneighbor coupling between cells produces parallel channels of uniformly distributed random analog values, with statis ..."
Abstract
 Add to MetaCart
We present a cellular neural network architecture for parallel analog random vector generation, including experimental results from an analog VLSI prototype with 64 channels. Nearestneighbor coupling between cells produces parallel channels of uniformly distributed random analog values, with statistics that are truly uncorrelated across channels and over time. The cell for each random channel emulates an integrating nonlinearity essentially implementing a deltasigma modulator, and measures 100 m 120 min2 m CMOS technology. Applications include analog encryption and secure communications, analog builtin selftest, stochastic neural networks, and simulated annealing optimization and learning. 1.