Results 1  10
of
16
Analog VLSI Stochastic Perturbative Learning Architectures
 J. Analog Integrated Circuits and Signal Processing
, 1997
"... We present analog VLSI neuromorphic architectures for a general class of learning tasks, which include supervised learning, reinforcement learning, and temporal di erence learning. The presented architectures are parallel, cellular, sparse in global interconnects, distributed in representation, and ..."
Abstract

Cited by 16 (7 self)
 Add to MetaCart
We present analog VLSI neuromorphic architectures for a general class of learning tasks, which include supervised learning, reinforcement learning, and temporal di erence learning. The presented architectures are parallel, cellular, sparse in global interconnects, distributed in representation, and robust to noise and mismatches in the implementation. They use a parallel stochastic perturbation technique to estimate the e ect of weight changes on network outputs, rather than calculating derivatives based on a model of the network. This \modelfree &quot; technique avoids errors due to mismatchesinthephysical implementation of the network, and more generally allows to train networks of which the exact characteristics and structure are not known. With additional mechanisms of reinforcement learning, networks of fairly general structure are trained e ectively from an arbitrarily supplied reward signal. No prior assumptions are required on the structure of the network nor on the speci cs of the desired network response.
Learning Curves for Stochastic Gradient Descent in Linear Feedforward Networks
, 2004
"... Gradientfollowing learning methods can encounter problems of implementation in many applications, and stochastic variants are sometimes used to overcome these difficulties. We analyze three online training methods used with a linear perceptron: direct gradient descent, node perturbation, and we ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
Gradientfollowing learning methods can encounter problems of implementation in many applications, and stochastic variants are sometimes used to overcome these difficulties. We analyze three online training methods used with a linear perceptron: direct gradient descent, node perturbation, and weight perturbation. Learning speed is defined as the rate of exponential decay in the learning curves. When the scalar parameter that controls the size of weight updates is chosen to maximize learning speed, node perturbation is slower than direct gradient descent by a factor equal to the number of output units; weight perturbation is slower still by an additional factor equal to the number of input units. Parallel perturbation allows faster learning than sequential perturbation, by a factor that does not depend on network size. We also characterize how uncertainty in quantities used in the stochastic updates affects the learning curves. This study suggests that in practice, weight perturbation may be slow for large networks, and node perturbation can have performance comparable to that of direct gradient descent when there are few output units. However, these statements depend on the specifics of the learning problem, such as the input distribution and the target function, and are not universally applicable.
Simultaneous Perturbation Learning Rule for Recurrent Neural Networks and Its FPGA
"... Implementation ..."
Analog VLSI neural network with digital perturbative learning
 IEEE Transactions on Circuits and Systems II : Analog and Digital Signal Processing
, 2002
"... [2] H. Fan, “A structural view of asymptotic convergence speed of adaptive ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
[2] H. Fan, “A structural view of asymptotic convergence speed of adaptive
HighSpeed, ModelFree Adaptive Control Using Parallel Synchronous Detection ABSTRACT
"... A VLSI implementation of an adaptive controller performing gradient descent optimization of external performance metrics using parallel synchronous detection is presented. Realtime modelfree gradient estimation is done by perturbation of the metrics ’ control parameters with narrowband determinis ..."
Abstract

Cited by 3 (3 self)
 Add to MetaCart
A VLSI implementation of an adaptive controller performing gradient descent optimization of external performance metrics using parallel synchronous detection is presented. Realtime modelfree gradient estimation is done by perturbation of the metrics ’ control parameters with narrowband deterministic dithers resulting in fast adaptation and robust performance. A fully translinear design has been employed for the architecture, making the controller operation scalable within a very wide range of frequencies and control bandwidths, and, therefore customizable for a variety of systems and applications. Experimental results from a SiGe BiCMOS implementation are provided demonstrating the broadband and highspeed performance of the controller.
Overview of Neural Hardware
, 1995
"... Neural hardware has undergone rapid development during the last few years. This paper presents an overview of neural hardware projects within industries and academia. It describes digital, analog, and hybrid neurochips and accelerator boards as well as largescale neurocomputers built from general p ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Neural hardware has undergone rapid development during the last few years. This paper presents an overview of neural hardware projects within industries and academia. It describes digital, analog, and hybrid neurochips and accelerator boards as well as largescale neurocomputers built from general purpose processors and communication elements. Special attention is given to multiprocessor projects that focus on scalability, flexibility, and adaptivity of the design and thus seem suitable for brainstyle (cognitive) processing. The sources used for this overview are taken from journal papers, conference proceedings, data sheets, and ftpsites and present an uptodate overview of current stateoftheart neural hardware implementations. 1 Categorization of neural hardware This paper presents an overview of timemultiplexed hardware designs, some of which are already commercially available, others representing design studies being carried out by research groups. A large number of design ...
Deltasigma cellular automata for analog VLSI random vector generation
 IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, Vol.46, No.3
, 1999
"... Abstract—We present a class of analog cellular automata for parallel analog random vector generation, including theory on the randomness properties, scalable parallel very large scale integration (VLSI) architectures, and experimental results from an analog VLSI prototype with 64 channels. Linear co ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Abstract—We present a class of analog cellular automata for parallel analog random vector generation, including theory on the randomness properties, scalable parallel very large scale integration (VLSI) architectures, and experimental results from an analog VLSI prototype with 64 channels. Linear congruential coupling between cells produces parallel channels of uniformly distributed random analog values, with statistics that are uncorrelated both across channels and over time. The cell for each random channel essentially implements a switchedcapacitor delta–sigma modulator, and measures 100 m 2 120 min2 m CMOS technology. The 64 cells are connected as a MASH cascade in a chain or ring topology on a twodimensional (2D) grid, and can be rearranged for use in various VLSI applications that require a parallel supply of random analog vectors, such as analog encryption and secure communications, analog builtin selftest, stochastic neural networks, and simulated annealing optimization and learning. Index Terms—Random generation, noise, delta–sigma modulation, cellular automata, analog VLSI, neural networks, switchedcapacitor circuits.
VLSI DELTASIGMA CELLULAR NEURAL NETWORK FOR ANALOG RANDOM VECTOR GENERATION
"... We present a cellular neural network architecture for parallel analog random vector generation, including experimental results from an analog VLSI prototype with 64 channels. Nearestneighbor coupling between cells produces parallel channels of uniformly distributed random analog values, with statis ..."
Abstract
 Add to MetaCart
We present a cellular neural network architecture for parallel analog random vector generation, including experimental results from an analog VLSI prototype with 64 channels. Nearestneighbor coupling between cells produces parallel channels of uniformly distributed random analog values, with statistics that are truly uncorrelated across channels and over time. The cell for each random channel emulates an integrating nonlinearity essentially implementing a deltasigma modulator, and measures 100 m 120 min2 m CMOS technology. Applications include analog encryption and secure communications, analog builtin selftest, stochastic neural networks, and simulated annealing optimization and learning. 1.
AdOpt: Analog VLSI Stochastic Optimization for Adaptive Optics Marc Cohen
"... Phase distortion in wavefront propagation is one of the key problems in optical imaging and laser optics applications. We present a hybrid VLSI and optical system for realtime adaptive phase distortion compensation. The system operates “modelfree”, independent of the specifics of the distorting op ..."
Abstract
 Add to MetaCart
Phase distortion in wavefront propagation is one of the key problems in optical imaging and laser optics applications. We present a hybrid VLSI and optical system for realtime adaptive phase distortion compensation. The system operates “modelfree”, independent of the specifics of the distorting optical medium and the compensation control elements. Our VLSI system implements stochastic parallel perturbative gradient descent/ascent so that we achieve fast optimization of the chosen performance metric to achieve realtime compensation. We include experimental results of the hybrid VLSIoptical system demonstrating successful operation for a laserbeam focusing/defocusing task.