Results 1  10
of
122
Waveletbased statistical signal processing using hidden Markov models
 IEEE Transactions on Signal Processing
, 1998
"... Abstract — Waveletbased statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly Gaussian. These models are unrealistic for many realworld signals. In this paper, we develop a new framework for statistical signal pr ..."
Abstract

Cited by 325 (52 self)
 Add to MetaCart
Abstract — Waveletbased statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly Gaussian. These models are unrealistic for many realworld signals. In this paper, we develop a new framework for statistical signal processing based on waveletdomain hidden Markov models (HMM’s) that concisely models the statistical dependencies and nonGaussian statistics encountered in realworld signals. Waveletdomain HMM’s are designed with the intrinsic properties of the wavelet transform in mind and provide powerful, yet tractable, probabilistic signal models. Efficient expectation maximization algorithms are developed for fitting the HMM’s to observational signal data. The new framework is suitable for a wide range of applications, including signal estimation, detection, classification, prediction, and even synthesis. To demonstrate the utility of waveletdomain HMM’s, we develop novel algorithms for signal denoising, classification, and detection. Index Terms — Hidden Markov model, probabilistic graph, wavelets.
Constructive Algorithms for Structure Learning in Feedforward Neural Networks for Regression Problems
 IEEE Transactions on Neural Networks
, 1997
"... In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole ..."
Abstract

Cited by 66 (2 self)
 Add to MetaCart
In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole problem as a state space search, we first describe the general issues in constructive algorithms, with special emphasis on the search strategy. A taxonomy, based on the differences in the state transition mapping, the training algorithm and the network architecture, is then presented. Keywords Constructive algorithm, structure learning, state space search, dynamic node creation, projection pursuit regression, cascadecorrelation, resourceallocating network, group method of data handling. I. Introduction A. Problems with Fixed Size Networks I N recent years, many neural network models have been proposed for pattern classification, function approximation and regression problems. Among...
New Neural Transfer Functions
 Neural Computing Surveys
, 1997
"... In this article advantages of various neural transfer functions are discussed. ..."
Abstract

Cited by 35 (28 self)
 Add to MetaCart
In this article advantages of various neural transfer functions are discussed.
Universal approximation using incremental constructive feedforward networks with . . .
 IEEE TRANSACTIONS ON NEURAL NETWORKS
, 2005
"... According to conventional neural network theories, singlehiddenlayer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implem ..."
Abstract

Cited by 33 (11 self)
 Add to MetaCart
According to conventional neural network theories, singlehiddenlayer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implementations, tuning all the parameters of the networks may cause learning complicated and inefficient, and it may be difficult to train networks with nondifferential activation functions such as threshold networks. Unlike conventional neural network theories, this paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer. In such SLFNs implementations, the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions X and the activation functions for RBF nodes can be any integrable piecewise continuous functions X and @ A aH. The proposed incremental method is efficient not only for SFLNs with continuous (including nondifferentiable) activation functions but also for SLFNs with piecewise continuous (such as threshold) activation functions. Compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters.
Regularisation in the Selection of Radial Basis Function Centres
 NEURAL COMPUTATION
, 1995
"... Subset selection and regularisation are two well known techniques which can improve the generalisation performance of nonparametric linear regression estimators, such as radial basis function networks. This paper examines regularised forward selection (RFS)  a combination of forward subset selecti ..."
Abstract

Cited by 30 (7 self)
 Add to MetaCart
Subset selection and regularisation are two well known techniques which can improve the generalisation performance of nonparametric linear regression estimators, such as radial basis function networks. This paper examines regularised forward selection (RFS)  a combination of forward subset selection and zeroorder regularisation. An efficient implementation of RFS into which either delete1 or generalised crossvalidation can be incorporated and a reestimation formula for the regularisation parameter are also discussed. Simulation studies are presented which demonstrate improved generalisation performance due to regularisation in the forward selection of radial basis function centres.
Learning without Local Minima in Radial Basis Function Networks
 IEEE Transactions on Neural Networks
, 1995
"... Learning from examples plays a central role in artificial neural networks (ANN). However, the success of many learning schemes is not guaranteed, since algorithms like Backpropagation (BP) may get stuck in local minima, thus providing suboptimal solutions. For feedforward networks, the theoretical ..."
Abstract

Cited by 25 (6 self)
 Add to MetaCart
Learning from examples plays a central role in artificial neural networks (ANN). However, the success of many learning schemes is not guaranteed, since algorithms like Backpropagation (BP) may get stuck in local minima, thus providing suboptimal solutions. For feedforward networks, the theoretical results reported in [5,6,15,20] show that optimal learning can be achieved provided that certain conditions on the network and the learning environment are met. A similar investigation is put forward in this paper for the case of networks using radial basis functions (RBF) [10,14]. The analysis proposed in [6] is extended naturally under the assumption that the patterns of the learning environment are separable by hyperspheres. In that case, we prove that the attached cost function is local minima free with respect to all the weights. This provides us with some theoretical foundations for a massive application of RBF in pattern recognition. Keywords Backpropagation, multilayered networks...
Combined Genetic Algorithm Optimization and Regularized Orthogonal Least Squares Learning for Radial Basis Function Networks
, 1999
"... The paper presents a twolevel learning method for radial basis function (RBF) networks. A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimiz ..."
Abstract

Cited by 25 (6 self)
 Add to MetaCart
The paper presents a twolevel learning method for radial basis function (RBF) networks. A regularized orthogonal least squares (ROLS) algorithm is employed at the lower level to construct RBF networks while the two key learning parameters, the regularization parameter and the RBF width, are optimized using a genetic algorithm (GA) at the upper level. Nonlinear time series modeling and prediction is used as an example to demonstrate the effectiveness of this hierarchical learning approach.
Constructive Feedforward Neural Networks for Regression Problems: A Survey
, 1995
"... In this paper, we review the procedures for constructing feedforward neural networks in regression problems. While standard backpropagation performs gradient descent only in the weight space of a network with fixed topology, constructive procedures start with a small network and then grow additiona ..."
Abstract

Cited by 21 (0 self)
 Add to MetaCart
In this paper, we review the procedures for constructing feedforward neural networks in regression problems. While standard backpropagation performs gradient descent only in the weight space of a network with fixed topology, constructive procedures start with a small network and then grow additional hidden units and weights until a satisfactory solution is found. The constructive procedures are categorized according to the resultant network architecture and the learning algorithm for the network weights. The Hong Kong University of Science & Technology Technical Report Series Department of Computer Science 1 Introduction In recent years, many neural network models have been proposed for pattern classification, function approximation and regression problems. Among them, the class of multilayer feedforward networks is perhaps the most popular. Standard backpropagation performs gradient descent only in the weight space of a network with fixed topology; this approach is analogous to ...
Identification and Control of Nonlinear Systems Using Neural Network Models: Design and Stability Analysis
 ELECTRICAL ENGINEERING—SYSTEMS REP
, 1991
"... The feasibility of applying neural network learning techniques in problems of system identification and control has been demonstrated through several empirical studies. These studies are based for the most part on gradient techniques for deriving parameter adjustment laws. While such schemes perf ..."
Abstract

Cited by 21 (2 self)
 Add to MetaCart
The feasibility of applying neural network learning techniques in problems of system identification and control has been demonstrated through several empirical studies. These studies are based for the most part on gradient techniques for deriving parameter adjustment laws. While such schemes perform well in many cases, in general, problems arise in attempting to prove stability of the overall system, or convergence of the output error to zero. This paper presents a stability theory approach to synthesizing and analyzing identification and control schemes for nonlinear dynamical systems using neural network models. The nonlinearities of the dynamical system are assumed to be unknown and are modelled by neural network architectures. Multilayer networks with sigmoidal activation functions and radial basis function networks are the two types of neural network models that are considered. These static network architectures are combined with dynamical elements, in the form of stable filters, to construct a type of recurrent network configuration which is shown to be capable of approximating a large class of dynamical systems.
Design of Neural Network Filters
 Electronics Institute, Technical University of Denmark
, 1993
"... Emnet for n rv rende licentiatafhandling er design af neurale netv rks ltre. Filtre baseret pa neurale netv rk kan ses som udvidelser af det klassiske line re adaptive lter rettet mod modellering af uline re sammenh nge. Hovedv gten l gges pa en neural netv rks implementering af den ikkerekursive, ..."
Abstract

Cited by 21 (12 self)
 Add to MetaCart
Emnet for n rv rende licentiatafhandling er design af neurale netv rks ltre. Filtre baseret pa neurale netv rk kan ses som udvidelser af det klassiske line re adaptive lter rettet mod modellering af uline re sammenh nge. Hovedv gten l gges pa en neural netv rks implementering af den ikkerekursive, uline re adaptive model med additiv st j. Formalet er at klarl gge en r kke faser forbundet med design af neural netv rks arkitekturer med henblik pa at udf re forskellige \blackbox " modellerings opgaver sa som: System identi kation, invers modellering og pr diktion af tidsserier. De v senligste bidrag omfatter: Formulering af en neural netv rks baseret kanonisk lter repr sentation, der danner baggrund for udvikling af et arkitektur klassi kationssystem. I hovedsagen drejer det sig om en skelnen mellem globale og lokale modeller. Dette leder til at en r kke kendte neurale netv rks arkitekturer kan klassi ceres, og yderligere abnes der mulighed for udvikling af helt nye strukturer. I denne sammenh ng ndes en gennemgang af en r kke velkendte arkitekturer. I s rdeleshed l gges der v gt pa behandlingen af multilags perceptron neural netv rket.