Results 1  10
of
78
A Constructive Algorithm for Training Cooperative Neural Network Ensembles
 IEEE Transactions on Neural Networks
, 2003
"... This paper presents a constructive algorithm for training cooperative neuralnetwork ensembles (CNNEs). CNNE combines ensemble architecture design with cooperative training for individual neural networks (NNs) in ensembles. Unlike most previous studies on training ensembles, CNNE puts emphasis on bo ..."
Abstract

Cited by 44 (16 self)
 Add to MetaCart
(Show Context)
This paper presents a constructive algorithm for training cooperative neuralnetwork ensembles (CNNEs). CNNE combines ensemble architecture design with cooperative training for individual neural networks (NNs) in ensembles. Unlike most previous studies on training ensembles, CNNE puts emphasis on both accuracy and diversity among individual NNs in an ensemble. In order to maintain accuracy among individual NNs, the number of hidden nodes in individual NNs are also determined by a constructive approach. Incremental training based on negative correlation is used in CNNE to train individual NNs for different numbers of training epochs. The use of negative correlation learning and different training epochs for training individual NNs reflect CNNEs emphasis on diversity among individual NNs in an ensemble. CNNE has been tested extensively on a number of benchmark problems in machine learning and neural networks, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, soybean, and MackeyGlass time series prediction problems. The experimental results show that CNNE can produce NN ensembles with good generalization ability.
Online retrainable neural networks: improving the performance of neural networks in image analysis problems
 IEEE Trans. Neural Networks
, 2000
"... Abstract—A novel approach is presented in this paper for improving the performance of neuralnetwork classifiers in image recognition, segmentation, or coding applications, based on a retraining procedure at the user level. The procedure includes: 1) a training algorithm for adapting the network wei ..."
Abstract

Cited by 44 (30 self)
 Add to MetaCart
(Show Context)
Abstract—A novel approach is presented in this paper for improving the performance of neuralnetwork classifiers in image recognition, segmentation, or coding applications, based on a retraining procedure at the user level. The procedure includes: 1) a training algorithm for adapting the network weights to the current condition; 2) a maximum a posteriori (MAP) estimation procedure for optimally selecting the most representative data of the current environment as retraining data; and 3) a decision mechanism for determining when network retraining should be activated. The training algorithm takes into consideration both the former and the current network knowledge in order to achieve good generalization. The MAP estimation procedure models the network output as a Markov random field (MRF) and optimally selects the set of training inputs and corresponding desired outputs. Results are presented which illustrate the theoretical developments as well as the performance of the proposed approach in reallife experiments. Index Terms—Image analysis, MPEG4, neuralnetwork retraining, segmentation, weight adaptation.
Extraction of Rules from Artificial Neural Networks for Nonlinear Regression
, 2002
"... Neural networks have been successfully applied to solve a variety of application problems including classification and function approximation. They are especially useful as function approximators because they do not require prior knowledge of the input data distribution and they have been shown to b ..."
Abstract

Cited by 18 (0 self)
 Add to MetaCart
(Show Context)
Neural networks have been successfully applied to solve a variety of application problems including classification and function approximation. They are especially useful as function approximators because they do not require prior knowledge of the input data distribution and they have been shown to be universal approximators. In many applications, it is desirable to extract knowledge that can explain how the problems are solved by the networks. Most existing approaches have focused on extracting symbolic rules for classification. Few methods have been devised to extract rules from trained neural networks for regression. This article presents an approach for extracting rules from trained neural networks for regression. Each rule in the extracted rule set corresponds to a subregion of the input space and a linear function involving the relevant input attributes of the data approximates the network output for all data samples in this subregion. Extensive experimental results on 32 benchmark data sets demonstrate the effectiveness of the proposed approach in generating accurate regression rules.
V.K.Devabhaktuni, “Artificial neural networks for RF and Microwave Design: from theory to practice”,IEEE Trans.MTT,vol.51,pp.13391350,March 2003
"... Abstract—Neuralnetwork computational modules have recently gained recognition as an unconventional and useful tool for RF and microwave modeling and design. Neural networks can be trained to learn the behavior of passive/active components/circuits. A trained neural network can be used for highleve ..."
Abstract

Cited by 17 (0 self)
 Add to MetaCart
Abstract—Neuralnetwork computational modules have recently gained recognition as an unconventional and useful tool for RF and microwave modeling and design. Neural networks can be trained to learn the behavior of passive/active components/circuits. A trained neural network can be used for highlevel design, providing fast and accurate answers to the task it has learned. Neural networks are attractive alternatives to conventional methods such as numerical modeling methods, which could be computationally expensive, or analytical methods which could be difficult to obtain for new devices, or empirical modeling solutions whose range and accuracy may be limited. This tutorial describes fundamental concepts in this emerging area aimed at teaching RF/microwave engineers what neural networks are, why they are useful, when they can be used, and how to use them. Neuralnetwork structures and their training methods are described from the RF/microwave designer’s perspective. Electromagneticsbased training for passive component models and physicsbased training for active device models are illustrated. Circuit design and yield optimization using passive/active neural models are also presented. A multimedia slide presentation along with narrative audio clips is included in the electronic version of this paper. A hyperlink to the NeuroModeler demonstration software is provided to allow readers practice neuralnetworkbased design concepts. Index Terms—Computeraided design (CAD), design automation, modeling, neural networks, optimization, simulation.
MLPs (monolayer polynomials and multilayer perceptrons) for nonlinear modeling. JMLR, 3:1383–1398 (this issue
 Journal of Machine Learning Research
, 2003
"... This paper presents a model selection procedure which stresses the importance of the classic polynomial models as tools for evaluating the complexity of a given modeling problem, and for removing nonsignificant input variables. If the complexity of the problem makes a neural network necessary, the ..."
Abstract

Cited by 15 (0 self)
 Add to MetaCart
(Show Context)
This paper presents a model selection procedure which stresses the importance of the classic polynomial models as tools for evaluating the complexity of a given modeling problem, and for removing nonsignificant input variables. If the complexity of the problem makes a neural network necessary, the selection among neural candidates can be performed in two phases. In an additive phase, the most important one, candidate neural networks with an increasing number of hidden neurons are trained. The addition of hidden neurons is stopped when the effect of the roundoff errors becomes significant, so that, for instance, confidence intervals cannot be accurately estimated. This phase leads to a set of approved candidate networks. In a subsequent subtractive phase, a selection among approved networks is performed using statistical Fisher tests. The series of tests starts from a possibly too large unbiased network (the full network), and ends with the smallest unbiased network whose input variables and hidden neurons all have a significant contribution to the regression estimate. This method was successfully tested against the realworld regression problems proposed at the NIPS2000 Unlabeled Data Supervised Learning Competition; two of them are included here as illustrative examples.
Neuralnetwork construction and selection in nonlinear modeling
 IEEE Transactions on Neural Networks
, 2003
"... In this paper, we study how statistical tools which are commonly used independently can advantageously be exploited together in order to improve neural network estimation and selection in nonlinear static modeling. The tools we consider are the analysis of the numerical conditioning of the neural ne ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
(Show Context)
In this paper, we study how statistical tools which are commonly used independently can advantageously be exploited together in order to improve neural network estimation and selection in nonlinear static modeling. The tools we consider are the analysis of the numerical conditioning of the neural network candidates, statistical hypothesis tests, and cross validation. We present and analyze each of these tools in order to justify at what stage of a construction and selection procedure they can be most useful. On the basis of this analysis, we then propose a novel and systematic construction and selection procedure for neural modeling. We finally illustrate its efficiency through large scale simulations experiments and real world modeling problems.
An Efficient Fully Unsupervised Video Object Segmentation Scheme Using an Adaptive NeuralNetwork Classifier Architecture
 IEEE Trans. Neural Netw
, 2003
"... In this paper, an unsupervised video object (VO) segmentation and tracking algorithm is proposed based on an adaptable neuralnetwork architecture. The proposed scheme comprises: 1) a VO tracking module and 2) an initial VO estimation module. Object tracking is handled as a classification problem an ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
(Show Context)
In this paper, an unsupervised video object (VO) segmentation and tracking algorithm is proposed based on an adaptable neuralnetwork architecture. The proposed scheme comprises: 1) a VO tracking module and 2) an initial VO estimation module. Object tracking is handled as a classification problem and implemented through an adaptive network classifier, which provides better results compared to conventional motionbased tracking algorithms. Network adaptation is accomplished through an efficient and cost effective weight updating algorithm, providing a minimum degradation of the previous network knowledge and taking into account the current content conditions. A retraining set is constructed and used for this purpose based on initial VO estimation results. Two different scenarios are investigated. The first concerns extraction of human entities in video conferencing applications, while the second exploits depth information to identify generic VOs in stereoscopic video sequences. Human face/ body detection based on Gaussian distributions is accomplished in the first scenario, while segmentation fusion is obtained using color and depth information in the second scenario. A decision mechanism is also incorporated to detect time instances for weight updating. Experimental results and comparisons indicate the good performance of the proposed scheme even in sequences with complicated content (object bending, occlusion).
Pruned Neural Networks for Regression
 In Proc. of the 6th Pacific Rim Conference on Artificial Intelligence, PRICAI 2000, Lecture Notes in AI 1886
, 2000
"... Neural networks have been widely used as a tool for regression. They are capable of approximating any function and they do not require any assumption about the distribution of the data. The most commonly used architectures for regression are the feedforward neural networks with one or more hidden la ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
(Show Context)
Neural networks have been widely used as a tool for regression. They are capable of approximating any function and they do not require any assumption about the distribution of the data. The most commonly used architectures for regression are the feedforward neural networks with one or more hidden layers. In this paper, we present a network pruning algorithm which determines the number of units in the input and hidden layers of the networks. We compare the performance of the pruned networks to four regression methods namely, linear regression (LR), Naive Bayes (NB), knearestneighbor (kNN), and a decision tree predictor M5 0 . On 32 publicly available data sets tested, the neural network method outperforms NB and kNN if the prediction errors are computed in terms of the root mean squared errors. Under this measurement metric, it also performs as well as LR and M5 0 . On the other hand, using the mean absolute error as the measurement metric, the neural network metho...