Results 1  10
of
30
Some Greedy Learning Algorithms for Sparse Regression and Classification with Mercer Kernels
 Journal of Machine Learning Research
, 2002
"... We present some greedy learning algorithms for building sparse nonlinear regression and classification models from observational data using Mercer kernels. Our objective is to develop efficient numerical schemes for reducing the training and runtime complexities of kernelbased algorithms applied ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
We present some greedy learning algorithms for building sparse nonlinear regression and classification models from observational data using Mercer kernels. Our objective is to develop efficient numerical schemes for reducing the training and runtime complexities of kernelbased algorithms applied to large datasets. In the spirit of Natarajan's greedy algorithm (Natarajan, 1995), we iteratively minimize the L 2 loss function subject to a specified constraint on the degree of sparsity required of the final model until a specified stopping criterion is reached. We discuss various greedy criteria for basis selection and numerical schemes for improving the robustness and computational efficiency. Subsequently, algorithms based on residual minimization and thin QR factorization are presented for constructing sparse regression and classification models. During the course of the incremental model construction, the algorithms are terminated using model selection principles such as the minimum descriptive length (MDL) and Akaike's information criterion (AIC). Finally, experimental results on benchmark data are presented to demonstrate the competitiveness of the algorithms developed in this paper.
Estimating Nonstationary Spatial Correlations
, 1996
"... this paper we consider alternative methods based on parametric maximum likelihood fits, using a radial basis function representation of the nonlinear map. A key part of the fitting procedure is model selection, or equivalently, reduction in dimensionality by selection of a subset of radial basis fun ..."
Abstract

Cited by 15 (4 self)
 Add to MetaCart
this paper we consider alternative methods based on parametric maximum likelihood fits, using a radial basis function representation of the nonlinear map. A key part of the fitting procedure is model selection, or equivalently, reduction in dimensionality by selection of a subset of radial basis functions. The methodology is illustrated with two examples, one based on tropospheric ozone and the other on U.S. climate data. However a number of cautions are noted: there is no guarantee of uniqueness of the estimates and the evidence that more complicated models result in improved spatial predictions is, at best, inconclusive. 1. INTRODUCTION
Towards LongTerm Prediction
, 2000
"... This paper describes a simple method of obtaining longerterm predictions from a nonlinear timeseries, assuming one already has a reasonably good shortterm predictor. The usefulness of the technique is that it eliminates, to some extent, the systematic errors of the iterated shortterm predictor. ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
This paper describes a simple method of obtaining longerterm predictions from a nonlinear timeseries, assuming one already has a reasonably good shortterm predictor. The usefulness of the technique is that it eliminates, to some extent, the systematic errors of the iterated shortterm predictor. The technique we describe also provides an indication of the prediction horizon. We consider systems with both observational and dynamic noise and analyse a number of artificial and experimental systems obtaining consistent results. We also compare this method of longerterm prediction with ensemble prediction.
Comparisons of New Nonlinear Modeling Techniques With Applications to Infant Respiration
, 1998
"... This paper concerns the application of new nonlinear timeseries modeling methods to recordings of infant respiratory patterns. The techniques used combine the concept of minimum description length modeling with radial basis models. Our first application of the methods produced results that were not ..."
Abstract

Cited by 7 (6 self)
 Add to MetaCart
This paper concerns the application of new nonlinear timeseries modeling methods to recordings of infant respiratory patterns. The techniques used combine the concept of minimum description length modeling with radial basis models. Our first application of the methods produced results that were not entirely satisfactory, particularly with respect to accurately modeling long term quantitative and qualitative features of respiration patterns. This paper describes a number of modifications of the original methods and makes a comparison of the improvements the various modifications gave. The modifications made were increasing the class of basis function, broadening the range of possible embedding strategies, improving the optimization of the likelihood of the model parameters and calculating a closer approximation to description length. The criteria used in the comparisons were description length, root mean square prediction error, model size, free run behavior and amplitude size and vari...
RadialBasis Models for Feedback Systems With Fading Memory
 IEEE Trans. on CAS–I
, 2001
"... We discuss how to build nonlinear inputoutput models of lowdimensional deterministic systems for both static and dynamic (feedback) systems with "fading memory." To build the dynamic models a new form of radialbasis functions is introduced which, in the absence of an input, have the property that ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
We discuss how to build nonlinear inputoutput models of lowdimensional deterministic systems for both static and dynamic (feedback) systems with "fading memory." To build the dynamic models a new form of radialbasis functions is introduced which, in the absence of an input, have the property that they converge to a constant solution. The utility of these models is illustrated by building accurate and stable models for electronic circuits with dynamic (memory) effects.
Detecting Nonlinearity in Experimental Data
 International Journal of Bifurcation and Chaos Submitted
, 1997
"... The technique of surrogate data has been used as a method to test for membership of particular classes of linear systems. We suggest an obvious extension of this to classes of nonlinear parametric models and demonstrate our methods with respiratory data from sleeping human infants. Although our data ..."
Abstract

Cited by 5 (5 self)
 Add to MetaCart
The technique of surrogate data has been used as a method to test for membership of particular classes of linear systems. We suggest an obvious extension of this to classes of nonlinear parametric models and demonstrate our methods with respiratory data from sleeping human infants. Although our data are clearly distinct from the different classes of linear systems we are unable to distinguish between our data and surrogates generated by nonlinear models. Hence we conclude that human respiration is likely to be a nonlinear system with more than 2 degrees of freedom with a limit cycle that is driven by high dimensional dynamics or noise.
Correlation dimension: A Pivotal statistic for nonconstrained realizations of composite hypotheses in surrogate data analysis.
"... Currently surrogate data analysis can be used to determine if data is consistent with various linear systems, or something else (a nonlinear system). In this paper we propose an extension of these methods in an attempt to make more specific classifications within the the class of nonlinear systems. ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
Currently surrogate data analysis can be used to determine if data is consistent with various linear systems, or something else (a nonlinear system). In this paper we propose an extension of these methods in an attempt to make more specific classifications within the the class of nonlinear systems. In the method of surrogate data one estimates the probability distribution of values of a test statistic for a set of experimental data under the assumption that the data is consistent with a given hypothesis. If the probability distribution of the test statistic is different for different dynamical systems consistent with the hypothesis one must ensure that the surrogate generation technique generates surrogate data that are a good approximation to the data. This is often achieved with a careful choice of surrogate generation method and for noise driven linear surrogates such methods are commonly used. This paper argues that, in many cases (particularly for nonlinear hypotheses), it is ea...
Towards intentional dynamics in supply chain conscious process operations
 In Proceedings of the Foundations of Computer Aided Process Operations Conference (FOCAPO 98
, 1998
"... Chemical and refinery process operations have to deal with an increasingly transient and competitive marketplace. The traditional strategy of operating a plant in isolation from its environment within and outside the supply chain, where exogenous influences are interpreted and handled as disturbance ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
Chemical and refinery process operations have to deal with an increasingly transient and competitive marketplace. The traditional strategy of operating a plant in isolation from its environment within and outside the supply chain, where exogenous influences are interpreted and handled as disturbances, is not any more appropriate. Rather, manufacturing must quickly adapt to the transient environment to exploit economical potentials to the degree possible. At least, the traditional plant focussed operational strategy must account for the dynamics in the disturbances by employing predictions of their future timevarying behavior in realtime optimization to result in an intentionally dynamic operational strategy. Further, plant performance can be significantly improved, if knowledge on the status and on the future policies and goals of some partners in the supply chain can be employed for plant optimization. Such a cooperative mode of interaction in the supply chain will lead from plant focussed to supply chain conscious, often intentionally dynamic plant operation. Advanced concepts are identified and discussed, a number of relevant research issues are suggested. Keywords
Detecting Determinism in Time Series: The method of Surrogate Data
 IEEE TRANS. ON CIRCUITS AND SYSTEMSI FUNDAMENTAL THEORY AND APPLICATIONS
, 2003
"... We review a relatively new statistical test that may be applied to determine whether an observed time series is inconsistent with a specific class of dynamical systems. These surrogate data methods may test an observed time series against the hypotheses of: i) independent and identically distributed ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
We review a relatively new statistical test that may be applied to determine whether an observed time series is inconsistent with a specific class of dynamical systems. These surrogate data methods may test an observed time series against the hypotheses of: i) independent and identically distributed noise; ii) linearly filtered noise; and iii) a monotonic nonlinear transformation of linearly filtered noise. A recently suggested fourth algorithm for testing the hypothesis of a periodic orbit with uncorrelated noise is also described. We propose several novel applications of these methods for various engineering problems, including: identifying a deterministic (message) signal in a noisy time series; and separating deterministic and stochastic components. When employed to separate deterministic and noise components, we show that the application of surrogate methods to the residuals of nonlinear models is equivalent to fitting that model subject to an information theoretic model selection criteria.
Extracting biochemical reaction kinetics from time series data
 In M. Gh. Negoita et al. (Eds.), KnowledgeBased Intelligent Information & Engineering Systems, Lecture Notes in Artificial Intelligence
, 2004
"... Abstract. We consider the problem of inferring kinetic mechanisms for biochemical reactions from time series data. Using a priori knowledge about the structure of chemical reaction kinetics we develop global nonlinear models which use elementary reactions as a basis set, and discuss model constructi ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Abstract. We consider the problem of inferring kinetic mechanisms for biochemical reactions from time series data. Using a priori knowledge about the structure of chemical reaction kinetics we develop global nonlinear models which use elementary reactions as a basis set, and discuss model construction using topdown and bottomup approaches. 1