Results 1  10
of
36
Bayesian Interpolation
 Neural Computation
, 1991
"... Although Bayesian analysis has been in use since Laplace, the Bayesian method of modelcomparison has only recently been developed in depth. In this paper, the Bayesian approach to regularisation and modelcomparison is demonstrated by studying the inference problem of interpolating noisy data. T ..."
Abstract

Cited by 520 (18 self)
 Add to MetaCart
Although Bayesian analysis has been in use since Laplace, the Bayesian method of modelcomparison has only recently been developed in depth. In this paper, the Bayesian approach to regularisation and modelcomparison is demonstrated by studying the inference problem of interpolating noisy data. The concepts and methods described are quite general and can be applied to many other problems. Regularising constants are set by examining their posterior probability distribution. Alternative regularisers (priors) and alternative basis sets are objectively compared by evaluating the evidence for them. `Occam's razor' is automatically embodied by this framework. The way in which Bayes infers the values of regularising constants and noise levels has an elegant interpretation in terms of the effective number of parameters determined by the data set. This framework is due to Gull and Skilling. 1 Data modelling and Occam's razor In science, a central task is to develop and compare models to a...
An algebra for probabilistic databases
"... An algebra is presented for a simple probabilistic data model that may be regarded as an extension of the standard relational model. The probabilistic algebra is developed in such a way that (restricted to αacyclic database schemes) the relational algebra is a homomorphic image of it. Strictly prob ..."
Abstract

Cited by 125 (1 self)
 Add to MetaCart
An algebra is presented for a simple probabilistic data model that may be regarded as an extension of the standard relational model. The probabilistic algebra is developed in such a way that (restricted to αacyclic database schemes) the relational algebra is a homomorphic image of it. Strictly probabilistic results are emphasized. Variations on the basic probabilistic data model are discussed. The algebra is used to explicate a commonly used statistical smoothing procedure and is shown to be potentially very useful for decision support with uncertain information.
Clustering for sparsely sampled functional data
 Journal of the American Statistical Association
, 2003
"... We develop a flexible modelbased procedure for clustering functional data. The technique can be applied to all types of curve data but is particularly useful when individuals are observed at a sparse set of time points. In addition to producing final cluster assignments, the procedure generates pre ..."
Abstract

Cited by 45 (6 self)
 Add to MetaCart
We develop a flexible modelbased procedure for clustering functional data. The technique can be applied to all types of curve data but is particularly useful when individuals are observed at a sparse set of time points. In addition to producing final cluster assignments, the procedure generates predictions and confidence intervals for missing portions of curves. Our approach also provides many useful tools for evaluating the resulting models. Clustering can be assessed visually via low dimensional representations of the curves, and the regions of greatest separation between clusters can be determined using a discriminant function. Finally, we extend the model to handle multiple functional and finite dimensional covariates and show how it can be applied to standard finite dimensional clustering problems involving missing data.
Functional linear discriminant analysis for irregularly sampled curves
 Journal of the Royal Statistical Society, Series B, Methodological
, 2001
"... We introduce a technique for extending the classical method of Linear Discriminant Analysis to data sets where the predictor variables are curves or functions. This procedure, which we call functionallinear discriminant analysis (FLDA), is particularly useful when only fragments of the curves are ob ..."
Abstract

Cited by 36 (7 self)
 Add to MetaCart
We introduce a technique for extending the classical method of Linear Discriminant Analysis to data sets where the predictor variables are curves or functions. This procedure, which we call functionallinear discriminant analysis (FLDA), is particularly useful when only fragments of the curves are observed. All the techniques associated with LDA can be extended for use with FLDA. In particular FLDA can be used to produce classifications on new (test) curves, give an estimate of the discriminant function between classes, and provide a one or two dimensional pictorial representation of a set of curves. We also extend this procedure to provide generalizations of quadratic and regularized discriminant analysis.
Image Denoising: Pointwise Adaptive Approach
 Annals of Statistics
, 1998
"... A new method of pointwise adaptation has been proposed and studied in Spokoiny (1998) in context of estimation of piecewise smooth univariate functions. The present paper extends that method to estimation of bivariate greyscale images composed of large homogeneous regions with smooth edges and obse ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
A new method of pointwise adaptation has been proposed and studied in Spokoiny (1998) in context of estimation of piecewise smooth univariate functions. The present paper extends that method to estimation of bivariate greyscale images composed of large homogeneous regions with smooth edges and observed with noise on a gridded design. The proposed estimator # f(x) at a point x is simply the average of observations over a window # U(x) selected in a datadriven way. The theoretical properties of the procedure are studied for the case of piecewise constant images. We present a nonasymptotic bound for the accuracy of estimation at a specific grid point x as a function of the number of pixel n, of the distance from the point of estimation to the closest boundary and of smoothness properties and orientation of this boundary. It is also shown that the proposed method provides a near optimal rate of estimation near edges and inside homogeneous regions. We briefly discuss algorithmic aspects and the complexity of the procedure. The numerical examples demonstrate a reasonable performance of the method and they are in agreement with the theoretical issues. An example from satellite (SAR) imaging illustrates the applicability of the method. # The authors thank A.Juditski, O. Lepski, A.Tsybakov and Yu.Golubev for important remarks and discussion. polzehl, j. and spokoiny, v. 1 1
Minimum variance in biased estimation: Bounds and asymptotically optimal estimators
 IEEE Trans. Signal Processing
, 2004
"... Abstract—We develop a uniform Cramér–Rao lower bound (UCRLB) on the total variance of any estimator of an unknown vector of parameters, with bias gradient matrix whose norm is bounded by a constant. We consider both the Frobenius norm and the spectral norm of the bias gradient matrix, leading to two ..."
Abstract

Cited by 12 (7 self)
 Add to MetaCart
Abstract—We develop a uniform Cramér–Rao lower bound (UCRLB) on the total variance of any estimator of an unknown vector of parameters, with bias gradient matrix whose norm is bounded by a constant. We consider both the Frobenius norm and the spectral norm of the bias gradient matrix, leading to two corresponding lower bounds. We then develop optimal estimators that achieve these lower bounds. In the case in which the measurements are related to the unknown parameters through a linear Gaussian model, Tikhonov regularization is shown to achieve the UCRLB when the Frobenius norm is considered, and the shrunken estimator is shown to achieve the UCRLB when the spectral norm is considered. For more general models, the penalized maximum likelihood (PML) estimator with a suitable penalizing function is shown to asymptotically achieve the UCRLB. To establish the asymptotic optimality of the PML estimator, we first develop the asymptotic mean and variance of the PML estimator for any choice of penalizing function satisfying certain regularity constraints and then derive a general condition on the penalizing function under which the resulting PML estimator asymptotically achieves the UCRLB. This then implies that from all linear and nonlinear estimators with bias gradient whose norm is bounded by a constant, the proposed PML estimator asymptotically results in the smallest possible variance. Index Terms—Asymptotic optimality, biased estimation, bias gradient norm, Cramér–Rao lower bound, penalized maximum likelihood, Tikhonov regularization.
Rethinking biased estimation: Improving maximum likelihood and the CramérRao bound,” Found
 Trends in Signal Process
"... One of the prime goals of statistical estimation theory is the development of performance bounds when estimating parameters of interest in a given model, as well as constructing estimators that achieve these limits. When the parameters to be estimated are deterministic, a popular approach is to boun ..."
Abstract

Cited by 12 (9 self)
 Add to MetaCart
One of the prime goals of statistical estimation theory is the development of performance bounds when estimating parameters of interest in a given model, as well as constructing estimators that achieve these limits. When the parameters to be estimated are deterministic, a popular approach is to bound the meansquared error (MSE) achievable within the class of unbiased estimators. Although it is wellknown that lower MSE can be obtained by allowing for a bias, in applications it is typically unclear how to choose an appropriate bias. In this survey we introduce MSE bounds that are lower than the unbiased Cramér–Rao bound (CRB) for all values of the unknowns. We then present a general framework for constructing biased estimators with smaller MSE than the standard maximumlikelihood (ML) approach, regardless of the true unknown values. Specializing the results to the linear Gaussian model, we derive a class of estimators that dominate leastsquares in terms of MSE. We also introduce methods for choosing regularization parameters in penalized ML estimators that outperform standard techniques such as cross validation. 1
Probabilistic data analysis: an introductory guide
 JOURNAL OF MICROSCOPY 190:28–36
, 1998
"... Quantitative science requires the assessment of uncertainty, and this means that measurements and inferences should be described as probability distributions. This is done by building data into a probabilistic likelihood function which produces a posterior “answer ” by modulating a prior “question”. ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
Quantitative science requires the assessment of uncertainty, and this means that measurements and inferences should be described as probability distributions. This is done by building data into a probabilistic likelihood function which produces a posterior “answer ” by modulating a prior “question”. Probability calculus is the only way of doing this consistently, so that data can be included gradually or all at once while the answer remains the same. But probability calculus is only a language: it does not restrict the questions one can ask by setting one’s prior. We discuss how to set sensible priors, in particular for a large problem like image reconstruction. We also introduce practical modern algorithms (Gibbs sampling, Metropolis algorithm, genetic algorithms, and simulated annealing) for computing probabilistic inference.
Weighted LeastSquares Criteria For Seismic Traveltime Tomography
 IEEE Trans. Geosci. Remote Sensing
, 1989
"... Methods are developed for design of linear tomographic reconstruction algorithms with specified properties. Assuming a starting model with constant slowness, an algorithm with the following properties is found: (1) The optimum constant for the starting model is determined automatically. (2) The weig ..."
Abstract

Cited by 8 (5 self)
 Add to MetaCart
Methods are developed for design of linear tomographic reconstruction algorithms with specified properties. Assuming a starting model with constant slowness, an algorithm with the following properties is found: (1) The optimum constant for the starting model is determined automatically. (2) The weighted leastsquares error between the predicted and measured traveltime data is as small as possible. (3) The variance of the reconstructed slowness from the starting model is minimized. (4) Rays with the greatest length have the least influence on the reconstructed slowness. (5) Cells with most ray coverage tend to deviate least from the background value. The resulting algorithm maps the reconstruction problem into a vector space where the contribution to the inversion from the background slowness remains invariant, while the optimum contributions in orthogonal directions are found. For a starting model with nonconstant slowness, the reconstruction algorithm has analogous properties.  2 ...