Results 1 
7 of
7
Frugal hypothesis testing and classification
, 2010
"... The design and analysis of decision rules using detection theory and statistical learning theory is important because decision making under uncertainty is pervasive. Three perspectives on limiting the complexity of decision rules are considered in this thesis: geometric regularization, dimensionalit ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
The design and analysis of decision rules using detection theory and statistical learning theory is important because decision making under uncertainty is pervasive. Three perspectives on limiting the complexity of decision rules are considered in this thesis: geometric regularization, dimensionality reduction, and quantization or clustering. Controlling complexity often reduces resource usage in decision making and improves generalization when learning decision rules from noisy samples. A new marginbased classifier with decision boundary surface area regularization and optimization via variational level set methods is developed. This novel classifier is termed the geometric level set (GLS) classifier. A method for joint dimensionality reduction and marginbased classification with optimization on the Stiefel manifold is developed. This dimensionality reduction approach is extended for information fusion in sensor networks. A new distortion is proposed for the quantization or clustering of prior probabilities appearing in the thresholds of likelihood ratio tests. This distortion is given the name mean Bayes risk error (MBRE). The quantization framework is extended to model human decision
R.: Adaptive metric dimensionality reduction
, 2013
"... Abstract. We study dataadaptive dimensionality reduction in the context of supervised learning in general metric spaces. Our main statistical contribution is a generalization bound for Lipschitz functions in metric spaces that are doubling, or nearly doubling, which yields a new theoretical explana ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
(Show Context)
Abstract. We study dataadaptive dimensionality reduction in the context of supervised learning in general metric spaces. Our main statistical contribution is a generalization bound for Lipschitz functions in metric spaces that are doubling, or nearly doubling, which yields a new theoretical explanation for empirically reported improvements gained by preprocessing Euclidean data by PCA (Principal Components Analysis) prior to constructing a linear classifier. On the algorithmic front, we describe an analogue of PCA for metric spaces, namely an efficient procedure that approximates the data’s intrinsic dimension, which is often much lower than the ambient dimension. Our approach thus leverages the dual benefits of low dimensionality: (1) more efficient algorithms, e.g., for proximity search, and (2) more optimistic generalization bounds. 1
THE MAXLENGTHVECTOR LINE OF BEST FIT TO A COLLECTION OF VECTOR SUBSPACES
"... (Communicated by) Abstract. Let C = {V1, V2,..., Vk} be a finite collection of nontrivial subspaces of a finite dimensional real vector space V. Let L denote a one dimensional subspace of V and let θ(L, Vi) denote the principal (or canonical) angle between L and Vi. We are interested in finding al ..."
Abstract
 Add to MetaCart
(Show Context)
(Communicated by) Abstract. Let C = {V1, V2,..., Vk} be a finite collection of nontrivial subspaces of a finite dimensional real vector space V. Let L denote a one dimensional subspace of V and let θ(L, Vi) denote the principal (or canonical) angle between L and Vi. We are interested in finding all lines that maximize the function F (L) = ∑k i=1 cos θ(L, Vi). Conceptually, this is the line through the origin that best represents C with respect to the criterion F (L). A reformulation shows that L is spanned by a vector v = ∑k i=1 vi which maximizes the function G(v1,..., vk) =  ∑k i=1 vi2 subject to the constraints vi ∈ Vi and vi  = 1. Using Lagrange multipliers, the critical points of G are solutions of a polynomial system corresponding to a multivariate eigenvector problem. We use homotopy continuation and numerical algebraic geometry to solve the system and obtain the maxlengthvector line(s) of best fit to C.
Balancing Lifetime and Classification Accuracy of Wireless Sensor Networks
"... ABSTRACT Wireless sensor networks are composed of distributed sensors that can be used for signal detection or classification. The likelihood functions of the hypotheses are often not known in advance, and decision rules have to be learned via supervised learning. A specific learning algorithm is F ..."
Abstract
 Add to MetaCart
(Show Context)
ABSTRACT Wireless sensor networks are composed of distributed sensors that can be used for signal detection or classification. The likelihood functions of the hypotheses are often not known in advance, and decision rules have to be learned via supervised learning. A specific learning algorithm is Fisher discriminant analysis (FDA), the classification accuracy of which has been previously studied in the context of wireless sensor networks. Previous work, however, does not take into account the communication protocol or battery lifetime; in this paper we extend existing studies by proposing a model that captures the relationship between battery lifetime and classification accuracy. To do so, we combine the FDA with a model that captures the dynamics of the carriersense multipleaccess (CSMA) algorithm, the randomaccess algorithm used to regulate communications in sensor networks. This allows us to study the interaction between the classification accuracy, battery lifetime and effort put towards learning, as well as the impact of the backoff rates of CSMA on the accuracy. We characterize the tradeoff between the length of the training stage and accuracy, and show that accuracy is nonmonotone in the backoff rate due to changes in the training sample size and overfitting.
Linear Dimensionality Reduction: Survey, Insights, and Generalizations
, 2015
"... Abstract Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, corr ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, inputoutput relationships, and margin between data classes. Methods have been developed with a variety of names and motivations in many fields, and perhaps as a result the connections between all these methods have not been highlighted. Here we survey methods from this disparate literature as optimization programs over matrix manifolds. We discuss principal component analysis, factor analysis, linear multidimensional scaling, Fisher's linear discriminant analysis, canonical correlations analysis, maximum autocorrelation factors, slow feature analysis, sufficient dimensionality reduction, undercomplete independent component analysis, linear regression, distance metric learning, and more. This optimization framework gives insight to some rarely discussed shortcomings of wellknown methods, such as the suboptimality of certain eigenvector solutions. Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal lowdimensional projection of the data. This simple optimization framework further allows straightforward generalizations and novel variants of classical methods, which we demonstrate here by creating an orthogonalprojection canonical correlations analysis. More broadly, this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox, objectiveagnostic numerical technology.
Unifying Linear Dimensionality Reduction
, 2014
"... Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation b ..."
Abstract
 Add to MetaCart
(Show Context)
Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, inputoutput relationships, and margin between data classes. Methods have been developed with a variety of names and motivations in many fields, and perhaps as a result the deeper connections between all these methods have not been understood. Here we unify methods from this disparate literature as optimization programs over matrix manifolds. We discuss principal component analysis, factor analysis, linear multidimensional scaling, Fisher’s linear discriminant analysis, canonical correlations analysis, maximum autocorrelation factors, slow feature analysis, undercomplete independent component analysis, linear regression, and more. This optimization framework helps elucidate some rarely discussed shortcomings of wellknown methods, such as the suboptimality of certain eigenvector solutions. Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal lowdimensional projection of the data. This optimization framework further allows rapid development of novel variants of classical methods, which we demonstrate here by creating an orthogonalprojection canonical correlations analysis. More broadly, we suggest that our generic linear dimensionality reduction solver can move linear dimensionality reduction toward becoming a blackbox, objectiveagnostic numerical technology. 1