The design and analysis of decision rules using detection theory and statistical learning theory is important because decision making under uncertainty is pervasive. Three perspectives on limiting the complexity of decision rules are considered in this thesis: geometric regularization, dimensionality reduction, and quantization or clustering. Controlling complexity often reduces resource usage in decision making and improves generalization when learning decision rules from noisy samples. A new margin-based classifier with decision boundary surface area regularization and optimization via variational level set methods is developed. This novel classifier is termed the geometric level set (GLS) classifier. A method for joint dimensionality reduction and margin-based classification with optimization on the Stiefel manifold is developed. This dimensionality reduction approach is extended for information fusion in sensor networks. A new distortion is proposed for the quantization or clustering of prior probabilities appearing in the thresholds of likelihood ratio tests. This distortion is given the name mean Bayes risk error (MBRE). The quantization framework is extended to model human decision

In signal detection, Bayesian hypothesis testing and minimax hypothesis testing represent two extremes in the knowledge of the prior probabilities of the hypotheses: full information and no information. We propose an intermediate formulation, also based on the likelihood ratio test, to allow for partial information. We partition the space of prior probabilities into a set of levels using a quantizationtheoretic approach with a minimax Bayes risk error criterion. Within each prior probability level, an optimal representative probability value is found, which is used to set the threshold of the likelihood ratio test. The formulation is demonstrated on signals with additive Gaussian noise. Index Terms — quantization, categorization, hypothesis testing, signal detection, Bayes risk error 1.