. The Bayesian approach to probability theory is presented as an alternative to the currently used long-run relative frequency approach, which does not offer clear, compelling criteria for the design of statistical methods. Bayesian probability theory offers unique and demonstrably optimal solutions to well-posed statistical problems, and is historically the original approach to statistics. The reasons for earlier rejection of Bayesian methods are discussed, and it is noted that the work of Cox, Jaynes, and others answers earlier objections, giving Bayesian inference a firm logical and mathematical foundation as the correct mathematical language for quantifying uncertainty. The Bayesian approaches to parameter estimation and model comparison are outlined and illustrated by application to a simple problem based on the gaussian distribution. As further illustrations of the Bayesian paradigm, Bayesian solutions to two interesting astrophysical problems are outlined: the measurement of wea...

In this contribution, models of wireless channels are derived from the maximum entropy principle, for several cases where only limited information about the propagation environment is available. First, analytical models are derived for the cases where certain parameters (channel energy, average energy, spatial correlation matrix) are known deterministically. Frequently, these parameters are unknown (typically because the received energy or the spatial correlation varies with the user position), but still known to represent meaningful system characteristics. In these cases, analytical channel models are derived by assigning entropy-maximizing distributions to these parameters, and marginalizing them out. For the MIMO case with spatial correlation, we show that the distribution of the covariance matrices is conveniently handled through its eigenvalues. The entropy-maximizing distribution of the covariance matrix is shown to be a Wishart distribution. Furthermore, the corresponding probability density function of the channel matrix is shown to be described analytically by a function of the channel Frobenius norm. This technique can provide channel models incorporating the effect of shadow fading and spatial correlation between antennas without the need to assume explicit values for these parameters. The results are compared in terms of mutual information to the classical i.i.d. Gaussian model.

Abstract — We investigate the problem of establishing the joint probability distribution of the entries of a Multiple-Input Multiple-Output (MIMO) spatially correlated flat-fading channel, when little or no information about the channel properties are available. We show that the entropy of a random positive semidefinite matrix is maximized by the Wishart distribution. We subsequently obtain the Maximum Entropy distribution of the MIMO transfer matrix by establishing its distribution conditioned on the covariance, and by later marginalizing over the covariance matrix. The obtained distribution is isotropic, and is described analytically as a function of the Frobenius norm of the channel matrix. I.

We consider entropy and relative entropy in Field theory and establish relevant monotonicity properties with respect to the couplings. The relative entropy in a field theory with a hierarchy of renormalization group fixed points ranks the fixed points, the lowest relative entropy being assigned to the highest multicritical point. We argue that as a consequence of a generalized H theorem Wilsonian RG flows induce an increase in entropy and propose the relative entropy as the natural quantity which increases from one fixed point to another in more than two dimensions. The concept of entropy was introduced by Clausius through the study of thermodynamical systems. However it was Boltzmann’s essential discovery that entropy is the natural quantity that bridges the microscopic and macroscopic descriptions of a system which gave it its modern interpretation. A more general definition, proposed by Gibbs allowed

Abstract not found

Abstract not found

Provenance of correlations in psychological data There are few truisms in the field of psychology, but one of them is surely that measurement error is found in all experiments. Data are inevitably produced that do not perfectly reflect the logic imposed by the experimental design. To the extent that a psychological experiment succeeds in measuring something or in making some sort of distinction, the data will partially reflect the design, and this leads to a way of thinking about data that is found throughout all the experimental sciences: data � signal � noise. This innocent equation almost always contains an implicit but critical assumption: that the noise may be regarded as independent samples from some distribution— typically taken to be the Gaussian distribution. In this way, the residual error is conceived of as a featureless background of white noise in which the interesting part, the treatment means, are more or less buried. Often this conception of data is justified. Whenever there is random assignment to cells and each participant contributes a single datum, errors may be expected to be uncorrelated. However, in all of sensory psychophysics and most of cognitive psychology, single individuals respond to entire blocks of trials in a given experimental session. Here, the residual error will develop correlations by virtue of the circumstance that the response history was laid down by a nervous system that has memory. In many situations, these correlations are little more than a Preparation of this article was supported by NIMH Grants R01-

This paper describes NYU's effort toward improving recognition accuracy for the 1996 ARPA Large Vocabulary Continuous Speech Recognition evaluation. We are trying to develop different kinds of language models including longer-range models and a linguistically motivated model. For the system described here, we used as a starting point the scores produced by SRI's acoustic and language models. These are linearly combined with the scores produced by the NYU language models. This paper also describes some experiments we tried which were not used in the official experiment, including experiments with perplexity minimization, MaximumEntropy modeling and parsing. 1. Introduction This paper describes NYU's effort toward improving recognition accuracy for the 1996 ARPA Large Vocabulary Continuous Speech Recognition evaluation. Our goal has been to study some longerrange language models and determine whether they can be a useful component of the language models used for speech recognition. We ...

From an information theoretic point of view, the inverse problem and the problem of data compression are intimately related. Optimal compression seeks the most concise representation of a data set, while Bayesian probability theory favors image reconstruction algorithms which minimally model the information present in the data. This should not be surprising. It is in keeping with a scientists intuitive need to satisfy the precepts of Occam's Razor, i.e. not to over interpret one's data. Information scientists might describe this process as quantifying the Algorithmic Information Content (AIC) of the image, and then using this "coordinate system" for optimal image reconstruction.

University of Glasgow for providing all the resources used during the implementation procedure and the researchers in the Inference Group for their help and support. Special thanks to: My supervisor, Dr. Vladislav Vyshemirsky, who was always able to find some time to explain and correct things; and without whom this work would have never been accomplished. My second supervisor, Prof. Mark Girolami, whose scientific guidance and advice were of great importance. In biochemical models defined by systems of ordinary differential equations, there is always a level of uncertainty regarding the appropriate parameter values. Bayesian methods of parameter inference and evidence-based model comparison are considered to be sound methods to handle such uncertainty; not assigning fixed values to the parameters, but using probability distributions, formally taking prior knowledge into account. BioBayes is a software