Results 1  10
of
12
Game Theory, Maximum Entropy, Minimum Discrepancy And Robust Bayesian Decision Theory
 ANNALS OF STATISTICS
, 2004
"... ..."
Probability Update: Conditioning vs. CrossEntropy
 In Proc. Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI
, 1997
"... Conditioning is the generally agreedupon method for updating probability distributions when one learns that an event is certainly true. But it has been argued that we need other rules, in particular the rule of crossentropy minimization, to handle updates that involve uncertain information. In thi ..."
Abstract

Cited by 13 (2 self)
 Add to MetaCart
Conditioning is the generally agreedupon method for updating probability distributions when one learns that an event is certainly true. But it has been argued that we need other rules, in particular the rule of crossentropy minimization, to handle updates that involve uncertain information. In this paper we reexamine such a case: van Fraassen's Judy Benjamin problem [1987], which in essence asks how one might update given the value of a conditional probability. We argue thatcontrary to the suggestions in the literatureit is possible to use simple conditionalization in this case, and thereby obtain answers that agree fully with intuition. This contrasts with proposals such as crossentropy, which are easier to apply but can give unsatisfactory answers. Based on the lessons from this example, we speculate on some general philosophical issues concerning probability update. 1 INTRODUCTION How should one update one's beliefs, represented as a probability distribution Pr over some ...
Relative Entropy and Inductive Inference
 in Bayesian Inference and Maximum Entropy Methods in Science and Engineering
, 2004
"... We discuss how the method of maximum entropy, MaxEnt, can be extended beyond its original scope, as a rule to assign a probability distribution, to a fullfledged method for inductive inference. The main concept is the (relative) entropy S[pq] which is designed as a tool to update from a prior prob ..."
Abstract

Cited by 11 (6 self)
 Add to MetaCart
We discuss how the method of maximum entropy, MaxEnt, can be extended beyond its original scope, as a rule to assign a probability distribution, to a fullfledged method for inductive inference. The main concept is the (relative) entropy S[pq] which is designed as a tool to update from a prior probability distribution q to a posterior probability distribution p when new information in the form of a constraint becomes available. The extended method goes beyond the mere selection of a single posterior p, but also addresses the question of how much less probable other distributions might be. Our approach clarifies how the entropy S[pq] is used while avoiding the question of its meaning. Ultimately, entropy is a tool for induction which needs no interpretation. Finally, being a tool for generalization from special examples, we ask whether the functional form of the entropy depends on the choice of the examples and we find that it does. The conclusion is that there is no single general theory of inductive inference and that alternative expressions for the entropy are possible. 1
The Constraint Rule of the Maximum Entropy Principle
, 1995
"... The principle of maximum entropy is a method for assigning values to probability distributions on the basis of partial information. In usual formulations of this and related methods of inference one assumes that this partial information takes the form of a constraint on allowed probability distri ..."
Abstract

Cited by 11 (0 self)
 Add to MetaCart
The principle of maximum entropy is a method for assigning values to probability distributions on the basis of partial information. In usual formulations of this and related methods of inference one assumes that this partial information takes the form of a constraint on allowed probability distributions. In practical applications, however, the information consists of empirical data. A constraint rule is then employed to construct constraints on probability distributions out of these data. Usually one adopts the rule to equate the expectation values of certain functions with their empirical averages. There are, however, various other ways in which one can construct constraints from empirical data, which makes the maximum entropy principle lead to very different probability assignments. This paper shows that an argument by Jaynes to justify the usual constraint rule is unsatisfactory and investigates several alternative choices. The choice of a constraint rule is also show...
Generalizing the lottery paradox
 The British Journal for the Philosophy of Science
"... This paper is concerned with formal solutions to the lottery paradox on which high probability defeasibly warrants acceptance. It considers some recently proposed solutions of this type and presents an argument showing that these solutions are trivial in that they boil down to the claim that perfect ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
This paper is concerned with formal solutions to the lottery paradox on which high probability defeasibly warrants acceptance. It considers some recently proposed solutions of this type and presents an argument showing that these solutions are trivial in that they boil down to the claim that perfect probability is sufficient for rational acceptability. The argument is then generalized, showing that a broad class of similar solutions faces the same problem. Over the past decades, there has been a steadily growing interest in utilizing probability theory to elucidate, or even analyze, concepts central to traditional epistemology. Special attention in this regard has been given to the notion of rational acceptability. Many have found the following thesis at least prima facie a promising starting point for a probabilistic elucidation of that notion: Sufficiency Thesis (ST) A propositionϕis rationally acceptable if Pr(ϕ)>t, where Pr is a probability distribution over propositions and t is a threshold value close to 1. 1 Another plausible constraint is that when some propositions are rationally
Seeing maximum entropy from the principle of virtual work
"... We propose an extension of the principle of virtual work of mechanics to random dynamics of mechanical systems. The total virtual work of the interacting forces and inertial forces on every particle of the system is calculated by considering the motion of each particle. Then according to the princip ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
We propose an extension of the principle of virtual work of mechanics to random dynamics of mechanical systems. The total virtual work of the interacting forces and inertial forces on every particle of the system is calculated by considering the motion of each particle. Then according to the principle of Lagranged’Alembert for dynamical equilibrium, the vanishing ensemble average of the virtual work gives rise to the thermodynamic equilibrium state with maximization of thermodynamic entropy. This approach establishes a close relationship between the maximum entropy approach for statistical mechanics and a fundamental principle of mechanics, and constitutes an attempt to give the maximum entropy approach, considered by many as only an inference principle based on the subjectivity of probability and entropy, the status of fundamental physics law.
CEA/Saclay DSM/SPhT
"... We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involving the ..."
Abstract
 Add to MetaCart
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involving the observers is required. The criterion of maximum von Neumann entropy is then used for making reasonable inferences. It means that no spurious information is introduced besides the known data. Its outcomes can be given a direct justification based on the principle of indifference of Laplace. We introduce the concept of relevant entropy associated with some set of relevant variables; it characterizes the information that is missing at the microscopic level when only these variables are known. For equilibrium problems, the relevant variables are the conserved ones, and the Second Law is recovered as a second step of the inference process. For nonequilibrium problems, the increase of the relevant entropy expresses an irretrievable loss of information from the relevant variables towards the irrelevant ones. Two examples illustrate the flexibility of the choice of relevant variables and the multiplicity of the associated entropies: the thermodynamic entropy (satisfying the Clausius–Duhem inequality) and the Boltzmann entropy (satisfying the Htheorem). The identification of entropy with missing information is also supported by the paradox of Maxwell’s demon. Spinecho experiments show that irreversibility itself is not an absolute concept: use of hidden information may overcome the arrow of time.
Spectrum Estimation Using Multirate Observations
, 2003
"... This article considers merging the statistical information gained in lowrate measurements of a nonobservable highrate signal. We consider a model where a widesense stationary random signal x(n) is being observed indirectly using several linear multirate sensors. Each sensor outputs a measurement ..."
Abstract
 Add to MetaCart
This article considers merging the statistical information gained in lowrate measurements of a nonobservable highrate signal. We consider a model where a widesense stationary random signal x(n) is being observed indirectly using several linear multirate sensors. Each sensor outputs a measurement signal v i (n) whose sampling rate is only a fraction of the sampling rate assumed for the original signal. We pose the following problem: Given certain autocorrelation coefficients of the observable signals v i (n), estimate the power spectral density of the original signal x(n). It turns