We prove that the Poisson distribution maximises entropy in the class of ultralog-concave distributions, extending a result of Harremoës. The proof uses ideas concerning log-concavity, and a semigroup action involving adding Poisson variables and thinning. We go on to show that the entropy is a concave function along this semigroup. 1 Maximum entropy distributions It is well-known that the distributions which maximise entropy under certain very natural conditions take a simple form. For example, among random variables with fixed mean and variance the entropy is maximised by the normal distribution. Similarly, for random variables with positive support and fixed mean, the entropy is maximised by the exponential distribution. The standard technique for proving such results uses the Gibbs inequality, and exploits the fact that, given a function f(x), and fixing Λ(p) = ∫ p(x)f(x)dx, the maximum entropy density is of the form α exp(−βf(x)). Example 1.1 For a density p with mean µ and variance σ2, write φµ,σ2 for the density of a N(µ, σ2) random variable, and define the function Λ(p) = − ∫ p(x) log φ µ,σ2(x)dx.

Abstract—Some entropy comparison results are presented concerning compound distributions on nonnegative integers. The main result shows that, under a log-concavity assumption, two compound distributions are ordered in terms of Shannon entropy if both the “numbers of claims ” and the “claim sizes ” are ordered accordingly in the convex order. Several maximum/minimum entropy theorems follow as a consequence. Most importantly, two recent results of Johnson et al. (2008) on maximum entropy characterizations of compound Poisson and compound binomial distributions are proved under fewer assumptions and with simpler arguments. Index Terms—compound binomial, compound Poisson, convex order, infinite divisibility, log-concavity, maximum entropy, minimum

Abstract—A version of the law of small numbers is analyzed in information-theoretic terms. Specifically, let f = {fi, i = 0, 1,...} be a probability mass function (pmf) on nonnegative integers with mean λ < ∞. Denote the nth convolution of f by f ∗n and denote the α-thinning of f by Tα(f). Then, as n → ∞, the entropy H(T1/n(f ∗n)) tends to H(po(λ)), where po(λ) denotes the pmf of the Poisson distribution with mean λ, and the relative entropy D(T1/n(f ∗n)|po(λ)) tends to zero, if it ever becomes finite. Moreover, α −1 D(Tα(f)|po(αλ)) increases in α ∈ (0,1), and n −1 D (f ∗n |po(nλ)) decreases in n = 1, 2,.... It follows that D(T1/n(f ∗n)|po(λ)) decreases monotonically in n. Furthermore, assuming that f is ultra-logconcave, we show that H(T1/n(f ∗n)) increases monotonically in n. This is a discrete analogue of the monotonicity of entropy considered by Artstein et al. (2004). A convergence rate is also obtained. If f is either ultra-log-concave or has finite support, then D(T1/n(f ∗n)|po(λ)) = O(n −2), as n → ∞. In general, our results extend the parallel between the information-theoretic central limit theorem and the information-theoretic law of small numbers explored by Kontoyiannis et al. (2005) and Harremoës et al. (2007, 2008). Ingredients in the proofs include convexity, majorization, and stochastic orders. Index Terms—binomial distribution; convex order; logarithmic Sobolev inequality; majorization; Poisson approximation; relative entropy; Schur-concavity; stochastic orders; thinning; ultra-logconcavity. I.

Abstract—A version of the law of small numbers is analyzed in information-theoretic terms. Specifically, let f = {fi, i = 0, 1,...} be a probability mass function (pmf) on nonnegative integers with mean λ < ∞. Denote the nth convolution of f by f ∗n and denote the α-thinning of f by Tα(f). Then, as n → ∞, the entropy H(T1/n(f ∗n)) tends to H(po(λ)), where po(λ) denotes the pmf of the Poisson distribution with mean λ, and the relative entropy D(T1/n(f ∗n)|po(λ)) tends to zero, if it ever becomes finite. Moreover, α −1 D(Tα(f)|po(αλ)) increases in α ∈ (0, 1), and n −1 D (f ∗n |po(nλ)) decreases in n = 1,2,.... It follows that D(T1/n(f ∗n)|po(λ)) decreases monotonically in n. Furthermore, assuming that f is ultra-log-concave (i.e., logconcave relative to the Poisson pmf), we show that H(T1/n(f ∗n)) increases monotonically in n. This is a discrete analogue of the monotonicity of entropy considered by Artstein et al. (2004). A convergence rate is also obtained. If f is either ultra-log-concave or has finite support, then D(T1/n(f ∗n)|po(λ)) = O(n −2), as n → ∞. In general, our results extend the parallel between the information-theoretic central limit theorem and the informationtheoretic law of small numbers explored by Kontoyiannis et al. (2005) and Harremoës et al. (2007, 2008). Ingredients in the proofs include convexity, majorization, and stochastic orders. Index Terms—binomial distribution; convex order; logarithmic Sobolev inequality; majorization; Poisson approximation; relative entropy; Schur-concavity; stochastic orders; thinning; ultra-logconcavity. I.

A new approach to Poisson approximation is proposed. The basic idea is very simple and based on properties of the Charlier polynomials and the Parseval identity. Such an approach quickly leads to new effective bounds for several Poisson approximation problems. A selected survey on diverse Poisson approximation results is also given.

to a proposed Stam inequality on finite groups Venkat Anantharam, Fellow, IEEE, Abstract — Gibilisco and Isola have recently proposed a definition of Fisher information for random variables taking values in a finite group that is analogous to the definition for real valued random variables with a density. Based on this Fisher information concept, they claim to prove a Stam inequality for finite-group valued random variables that is analogous to the one in the case of real values. In this note we show these results, unfortunately, do not hold for non-abelian groups in general, by constructing explicit counterexamples. Finite group, Fisher information, Group-valued random variables, Stam inequality.

Abstract.—Estimation of divergence times is usually done using either the fossil record or sequence data from modern species. We provide an integrated analysis of palaeontological and molecular data to give estimates of primate divergence times that utilize both sources of information. The number of preserved primate species discovered in the fossil record, along with their geological age distribution, is combined with the number of extant primate species to provide initial estimates of the primate and anthropoid divergence times. This is done by using a stochastic forwards-modeling approach where speciation and fossil preservation and discovery are simulated forward in time. We use the posterior distribution from the fossil analysis as a prior distribution on node ages in a molecular analysis. Sequence data from two genomic

Abstract—An “entropy increasing to the maximum ” result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic convergence in relative entropy is established for general discrete distributions, while monotonic increase of Shannon entropy is proved for the special class of ultra-log-concave distributions. Overall we extend the parallel between the information-theoretic central limit theorem and law of small numbers explored by Kontoyiannis et al. (2005) and Harremoës et al. (2007, 2008). Ingredients in the proofs include convexity, majorization, and stochastic orders. Index Terms—binomial thinning; convex order; logarithmic Sobolev inequality; majorization; Poisson approximation; relative entropy; Schur-concavity; ultra-log-concavity.

We consider the entropy of sums of independent discrete random variables, in analogy with Shannon’s Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for Poisson variables. We show that some natural analogues of the Entropy Power Inequality do not in fact hold, but propose an alternative formulation which does always hold. The key to many proofs of Shannon’s Entropy Power Inequality is the behaviour of entropy on scaling of continuous random variables. We believe that Rényi’s operation of thinning discrete random variables plays a similar role to scaling, and give a sharp bound on how the entropy of ultra log-concave random variables behaves on thinning. In the spirit of the monotonicity results established by Artstein, Ball, Barthe and Naor, we prove a stronger version of concavity of entropy, which implies a strengthened form of our discrete Entropy Power Inequality.

Abstract not found