@MISC{Reyes_covarianceand, author = {Matthew G. Reyes}, title = {Covariance and entropy in Markov random fields}, year = {} }

Share

OpenURL

Abstract

Abstract—We consider families of Markov random fields (MRFs) on an undirected graph using the exponential family representation. In earlier work [13] we proved that if the statistic that defines a family of MRFs is positively correlated, then the entropy is monotone decreasing in the exponential parameters. In this paper we address the converse, specifically within the context of the Ising model. The statistic for an edge is viewed as positive or negative as it favors similar or dissimilar values at the endpoints of the edge. We show that for an acyclic Ising model with no self statistics, the statistic is positively correlated regardless of the polarity of the edges. We further show that for a cyclic Ising model, the statistic is positively correlated if and only if the statistic is not frustrated; and that the entropy is monotone decreasing in the exponential parameters, if and only if the statistic is not frustrated. I. PREAMBLE In this paper we pick up the discussion started in [13] and continued in [14]; namely, examining the relationship between the statistic defining a family of Markov random fields (MRFs) and the behavior of information-theoretic quantities within that family of MRFs. In particular, we address the question of whether monotonicity of entropy over the family of MRFs implies that the statistic is positively correlated. The converse was shown in [13]. The interest in such questions arises from both engineering [15] and social science [16], [7] concerns. Let G = (V, E) be a graph. A family of exponential distributions is specified by a vector statistic t = (tij) defined on the endpoints of the edges E of the graph. 1 That is, for a given image x = {xi: i ∈ V} and each edge {i, j} ∈ E, the function tij: Xi×Xj − → R determines the contribution of the pair (xi, xj) to the probability of x. We say that X is Markov with respect to G, in that conditioning on a cutset renders separated subsets independent of one another [17]. The entire family of MRFs based on t is generated by introducing an exponential parameter θ = (θij) where for each edge {i, j}, θij scales the sensitivity of the distribution p(x) = p(x; θ) to the function tij. Specifically, if X is an MRF based on t with exponential parameter θ, the probability of an image x is p(x; θ) = exp { ∑ θijtij(xi, xj) − Φ(θ)}