We present a technique for constructing random fields from a set of training samples. The learning paradigm builds increasingly complex fields by allowing potential functions, or features, that are supported by increasingly large subgraphs. Each feature has a weight that is trained by minimizing the Kullback-Leibler divergence between the model and the empirical distribution of the training data. A greedy algorithm determines how features are incrementally added to the field and an iterative scaling algorithm is used to estimate the optimal values of the weights. The random field models and techniques introduced in this paper differ from those common to much of the computer vision literature in that the underlying random fields are non-Markovian and have a large number of parameters that must be estimated. Relations to other learning approaches, including decision trees, are given. As a demonstration of the method, we describe its application to the problem of automatic word classifica...

Discontinuity-preserving Bayesian image restoration typically involves two Markov random fields: one representing the image intensities/gray levels to be recovered and another one signaling discontinuities/edges to be preserved. The usual strategy is to perform joint maximum a posteriori (MAP) estimation of the image and its edges, which requires the specification of priors for both fields. In this paper, instead of taking an edge prior, we interpret discontinuities (in fact their locations) as deterministic unknown parameters of the compound Gauss--Markov random field (CGMRF), which is assumed to model the intensities. This strategy should allow inferring the discontinuity locations directly from the image with no further assumptions. However, an additional problem emerges: The number of parameters (edges) is unknown. To deal with it, we invoke the minimum description length (MDL) principle; according to MDL, the best edge configuration is the one that allows the shortest description of the image and its edges. Taking the other model parameters (noise and CGMRF variances) also as unknown, we propose a new unsupervised discontinuity-preserving image restoration criterion. Implementation is carried out by a continuation-type iterative algorithm which provides estimates of the number of discontinuities, their locations, the noise variance, the original image variance, and the original image itself (restored image). Experimental results with real and synthetic images are reported.

Abstract—Hyperspectral sensors collect hundreds of narrow and contiguously spaced spectral bands of data. Such sensors provide fully registered high resolution spatial and spectral images that are invaluable in discriminating between man-made objects and natural clutter backgrounds. The price paid for this high resolution data is extremely large data sets, several hundred of Mbytes for a single scene, that make storage and transmission difficult, thus requiring fast onboard processing techniques to reduce the data being transmitted. Attempts to apply traditional maximum likelihood detection techniques for in-flight processing of these massive amounts of hyperspectral data suffer from two limitations: first, they neglect the spatial correlation of the clutter by treating it as spatially white noise; second, their computational cost renders them prohibitive without significant data reduction like by grouping the spectral bands into clusters, with a consequent loss of spectral resolution. This paper presents a maximum likelihood detector that successfully confronts both problems: rather than ignoring the spatial and spectral correlations, our detector exploits them to its advantage; and it is computationally expedient, its complexity increasing only linearly with the number of spectral bands available. Our approach is based on a Gauss–Markov random field (GMRF) modeling of the clutter, which has the advantage of providing a direct parameterization of the inverse of the clutter covariance, the quantity of interest in the test statistic. We discuss in detail two alternative GMRF detectors: one based on a binary hypothesis approach, and the other on a ‘single ’ hypothesis formulation. We analyze extensively with real hyperspectral imagery data (HYDICE and SEBASS) the performance of the detectors, comparing them to a benchmark detector, the RX-algorithm. Our results show that the GMRF ‘single ’ hypothesis detector outperforms significantly in computational cost the RX-algorithm, while delivering noticeable detection performance improvement. Index Terms—Gauss–Markov random field, hyperspectral sensor imagery, maximum-likelihood detection, ‘single ’ hypothesis test. I.

Abstract—Hyperspectral sensors are passive sensors that simultaneously record images for hundreds of contiguous and narrowly spaced regions of the electromagnetic spectrum. Each image corresponds to the same ground scene, thus creating a cube of images that contain both spatial and spectral information about the objects and backgrounds in the scene. In this paper, we present an adaptive anomaly detector designed assuming that the background clutter in the hyperspectral imagery is a three-dimensional Gauss–Markov random field. This model leads to an efficient and effective algorithm for discriminating man-made objects (the anomalies) in real hyperspectral imagery. The major focus of the paper is on the adaptive stage of the detector, i.e., the estimation of the Gauss–Markov random field parameters. We develop three methods: maximum-likelihood; least squares; and approximate maximum-likelihood. We study these approaches along three directions: estimation error performance, computational cost, and detection performance. In terms of estimation error, we derive the Cramér–Rao bounds and carry out Monte Carlo simulation studies that show that the three estimation procedures have similar performance when the fields are highly correlated, as is often the case with real hyperspectral imagery. The approximate maximum-likelihood method has a clear advantage from the computational point of view. Finally, we test extensively with real hyperspectral imagery the adaptive anomaly detector incorporating either the least squares or the approximate maximum-likelihood estimators. Its performance compares very favorably with that of the RX algorithm, an alternative detector commonly used with multispectral data, while reducing by up to an order of magnitude the associated computational cost. Index Terms—Anomaly detection, Cramér–Rao bounds, Gauss– Markov random field, hyperspectral imagery, least squares, maximum

This paper proposes a new approach to model-based clustering under prior knowledge. The proposed formulation can be interpreted from two different angles: as penalized logistic regression, where the class labels are only indirectly observed (via the probability density of each class); as finite mixture learning under a grouping prior. To estimate the parameters of the proposed model, we derive a (generalized) EM algorithm with a closed-form E-step, in contrast with other recent approaches to semi-supervised probabilistic clustering which require Gibbs sampling or suboptimal shortcuts. We show that our approach is ideally suited for image segmentation: it avoids the combinatorial nature Markov random field priors, and opens the door to more sophisticated spatial priors (e.g., wavelet-based) in a simple and computationally efficient way. Finally, we extend our formulation to work in unsupervised, semi-supervised, or discriminative modes. 1

Abstract—This paper presents a distributed Kalman filter to estimate the state of a sparsely connected, large-scale,-dimensional, dynamical system monitored by a network of sensors. Local Kalman filters are implemented on-dimensional subsystems,, obtained by spatially decomposing the large-scale system. The distributed Kalman filter is optimal under an th order Gauss–Markov approximation to the centralized filter. We quantify the information loss due to this th-order approximation by the divergence, which decreases as increases. The order of the approximation leads to a bound on the dimension of the subsystems, hence, providing a criterion for subsystem selection. The (approximated) centralized Riccati and Lyapunov equations are computed iteratively with only local communication and low-order computation by a distributed iterate collapse inversion (DICI) algorithm. We fuse the observations that are common among the local Kalman filters using bipartite fusion graphs and consensus averaging algorithms. The proposed algorithm achieves full distribution of the Kalman filter. Nowhere in the network, storage, communication, or computation of-dimensional vectors and matrices is required; only dimensional vectors and matrices are communicated or used in the local computations at the sensors. In other words, knowledge of the state is itself distributed. Index Terms—Distributed algorithms, distributed estimation, information filters, iterative methods, Kalman filtering, large-scale systems, matrix inversion, sparse matrices. I.

Multiple views of a scene, obtained from cameras positioned at distinct viewpoints, can provide a viewer with the benefits of added realism, selective viewing, and improved scene understanding. The importance of these signals is evidenced by the recently proposed Multi-View Profile (MVP) extension to the MPEG-2 video compression standard, and their explicit incorporation into the future MPEG-4 standard. However, multi-view compression implementations typically rely on single-view image sequence model assumptions. We hypothesize (and demonstrate) that impressive system bandwidth reduction can be achieved by utilizing displacement vector field and image intensity models tuned to the special characteristics of multi-view video signals. This thesis focuses on the predictive coding of non-periodic, i.e., arbitrary, multi-view video signals for the applications of simulated motion parallax and viewer-specified degree of stereoscopy. To facilitate their practical use, we desire algorithms tha...

This paper considers the achievable accuracy in jointly estimating the parameters of a real valued two-dimensional homogeneous random field with mixed spectral distribution, from a single observed realization of it. On the basis of a 2-D Wold-like decomposition, the field is represented as a sum of mutually orthogonal components of three types: purelyindeterministic, harmonic, and evanescent. An exact form of the Cramer-Rao lower bound on the error variance in jointly estimating the parameters of the different components is derived. It is shown that the estimation of the harmonic component is decoupled from that of the purely-indeterministic and evanescent components. Moreover, the bound on the parameters of the purely-indeterministic and evanescent components is independent of the harmonic component. Numerical evaluation of the bounds provides some insight into the effects of various parameters on the achievable estimation accuracy.

: The Gibbs random field (GRF) has become a popular image model with applications in restoration, segmentation, reconstruction, edge detection, compression, and motion estimation. Its synthesis of natural-looking texture using only a small number of parameters is a key motivation for its widespread use. However, its wide use belies a number of difficulties inherent in the application of the model. In particular, it has proven difficult to control scale and patterning within the GRF framework, and to estimate parameters for a given pattern. The image processing literature has largely ignored the role of the temperature in the GRF, a parameter that appears in the original statistical mechanics formulation of the GRF. In applications such as simulated annealing, temperature is known to control scale, and in nature, temperature plays a critical role in multiresolution pattern formation, e.g., crystallization. Consequently, examination of GRF temperature parameters provides important insigh...

In this paper, we present a distributed algorithm to invert L−banded matrices that are symmetric positive definite (SPD), when the submatrices in the band are distributed among several processing nodes. We provide a distributed iterate-collapse inversion (DICI) algorithm that converges, at each node, to the corresponding submatrices in the inverse of the L−banded matrix. The computational complexity of the DICI algorithm to invert an SPD L−banded n × n matrix can be shown at each node to be independent of the size, n, of the matrix. Local information exchange is carried out after each iteration to guarantee convergence. We apply this algorithm to invert the information matrices in a computationally efficient distributed implementation of the Kalman filter and show its application towards inverting arbitrary sparse SPD matrices.