Results 1  10
of
15
Coordination Capacity
, 2009
"... We develop elements of a theory of cooperation and coordination in networks. Rather than considering a communication network as a means of distributing information, or of reconstructing random processes at remote nodes, we ask what dependence can be established among the nodes given the communicatio ..."
Abstract

Cited by 47 (17 self)
 Add to MetaCart
(Show Context)
We develop elements of a theory of cooperation and coordination in networks. Rather than considering a communication network as a means of distributing information, or of reconstructing random processes at remote nodes, we ask what dependence can be established among the nodes given the communication constraints. Specifically, in a network with communication rates {Ri,j} between the nodes, we ask what is the set of all achievable joint distributions p(x1,..., xm) of actions at the nodes on the network. Several networks are solved, including arbitrarily large cascade networks. Distributed cooperation can be the solution to many problems such as distributed games, distributed control, and establishing mutual information bounds on the influence of one part of a physical system on another.
COMMUNICATION IN NETWORKS FOR COORDINATING BEHAVIOR
, 2009
"... in my opinion, it ..."
(Show Context)
Optimal information storage: Nonsequential sources and neural channels
, 2006
"... Information storage and retrieval systems are communication systems from the present to the future and fall naturally into the framework of information theory. The goal of information storage is to preserve as much signal fidelity under resource constraints as possible. The information storage theor ..."
Abstract

Cited by 9 (6 self)
 Add to MetaCart
(Show Context)
Information storage and retrieval systems are communication systems from the present to the future and fall naturally into the framework of information theory. The goal of information storage is to preserve as much signal fidelity under resource constraints as possible. The information storage theorem delineates average fidelity and average resource values that are achievable and those that are not. Moreover, observable properties of optimal information storage systems and the robustness of optimal systems
Achieving the ratedistortion bound with lowdensity generator matrix codes
 IEEE Trans. Comm
, 2010
"... Abstract—It is shown that binary lowdensity generator matrix codes can achieve the ratedistortion bound of discrete memoryless sources with general distortion measure via multilevel quantization. A practical encoding scheme based on the surveypropagation algorithm is proposed. The effectiveness ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
(Show Context)
Abstract—It is shown that binary lowdensity generator matrix codes can achieve the ratedistortion bound of discrete memoryless sources with general distortion measure via multilevel quantization. A practical encoding scheme based on the surveypropagation algorithm is proposed. The effectiveness of the proposed scheme is verified through simulation. Index Terms—Ratedistortion bound, linear code, lowdensity generator matrix, messagepassing algorithm. I.
Approximations for the Entropy Rate of a Hidden Markov Process
"... Abstract—Let {Xt} be a stationary finitealphabet Markov chain and {Zt} denote its noisy version when corrupted by a discrete memoryless channel. We present an approach to bounding the entropy rate of {Zt} by the construction and study of a related measurevalued Markov process. To illustrate its ef ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
Abstract—Let {Xt} be a stationary finitealphabet Markov chain and {Zt} denote its noisy version when corrupted by a discrete memoryless channel. We present an approach to bounding the entropy rate of {Zt} by the construction and study of a related measurevalued Markov process. To illustrate its efficacy, we specialize it to the case of a BSCcorrupted binary Markov chain. The bounds obtained are sufficiently tight to characterize the behavior of the entropy rate in asymptotic regimes that exhibit a “concentration of the support”. Examples include the ‘high SNR’, ‘low SNR’, ‘rare spikes’, and ‘weak dependence’ regimes. Our analysis also gives rise to a deterministic algorithm for approximating the entropy rate, achieving the best known precisioncomplexity tradeoff, for a significant subset of the process parameter space. I.
Randomized Quantization and Source Coding With Constrained Output Distribution
"... Abstract — This paper studies fixedrate randomized vector quantization under the constraint that the quantizer’s output has a given fixed probability distribution. A general representation of randomized quantizers that includes the common models in the literature is introduced via appropriate mixtu ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Abstract — This paper studies fixedrate randomized vector quantization under the constraint that the quantizer’s output has a given fixed probability distribution. A general representation of randomized quantizers that includes the common models in the literature is introduced via appropriate mixtures of joint probability measures on the product of the source and reproduction alphabets. Using this representation and results from optimal transport theory, the existence of an optimal (minimum distortion) randomized quantizer having a given output distribution is shown under various conditions. For sources with densities and the mean square distortion measure, it is shown that this optimum can be attained by randomizing quantizers having convex codecells. For stationary and memoryless source and output distributions, a ratedistortion theorem is proved, providing a singleletter expression for the optimum distortion in the limit of large blocklengths. Index Terms — Source coding, quantization, randomization, random coding, outputconstrained distortionrate function.
Joint universal lossy coding and identification of stationary mixing sources with general alphabets
 IEEE Trans. Inform. Theory
"... Abstract — We consider the problem of joint universal variablerate lossy coding and identification for parametric classes of stationary βmixing sources with general (Polish) alphabets. Compression performance is measured in terms of Lagrangians, while identification performance is measured by the ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Abstract — We consider the problem of joint universal variablerate lossy coding and identification for parametric classes of stationary βmixing sources with general (Polish) alphabets. Compression performance is measured in terms of Lagrangians, while identification performance is measured by the variational distance between the true source and the estimated source. Provided that the sources are mixing at a sufficiently fast rate and satisfy certain smoothness and Vapnik–Chervonenkis learnability conditions, it is shown that, for bounded metric distortions, there exist universal schemes for joint lossy compression and identification whose Lagrangian redundancies converge to zero as p Vn log n/n as the block length n tends to infinity, where Vn is the Vapnik–Chervonenkis dimension of a certain class of decision regions defined by the ndimensional marginal distributions of the sources; furthermore, for each n, the decoder can identify ndimensional marginal of the active source up to a ball of radius O ( p Vn log n/n) in variational distance, eventually with probability one. The results are supplemented by several examples of parametric sources satisfying the regularity conditions. Index Terms—Learning, minimumdistance density estimation, twostage codes, universal vector quantization, Vapnik– Chervonenkis dimension. I.
Near Optimal Lossy Source Coding and CompressionBased Denoising via Markov Chain
"... Abstract — We propose an implementable new universal lossy source coding algorithm. The new algorithm utilizes two wellknown tools from statistical physics and computer science: Gibbs sampling and simulated annealing. In order to code a source sequence x n, the encoder initializes the reconstruction ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract — We propose an implementable new universal lossy source coding algorithm. The new algorithm utilizes two wellknown tools from statistical physics and computer science: Gibbs sampling and simulated annealing. In order to code a source sequence x n, the encoder initializes the reconstruction block as ˆx n = x n, and then at each iteration uniformly at random chooses one of the symbols of ˆx n, and updates it. This updating is based on some conditional probability distribution which depends on a parameter β representing inverse temperature, an integer parameter k = o(log n) representing context length, and the original source sequence. At the end of this process, the encoder outputs the LempelZiv description of ˆx n, which the decoder deciphers perfectly, and sets as its reconstruction. The complexity of the proposed algorithm in each iteration is linear in k and independent of n. We prove that, for any stationary ergodic source, the algorithm achieves the optimal ratedistortion performance asymptotically in the limits of large number of iterations, β, and n. We also show how our approach carries over to such problems as universal WynerZiv coding and compressionbased denoising. I.
Denoising of Quality Scores for Boosted Inference and Reduced Storage
"... Massive amounts of sequencing data are being generated thanks to advances in sequencing technology and a dramatic drop in the sequencing cost. Much of the raw data are comprised of nucleotides and the corresponding quality scores that indicate their reliability. The latter are more difficult to comp ..."
Abstract
 Add to MetaCart
Massive amounts of sequencing data are being generated thanks to advances in sequencing technology and a dramatic drop in the sequencing cost. Much of the raw data are comprised of nucleotides and the corresponding quality scores that indicate their reliability. The latter are more difficult to compress and are themselves noisy. Lossless and lossy compression of the quality scores has recently been proposed to alleviate the storage costs, but reducing the noise in the quality scores has remained largely unexplored. This raw data is processed in order to identify variants; these genetic variants are used in important applications, such as medical decision making. Thus improving the performance of the variant calling by reducing the noise contained in the quality scores is important. We propose a denoising scheme that reduces the noise of the quality scores and we demonstrate improved inference with this denoised data. Specifically, we show that replacing the quality scores with those generated by the proposed denoiser results in more accurate variant calling in general. Moreover, a consequence of the denoising is that the entropy of the produced quality scores is smaller, and thus significant compression can be achieved with respect to lossless compression of the original quality scores. We expect our results to provide a baseline for future research in denoising of quality scores. The code used in this work as well as a Supplement with all the results are available at