Results 1  10
of
327
The ratedistortion function for source coding with side information at the decoder
 IEEE Trans. Inform. Theory
, 1976
"... AbstractLet {(X,, Y,J}r = 1 be a sequence of independent drawings of a pair of dependent random variables X, Y. Let us say that X takes values in the finite set 6. It is desired to encode the sequence {X,} in blocks of length n into a binary stream*of rate R, which can in turn be decoded as a seque ..."
Abstract

Cited by 756 (1 self)
 Add to MetaCart
AbstractLet {(X,, Y,J}r = 1 be a sequence of independent drawings of a pair of dependent random variables X, Y. Let us say that X takes values in the finite set 6. It is desired to encode the sequence {X,} in blocks of length n into a binary stream*of rate R, which can in turn be decoded as a sequence { 2k}, where zk E %, the reproduction alphabet. The average distorjion level is (l/n) cl = 1 E[D(X,,z&, where D(x, $ 2 0, x E I, 2 E J, is a preassigned distortion measure. The special assumption made here is that the decoder has access to the side information {Yk}. In this paper we determine the quantity R*(d). defined as the infimum of rates R such that (with E> 0 arbitrarily small and with suitably large n) communication is possible in the above setting at an average distortion level (as defined above) not exceeding d + E. The main result is that R*(d) = inf[Z(X,Z) Z(Y,Z)], where the infimum is with respect to all auxiliary random variables Z (which take values in a finite set 3) that satisfy: i) Y,Z conditiofally independent given X; ii) there exists a functionf: “Y x E +.%, such that E[D(X,f(Y,Z))] 5 d. Let Rx, y(d) be the ratedistortion function which results when the encoder as well as the decoder has access to the side information {Y,}. In nearly all cases it is shown that when d> 0 then R*(d)> Rx, y(d), so that knowledge of the side information at the encoder permits transmission of the {X,} at a given distortion level using a smaller transmission rate. This is in contrast to the situation treated by Slepian and Wolf [5] where, for arbitrarily accurate reproduction of {X,}, i.e., d = E for any E> 0, knowledge of the side information at the encoder does not allow a reduction of the transmission rate.
Clustering with Bregman Divergences
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2005
"... A wide variety of distortion functions are used for clustering, e.g., squared Euclidean distance, Mahalanobis distance and relative entropy. In this paper, we propose and analyze parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergence ..."
Abstract

Cited by 323 (50 self)
 Add to MetaCart
A wide variety of distortion functions are used for clustering, e.g., squared Euclidean distance, Mahalanobis distance and relative entropy. In this paper, we propose and analyze parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences. The proposed algorithms unify centroidbased parametric clustering approaches, such as classical kmeans and informationtheoretic clustering, which arise by special choices of the Bregman divergence. The algorithms maintain the simplicity and scalability of the classical kmeans algorithm, while generalizing the basic idea to a very large class of clustering loss functions. There are two main contributions in this paper. First, we pose the hard clustering problem in terms of minimizing the loss in Bregman information, a quantity motivated by ratedistortion theory, and present an algorithm to minimize this loss. Secondly, we show an explicit bijection between Bregman divergences and exponential families. The bijection enables the development of an alternative interpretation of an ecient EM scheme for learning models involving mixtures of exponential distributions. This leads to a simple soft clustering algorithm for all Bregman divergences.
On The Capacity Of Wireless Networks: The Relay Case
 in Proc. IEEE INFOCOM
, 2002
"... In [1], Gupta and Kumar determined the capacity of wireless networks under certain assumptions, among them pointtopoint coding, which excludes for example multiaccess and broadcast codes. In this paper, we consider essentially the same physical model of a wireless network under a different traffi ..."
Abstract

Cited by 195 (9 self)
 Add to MetaCart
In [1], Gupta and Kumar determined the capacity of wireless networks under certain assumptions, among them pointtopoint coding, which excludes for example multiaccess and broadcast codes. In this paper, we consider essentially the same physical model of a wireless network under a different traffic pattern, namely the relay traffic pattern, but we allow for arbitrarily complex network coding. In our model, there is only one active source/destination pair, while all other nodes assist this transmission. We show code constructions leading to achievable rates and derive upper bounds from the maxflow mincut theorem. It is shown that lower and upper bounds meet asymptotically as the number of nodes in the network goes to infinity, thus proving that the capacity of the wireless network with n nodes under the relay traffic pattern behaves like log n bits per second. This demonstrates also that network coding is essential: under the pointtopoint coding assumption considered in [1], the achievable rate is constant, independent of the number of nodes.
Adapting Multimedia Internet Content for Universal Access
 IEEE Transactions on Multimedia
, 1999
"... Content delivery over the Internet needs to address both the multimedia nature of the content and the capabilities of the diverse client platforms the content is being delivered to. We present a system that adapts multimedia Web documents to optimally match the capabilities of the client device requ ..."
Abstract

Cited by 170 (2 self)
 Add to MetaCart
Content delivery over the Internet needs to address both the multimedia nature of the content and the capabilities of the diverse client platforms the content is being delivered to. We present a system that adapts multimedia Web documents to optimally match the capabilities of the client device requesting it. This system has two key components: (1) A representation scheme called the InfoPyramid that provides a multimodal, multiresolution representation hierarchy for multimedia. (2) A customizer that selects the best content representation to meet the client capabilities while delivering the most value. We model the selection process as a resource allocation problem in a generalized ratedistortion framework. In this framework, we address the issue of both multiple media types in a Web document and multiple resource types at the client. We extend this framework to allow prioritization on the content items in a Web document. We illustrate our content adaptation technique with a web ...
Data compression and harmonic analysis
 IEEE Trans. Inform. Theory
, 1998
"... In this paper we review some recent interactions between harmonic analysis and data compression. The story goes back of course to Shannon’s R(D) theory... ..."
Abstract

Cited by 149 (24 self)
 Add to MetaCart
In this paper we review some recent interactions between harmonic analysis and data compression. The story goes back of course to Shannon’s R(D) theory...
The Method of Types
, 1998
"... The method of types is one of the key technical tools in Shannon Theory, and this tool is valuable also in other fields. In this paper, some key applications will be presented in sufficient detail enabling an interested nonspecialist to gain a working knowledge of the method, and a wide selection of ..."
Abstract

Cited by 102 (0 self)
 Add to MetaCart
The method of types is one of the key technical tools in Shannon Theory, and this tool is valuable also in other fields. In this paper, some key applications will be presented in sufficient detail enabling an interested nonspecialist to gain a working knowledge of the method, and a wide selection of further applications will be surveyed. These range from hypothesis testing and large deviations theory through error exponents for discrete memoryless channels and capacity of arbitrarily varying channels to multiuser problems. While the method of types is suitable primarily for discrete memoryless models, its extensions to certain models with memory will also be discussed. Index TermsArbitrarily varying channels, choice of decoder, counting approach, error exponents, extended type concepts, hypothesis testing, large deviations, multiuser problems, universal coding. I.
The Rate Loss in the WynerZiv Problem
 IEEE Trans. Inform. Theory
, 1996
"... The ratedistortion function for source coding with side information at the decoder (the "WynerZiv problem") is given in terms of an auxiliary random variable, which forms a Markov chain with the source and the side information. This Markov chain structure, typical to the solution of mult ..."
Abstract

Cited by 79 (15 self)
 Add to MetaCart
The ratedistortion function for source coding with side information at the decoder (the "WynerZiv problem") is given in terms of an auxiliary random variable, which forms a Markov chain with the source and the side information. This Markov chain structure, typical to the solution of multiterminal source coding problems, corresponds to a loss in coding rate with respect to the conditional ratedistortion function, i.e., to the case where the encoder is fully informed. We show that for difference (or balanced) distortion measures, this loss is bounded by a universal constant, which is the minimax capacity of a suitable additive noise channel. Furthermore, in the worst case this loss is equal to the maximin redundancy over the rate distortion function of the additive noise "test" channel. For example, the loss in the WynerZiv problem is less than 0:5 bit per sample in the squarederror distortion case, and it is less than 0:22 bits for a binary source with Hammingdistance. These resul...
Coding for Computing
 IEEE Transactions on Information Theory
, 1998
"... A sender communicates with a receiver who wishes to reliably evaluate a function of their combined data. We show that if only the sender can transmit, the number of bits required is a conditional entropy of a naturally defined graph. We also determine the number of bits needed when the communicators ..."
Abstract

Cited by 76 (0 self)
 Add to MetaCart
A sender communicates with a receiver who wishes to reliably evaluate a function of their combined data. We show that if only the sender can transmit, the number of bits required is a conditional entropy of a naturally defined graph. We also determine the number of bits needed when the communicators exchange two messages. 1 Introduction Let f be a function of two random variables X and Y . A sender PX knows X, a receiver PY knows Y , and both want PY to reliably determine f(X; Y ). How many bits must PX transmit? Embedding this communicationcomplexity scenario (Yao [22]) in the standard informationtheoretic setting (Shannon [17]), we assume that (1) f(X; Y ) must be determined for a block of many independent (X; Y )instances, (2) PX transmits after observing the whole block of X instances, (3) a vanishing block error probability is allowed, and (4) the problem's rate L f (XjY ) is the number of bits transmitted for the block, normalized by the number of instances. Two simple bou...
The Distributed KarhunenLoève Transform
 IEEE Trans. Inform. Theory
, 2003
"... The KarhunenLoeve transform (KLT) is a key element of many signal processing tasks, including approximation, compression, and classification. Many recent applications involve distributed signal processing where it is not generally possible to apply the KLT to the signal; rather, the KLT must be ..."
Abstract

Cited by 70 (12 self)
 Add to MetaCart
The KarhunenLoeve transform (KLT) is a key element of many signal processing tasks, including approximation, compression, and classification. Many recent applications involve distributed signal processing where it is not generally possible to apply the KLT to the signal; rather, the KLT must be approximated in a distributed fashion.