Results 1  10
of
38
A general framework for codes involving redundancy minimization
 IEEE Transactions on Information Theory
, 2006
"... Abstract — A framework with two scalar parameters is introduced for various problems of finding a prefix code minimizing a coding penalty function. The framework involves a twoparameter class encompassing problems previously proposed by Huffman [1], Campbell [2], Nath [3], and Drmota and Szpankowsk ..."
Abstract

Cited by 9 (6 self)
 Add to MetaCart
(Show Context)
Abstract — A framework with two scalar parameters is introduced for various problems of finding a prefix code minimizing a coding penalty function. The framework involves a twoparameter class encompassing problems previously proposed by Huffman [1], Campbell [2], Nath [3], and Drmota and Szpankowski [4]. It sheds light on the relationships among these problems. In particular, Nath’s problem can be seen as bridging that of Huffman with that of Drmota and Szpankowski. This leads to a lineartime algorithm for the last of these with a solution that solves a range of Nath subproblems. We find simple bounds and lineartime Huffmanlike optimization algorithms for all nontrivial problems within the class.
Minimum Expected Length of FixedtoVariable Lossless Compression of Memoryless Sources
"... Abstract—Conventional wisdom states that the minimum expected length for fixedtovariable length encoding of an nblock memoryless source with entropy H grows as nH+O(1). However, this performance is obtained under the constraint that the code assigned to the whole nblock is a prefix code. Droppin ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
Abstract—Conventional wisdom states that the minimum expected length for fixedtovariable length encoding of an nblock memoryless source with entropy H grows as nH+O(1). However, this performance is obtained under the constraint that the code assigned to the whole nblock is a prefix code. Dropping this unnecessary constraint we show that the minimum expected length grows as nH − 1 log n + O(1) 2 unless the source is equiprobable. I.
The minimum average code for finite memoryless monotone sources
 in Proc., IEEE Information Theory Workshop
, 2002
"... Abstract—The problem of selecting a code for finite monotone sources with x symbols is considered. The selection criterion is based on minimizing the average redundancy (called Minave criterion) instead of its maximum (i.e., Minimax criterion). The average probability distribution € x, whose associa ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
Abstract—The problem of selecting a code for finite monotone sources with x symbols is considered. The selection criterion is based on minimizing the average redundancy (called Minave criterion) instead of its maximum (i.e., Minimax criterion). The average probability distribution € x, whose associated Huffman code has the minimum average redundancy, is derived. The entropy of the average distribution (i.e.,
Tight Bounds on Profile Redundancy and Distinguishability
"... The minimax KLdivergence of any distribution from all distributions in a collection P has several practical implications. In compression, it is called redundancy and represents the least additional number of bits over the entropy needed to encode the output of any distribution in P. In online estim ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
(Show Context)
The minimax KLdivergence of any distribution from all distributions in a collection P has several practical implications. In compression, it is called redundancy and represents the least additional number of bits over the entropy needed to encode the output of any distribution in P. In online estimation and learning, it is the lowest expected logloss regret when guessing a sequence of random values generated by a distribution in P. In hypothesis testing, it upper bounds the largest number of distinguishable distributions in P. Motivated by problems ranging from population estimation to text classification and speech recognition, several machinelearning and informationtheory researchers have recently considered labelinvariant observations and properties induced by i.i.d. distributions. A sufficient statistic for all these properties is the data’s profile, the multiset of the number of times each data element appears. Improving on a sequence of previous works, we show that the redundancy of the collection of distributions induced over profiles by lengthn i.i.d. sequences is between 0.3 · n 1/3 and n 1/3 log 2 n, in particular, establishing its exact growth power. 1
Precise asymptotic analysis of the Tunstall code
 Proc. 2006 International Symposium on Information Theory (Seattle
"... A variabletofixed length encoder partitions the source string over an mary alphabet A into a concatenation of variablelength phrases. Each phrase except the last one is constrained to belong to a given dictionary D of source strings; the last phrase is a nonnull prefix of a dictionary entry. On ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
(Show Context)
A variabletofixed length encoder partitions the source string over an mary alphabet A into a concatenation of variablelength phrases. Each phrase except the last one is constrained to belong to a given dictionary D of source strings; the last phrase is a nonnull prefix of a dictionary entry. One common constraint on a dictionary is that it leads to a unique parsing of any string over A. We will assume that all dictionaries are uniquely parsable. It is convenient to represent a uniquely parsable dictionary by a complete parsing tree T, i.e., a tree in which every internal node has all m children nodes in the tree. The dictionary entries d ∈Dcorrespond to the leaves of parsing tree. The encoder represents each parsed string by the fixed length binary code word corresponding to its dictionary entry. If the dictionary D is has M entries, then the code word for each phrase has
Universal Coding on Infinite Alphabets: Exponentially Decreasing Envelopes
, 2008
"... This paper deals with the problem of universal lossless coding on a countable infinite alphabet. It focuses on some classes of sources defined by an envelope condition on the marginal distribution, namely exponentially decreasing envelope classes with exponent α. The minimax redundancy of exponentia ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
This paper deals with the problem of universal lossless coding on a countable infinite alphabet. It focuses on some classes of sources defined by an envelope condition on the marginal distribution, namely exponentially decreasing envelope classes with exponent α. The minimax redundancy of exponentially decreasing envelope 1 classes is proved to be equivalent to 4α log e log² n. Then a coding strategy is proposed, with a Bayes redundancy equivalent to the maximin redundancy. At last, an adaptive algorithm is provided, whose redundancy is equivalent to the minimax redundancy.
Optimal prefix codes for infinite alphabets with nonlinear costs
 IEEE Trans. Inf. Theory
, 2008
"... Abstract — Let P = {p(i)} be a measure of strictly positive probabilities on the set of nonnegative integers. Although the countable number of inputs prevents usage of the Huffman algorithm, there are nontrivial P for which known methods find a source code that is optimal in the sense of minimizing ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
Abstract — Let P = {p(i)} be a measure of strictly positive probabilities on the set of nonnegative integers. Although the countable number of inputs prevents usage of the Huffman algorithm, there are nontrivial P for which known methods find a source code that is optimal in the sense of minimizing expected codeword length. For some applications, however, a source code should instead minimize one of a family of nonlinear objective P functions, βexponential means, those of the form loga i p(i)an(i) , where n(i) is the length of the ith codeword and a is a positive constant. Applications of such minimizations include a novel problem of maximizing the chance of message receipt in singleshot communications (a < 1) and a previously known problem of minimizing the chance of buffer overflow in a queueing system (a> 1). This paper introduces methods for finding codes optimal for such exponential means. One method applies to geometric distributions, while another applies to distributions with lighter tails. The latter algorithm is applied to Poisson distributions and both are extended to alphabetic codes, as well as to minimizing maximum pointwise redundancy. The aforementioned application of minimizing the chance of buffer overflow is also considered. Index Terms — Communication networks, generalized entropies, generalized means, Golomb codes, Huffman algorithm,
Tight bounds on minimum maximum pointwise redundancy
 In Proceedings of the International Symposium on Information Theory
, 1944
"... Abstract — This paper presents new lower and upper bounds for the optimal compression of binary prefix codes in terms of the most probable input symbol, where compression efficiency is determined by the nonlinear codeword length objective of minimizing maximum pointwise redundancy. This objective re ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
Abstract — This paper presents new lower and upper bounds for the optimal compression of binary prefix codes in terms of the most probable input symbol, where compression efficiency is determined by the nonlinear codeword length objective of minimizing maximum pointwise redundancy. This objective relates to both universal modeling and Shannon coding, and these bounds are tight throughout the interval. The upper bounds also apply to a related objective, that of d th exponential redundancy. I.