Results 1  10
of
15
Asymptotic Properties Of Data Compression And Suffix Trees
 IEEE Trans. Inform. Theory
, 1993
"... Recently, Wyner and Ziv have proved that the typical length of a repeated subword found within the first n positions of a stationary ergodic sequence is (1=h) log n in probability where h is the entropy of the alphabet. This finding was used to obtain several insights into certain universal data com ..."
Abstract

Cited by 40 (11 self)
 Add to MetaCart
Recently, Wyner and Ziv have proved that the typical length of a repeated subword found within the first n positions of a stationary ergodic sequence is (1=h) log n in probability where h is the entropy of the alphabet. This finding was used to obtain several insights into certain universal data compression schemes, most notably the LempelZiv data compression algorithm. Wyner and Ziv have also conjectured that their result can be extended to a stronger almost sure convergence. In this paper, we settle this conjecture in the negative in the so called right domain asymptotic, that is, during a dynamic phase of expanding the data base. We prove  under an additional assumption involving mixing conditions  that the length of a typical repeated subword oscillates almost surely (a.s.) between (1=h 1 ) log n and (1=h 2 ) log n where 0 ! h 2 ! h h 1 ! 1. We also show that the length of the nth block in the LempelZiv parsing algorithm reveals a similar behavior. We relate our findings to...
An Implementable Lossy Version of the LempelZiv Algorithm  Part I: Optimality. . . Optimality for Memoryless Sources
, 1998
"... A new lossy variant of the FixedDatabase LempelZiv coding algorithm for encoding at a fixed distortion level is proposed, and its asymptotic optimality and universality for memoryless sources (with respect to bounded singleletter distortion measures) is demonstrated: As the database size m increa ..."
Abstract

Cited by 27 (8 self)
 Add to MetaCart
A new lossy variant of the FixedDatabase LempelZiv coding algorithm for encoding at a fixed distortion level is proposed, and its asymptotic optimality and universality for memoryless sources (with respect to bounded singleletter distortion measures) is demonstrated: As the database size m increases to infinity, the expected compression ratio approaches the ratedistortion function. The complexity and redundancy characteristics of the algorithm are comparable to those of its lossless counterpart. A heuristic argument suggests that the redundancy is of order (log log m)= log m, and this is also confirmed experimentally; simulation results are presented that agree well with this rate. Also, the complexity of the algorithm is seen to be comparable to that of the corresponding lossless scheme. We show that there is a tradeoff between compression performance and encoding complexity, and we discuss how the relevant parameters can be chosen to balance this tradeoff in practice. We also d...
Source coding, large deviations, and approximate pattern matching
 IEEE Trans. Inform. Theory
, 2002
"... Dedicated to the memory of Aaron Wyner, a valued friend and colleague. Abstract—In this review paper, we present a development of parts of ratedistortion theory and patternmatching algorithms for lossy data compression, centered around a lossy version of the asymptotic equipartition property (AEP) ..."
Abstract

Cited by 23 (10 self)
 Add to MetaCart
Dedicated to the memory of Aaron Wyner, a valued friend and colleague. Abstract—In this review paper, we present a development of parts of ratedistortion theory and patternmatching algorithms for lossy data compression, centered around a lossy version of the asymptotic equipartition property (AEP). This treatment closely parallels the corresponding development in lossless compression, a point of view that was advanced in an important paper of Wyner and Ziv in 1989. In the lossless case, we review how the AEP underlies the analysis of the Lempel–Ziv algorithm by viewing it as a random code and reducing it to the idealized Shannon code. This also provides information about the redundancy of the Lempel–Ziv algorithm and about the asymptotic behavior of several relevant quantities. In the lossy case, we give various versions of the statement of the generalized AEP and we outline the general methodology of its proof via large deviations. Its relationship with Barron and Orey’s generalized AEP is also discussed. The lossy AEP is applied to i) prove strengthened versions of Shannon’s direct sourcecoding theorem and universal coding theorems; ii) characterize the performance of “mismatched ” codebooks in lossy data compression; iii) analyze the performance of patternmatching algorithms for lossy compression (including Lempel–Ziv schemes); and iv) determine the firstorder asymptotic of waiting times between stationary processes. A refinement to the lossy AEP is then presented, and it is used to i) prove secondorder (direct and converse) lossy sourcecoding theorems, including universal coding theorems; ii) characterize which sources are quantitatively easier to compress; iii) determine the secondorder asymptotic of waiting times between stationary processes; and iv) determine the precise asymptotic behavior of longest matchlengths between stationary processes. Finally, we discuss extensions of the above framework and results to random fields. Index Terms—Data compression, large deviations, patternmatching, ratedistortion theory.
The Asymptotics of Waiting Times Between Stationary Processes, Allowing Distortion
 ANN. APPL. PROBAB
, 1999
"... ..."
Sequential Neural Text Compression
, 1996
"... The purpose of this paper is to show that neural networks may be promising tools for data compression without loss of information. We combine predictive neural nets and statistical coding techniques to compress text les. We apply our methods to certain short newspaper articles and obtain compression ..."
Abstract

Cited by 21 (6 self)
 Add to MetaCart
The purpose of this paper is to show that neural networks may be promising tools for data compression without loss of information. We combine predictive neural nets and statistical coding techniques to compress text les. We apply our methods to certain short newspaper articles and obtain compression ratios exceeding those of widely used LempelZiv algorithms (which build the basis of the UNIX functions "compress" and "gzip"). The main disadvantage of our methods is that they are about three orders of magnitude slower than standard methods.
Neural Predictors For Detecting And Removing Redundant Information
 IN ADAPTIVE BEHAVIOR AND LEARNING
, 1998
"... The components of most realworld patterns contain redundant information. However, most pattern classifiers (e.g., statistical classifiers and neural nets) work better if pattern components are nonredundant. I present various unsupervised nonlinear predictorbased "neural" learning algorithms that t ..."
Abstract

Cited by 6 (4 self)
 Add to MetaCart
The components of most realworld patterns contain redundant information. However, most pattern classifiers (e.g., statistical classifiers and neural nets) work better if pattern components are nonredundant. I present various unsupervised nonlinear predictorbased "neural" learning algorithms that transform patterns and pattern sequences into less redundant patterns without loss of information. The first part of the paper shows how a neural predictor can be used to remove redundant information from input sequences. Experiments with artificial sequences demonstrate that certain supervised classification techniques can greatly benefit from this kind of unsupervised preprocessing. In the second part of the paper, a neural predictor is used to remove redundant information from natural text. With certain short newspaper articles, the neural method can achieve better compression ratios than the widely used asymptotically optimal LempelZiv string compression algorithm. The third part of the ...
Optimal Lossless Compression of a Class of Dynamic Sources
 Proc Data Compression Conference, edited by J.A. Storer and J.H. Reif. IEEE Computer Society Press, Los Alamitos, CA
, 1997
"... . The usual assumption for proofs of the optimality of lossless encoding is a stationary ergodic source. Dynamic sources with nonstationary probability distributions occur in many practical situations where the data source is constructed by a composition of distinct sources, for example, a document ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
. The usual assumption for proofs of the optimality of lossless encoding is a stationary ergodic source. Dynamic sources with nonstationary probability distributions occur in many practical situations where the data source is constructed by a composition of distinct sources, for example, a document with multiple authors, a multimedia document, or the composition of distinct packets sent over a communication channel. There is a vast literature of adaptive methods used to tailor the compression to dynamic sources. However, little is known about optimal or near optimal methods for lossless compression of strings generated by sources that are not stationary ergodic. Here we do not assume the source is stationary. Instead we assume that the source produces an infinite sequence of concatenated finite strings s 1 ; s 2 ; : : : where (i) each finite string s i is generated by a sampling of a (possibly distinct) stationary ergodic source S i , and (ii) the length of each of the s i is lower b...
Pattern Matching and Lossy Data Compression on Random Fields
, 2001
"... We consider the problem of lossy data compression for data arranged on twodimensional arrays (such as images), or more generally on higherdimensional arrays (such as video sequences). Several of the most commonly used algorithms are based on pattern matching: Given a distortion level D and a block ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
We consider the problem of lossy data compression for data arranged on twodimensional arrays (such as images), or more generally on higherdimensional arrays (such as video sequences). Several of the most commonly used algorithms are based on pattern matching: Given a distortion level D and a block of data to be compressed, the encoder rst nds a D close match of this block into some database, and then describes the position of the match. We consider two idealized versions of this scenario. In the rst one, the database is taken to be a collection of independent realizations of the same size and from the same distribution as the original data. In the second, the database is assumed to be a single long realization from the same source as the data. We show that the compression rate achieved (in either version) is no worse than R(D=2) bits per symbol, where R(D) is the ratedistortion function. This is proved under the assumption that the data is generated by a Gibbs distribution, and it generalizes the corresponding onedimensional bound of Steinberg and Gutman. Using recent large deviations results by Dembo and Kontoyiannis and by Chi, we are able to give short proofs for the present results.
Predictive Coding With Neural Nets: Application To Text Compression
 ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 7
, 1995
"... To compress text files, a neural predictor network P is used to approximate the conditional probability distribution of possible "next characters", given n previous characters. P 's outputs are fed into standard coding algorithms that generate short codes for characters with high predicted probabili ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
To compress text files, a neural predictor network P is used to approximate the conditional probability distribution of possible "next characters", given n previous characters. P 's outputs are fed into standard coding algorithms that generate short codes for characters with high predicted probability and long codes for highly unpredictable characters. Tested on short German newspaper articles, our method outperforms widely used LempelZiv algorithms (used in UNIX functions such as "compress" and "gzip").
Lossy Compression in NearLinear Time via Efficient Random Codebooks and Databases
, 904
"... The compressioncomplexity tradeoff of lossy compression algorithms that are based on a random codebook or a random database is examined. Motivated, in part, by recent results of GuptaVerdúWeissman (GVW) and their underlying connections with the patternmatching scheme of Kontoyiannis ’ lossy Lem ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
The compressioncomplexity tradeoff of lossy compression algorithms that are based on a random codebook or a random database is examined. Motivated, in part, by recent results of GuptaVerdúWeissman (GVW) and their underlying connections with the patternmatching scheme of Kontoyiannis ’ lossy LempelZiv algorithm, we introduce a nonuniversal version of the lossy LempelZiv method (termed LLZ). The optimality of LLZ for memoryless sources is established, and its performance is compared to that of the GVW divideandconquer approach. Experimental results indicate that the GVW approach often yields better compression than LLZ, but at the price of much higher memory requirements. To combine the advantages of both, we introduce a hybrid algorithm (HYB) that utilizes both the divideandconquer idea of GVW and the singledatabase structure of LLZ. It is proved that HYB shares with GVW the exact same ratedistortion performance and implementation complexity, while, like LLZ, requiring less memory, by a factor which may become unbounded, depending on the choice or the relevant design parameters. Experimental results are also presented, illustrating the performance of all three methods on data generated by simple discrete memoryless sources. In particular, the HYB algorithm is shown to outperform existing schemes for the compression of some simple discrete sources with respect to the Hamming distortion criterion.