We provide a tutorial on arithmetic coding, showing how it provides nearly optimal data compression and how it can be matched with almost any probabilistic model. We indicate the main disadvantage of arithmetic coding, its slowness, and give the basis of a fast, space-efficient, approximate arithmetic coder with only minimal loss of compression efficiency. Our coder is based on the replacement of arithmetic by table lookups coupled with a new deterministic probability estimation scheme. Index terms : Data compression, arithmetic coding, adaptive modeling, analysis of algorithms, data structures, low precision arithmetic. 1 A similar version of this paper appears in Image and Text Compression, James A. Storer, ed., Kluwer Academic Publishers, Norwell, MA, 1992, 85--112. A shortened version of this paper appears in the proceedings of the International Conference on Advances in Communication and Control (COMCON 3), Victoria, British Columbia, Canada, October 16--18, 1991. 2 Support was...
|
3170
|
The mathematical theory of communication
– Shannon
- 1962
|
|
799
|
A universal algorithm for sequential data compression
– Ziv, Lempel
- 1977
|
|
613
|
A method for the construction of minimum-redundancy codes
– Huffman
- 1952
|
|
536
|
Text Compression
– Bell, Cleary, et al.
- 1990
|
|
515
|
Compression of individual sequences via variable-rate coding
– Ziv, Lempel
- 1978
|
|
288
|
A universal prior for integers and estimation by minimum description length. Annals of Statistics
– Rissanen
- 1983
|
|
229
|
Universal codeword sets and representation of the integers
– Elias
- 1975
|
|
213
|
Probabilistic counting algorithms for data base applications
– Flajolet, Martin
- 1985
|
|
202
|
Universal coding, information, prediction, and estimation
– Rissanen
- 1984
|
|
151
|
Arithmetic coding
– Langdon
- 1979
|
|
111
|
A locally adaptive data compression scheme
– Bentley, Sleator, et al.
- 1986
|
|
108
|
Information Theory and Coding
– Abramson
- 1963
|
|
93
|
Universal modeling and coding
– Rissanen, Langdon
- 1981
|
|
92
|
Implementing the PPM Data Compression Scheme
– Moffat
- 1990
|
|
83
|
Variations on a theme by Huffman
– Gallager
- 1978
|
|
81
|
Dynamic Huffman coding
– Knuth
- 1985
|
|
78
|
An Overview of the Basic Principles of the Q-Coder Adaptive Binary Arithmetic
– Pennebaker, Mitchell
- 1988
|
|
72
|
Generalized Kraft Inequality and Arithmetic Coding
– Rissanen
- 1976
|
|
65
|
The design and analysis of dynamic huffman codes
– Vitter
- 1987
|
|
61
|
R.N.Horspool, `Data Compression Using Dynamic Markov Modeling
– Cormack
- 1987
|
|
43
|
Compression of black-white images with arithmetic coding
– Langdon, Rissanen
- 1981
|
|
41
|
Source Coding Algorithm for Fast Data Compression
– Pasco
- 1976
|
|
34
|
An Adaptive System for Data Compression
– Faller
- 1973
|
|
31
|
Approximate counting: a detailed analysis
– Flajolet
- 1985
|
|
30
|
Analysis of Arithmetic Coding for Data Compression
– Howard, Vitter
- 1992
|
|
29
|
Interval and recency rank source coding: two on-line adaptive variable-length schemes
– Elias
- 1987
|
|
29
|
New methods for lossless image compression using arithmetic coding
– Howard, Vitter
- 1992
|
|
25
|
A note on the Ziv-Lempel model for compressing individual sequences
– Langdon
- 1983
|
|
25
|
Probability estimation for the Q-coder
– Pennebaker, Mitchell
- 1988
|
|
24
|
Bounds on the redundancy of Huffman codes
– Capocelli, Giancarlo, et al.
- 1986
|
|
24
|
Arithmetic stream coding using fixed precision registers
– Rubin
- 1979
|
|
23
|
The zero frequency problem: Estimating the probabilities of novel events in adaptive text compression
– Witten, Bell
- 1991
|
|
22
|
Design and analysis of fast text compression based on quasi-ari thmetic coding
– Howard, Vitter
- 1993
|
|
20
|
A Multiplication-Free Multialphabet Arithmetic Code
– Rissanen, Mohiuddin
- 1989
|
|
16
|
Optimal Hardware and Software Arithmetic Coding Procedures for the Q-Coder
– Mitchell, Pennebaker
- 1988
|
|
16
|
Software Implementations of the Q-coder
– Mitchell, Pennebaker
- 1988
|
|
15
|
A Multi-Purpose VLSI Chip for Adaptive Data Compression of Bilevel Images
– Arps, Truong, et al.
- 1988
|
|
15
|
A note on the DMC data compression scheme
– Bell, Moffat
- 1989
|
|
14
|
for Adaptive Huffman Codes
– Cormack, Horspool
- 1984
|
|
13
|
Estimating a probability using finite memory
– Leighton, Rivest
- 1986
|
|
13
|
Variations on a theme by Ziv and Lempel
– Miller, Wegman
- 1985
|
|
13
|
Conditions for optimality of the Huffman algorithm
– Parker
- 1980
|
|
12
|
A general minimum-redundancy source-coding algorithm
– Guazzo
- 1980
|
|
12
|
Modeling by Shortest Data Description," Automatica 14
– Rissanen
- 1978
|
|
11
|
Dynamic Huffman coding
– Vitter
- 1989
|
|
10
|
Arithmetic Coding For Data
– Witten, Neal, et al.
- 1987
|
|
9
|
The Rényi redundancy of generalized Huffman codes
– Blumer, McEliece
- 1988
|
|
9
|
Linear time adaptive arithmetic coding
– Moffat
- 1990
|
|
7
|
A unifying theory and improvements for existing approaches to text compression
– Bell
- 1987
|
|
7
|
High efficiency, multiplication free approximation of arithmetic coding
– Chevion, Karnin
- 1991
|
