Over the last decade, arithmetic coding has emerged as an important compression tool. It is now the method of choice for adaptive coding on multisymbol alphabets because of its speed, low storage requirements, and effectiveness of compression. This article describes a new implementation of arithmetic coding that incorporates several improvements over a widely used earlier version by Witten, Neal, and Cleary, which has become a de facto standard. These improvements include fewer multiplicative operations, greatly extended range of alphabet sizes and symbol probabilities, and the use of low-precision arithmetic, permitting implementation by fast shift/add operations. We also describe a modular structure that separates the coding, modeling, and probability estimation components of a compression system. To motivate the improved coder, we consider the needs of a word-based text compression program. We report a range of experimental results using this and other models. Complete source code is available.

Arithmetic coding, in conjunction with a suitable probabilistic model, can provide nearly optimal data compression. In this article we analyze the effect that the model and the particular implementation of arithmetic coding have on the code length obtained. Periodic scaling is often used in arithmetic coding implementations to reduce time and storage requirements; it also introduces a recency effect which can further affect compression. Our main contribution is introducing the concept of weighted entropy and using it to characterize in an elegant way the effect that periodic scaling has on the code length. We explain why and by how much scaling increases the code length for files with a homogeneous distribution of symbols, and we characterize the reduction in code length due to scaling for files exhibiting locality of reference. We also give a rigorous proof that the coding effects of rounding scaled weights, using integer arithmetic, and encoding end-of-file are negligible.

We provide a tutorial on arithmetic coding, showing how it provides nearly optimal data compression and how it can be matched with almost any probabilistic model. We indicate the main disadvantage of arithmetic coding, its slowness, and give the basis of a fast, space-efficient, approximate arithmetic coder with only minimal loss of compression efficiency. Our coder is based on the replacement of arithmetic by table lookups coupled with a new deterministic probability estimation scheme.

The ability to predict the future based upon the past in finite-alphabet sequences has many applications, including communications, data security, pattern recognition, and natural language processing. By Shannon's theory and the breakthrough development of arithmetic coding, any sequence, a 1 a 2 \Delta \Delta \Delta a n , can be encoded in a number of bits that is essentially equal to the minimal information-lossless codelength, P i \Gamma log 2 p(a i ja 1 \Delta \Delta \Delta a i\Gamma1 ). The goal of universal on-line modeling, and therefore of universal data compression, is to deduce the model of the input sequence a 1 a 2 \Delta \Delta \Delta a n that can estimate each p(a i ja 1 \Delta \Delta \Delta a i\Gamma1 ) knowing only a 1 a 2 \Delta \Delta \Delta a i\Gamma1 so that the ex...

The problem of scalar quantization of certain memoryless sources with entropy coding is considered. The work is divided into two parts. In the first

We show that high-resolution images can be encoded and decoded efficiently in parallel. We present an algorithm based on the hierarchical MLP method, used either with Huffman coding or with a new variant of arithmetic coding called quasi-arithmetic coding. The coding step can be parallelized, even though the codes for different pixels are of different lengths; parallelization of the prediction and error modeling components is straightforward.

1 Multispectral images are composed of a series of images at differing optical wavelengths. Since these images can be quite large, they invite efficient source coding schemes for reducing storage and transmission requirements. Because multispectral images include a third (spectral) dimension with nonstationary behavior, these multilayer data sets require specialized coding techniques. In this paper, we develop both a theory and specific methods for performing optimal transform coding of multispectral images. The theory is based on the assumption that a multispectral image may be modeled as a set of jointly stationary Gaussian random processes. Therefore, the methods may be applied to any multilayer data set which meets this assumption. Although we do not assume the autocorrelation has a separable form, we show that the optimal transform for coding has a partially separable structure. In particular, we prove that a coding scheme consisting of a frequency transform within each layer foll...

Most data that is inherently discrete needs to be compressed in such a way that it can be recovered exactly, without any loss. Examples include text of all kinds, experimental results, and statistical databases. Other forms of data may need to be stored exactly, such as images---particularly bilevel ones, or ones arising in medical and remotesensing applications, or ones that may be required to be certified true for legal reasons. Moreover, during the process of lossy compression, many occasions for lossless compression of coefficients or other information arise. This paper surveys techniques for lossless compression. The process of compression can be broken down into modeling and coding. We provide an extensive discussion of coding techniques, and then introduce methods of modeling that are appropriate for text and images. Standard methods used in popular utilities (in the case of text) and international standards (in the case of images) are described. Keywords Text compression, ima...

Arithmetic coding is an efficient data compression technique. This paper describes the VLSI implementation of an arithmetic coder for a multilevel alphabet (256 symbols). The design we propose is based on the use of redundant arithmetic and the development of new schemes for storing and updating the cumulative probabilities and updating the range and left point of the interval. The use of redundant arithmetic reduces the delays of the modules, so the speed of the design it is improved. The resulting chip has an area of 31mm 2 and a operating frequency of 39 Mhz. 1. Introduction Arithmetic coding has become an important and efficient coding method [4] [5] [8]. There are two main features in this kind of coding. Firstly, it can be used to encode data strings modelled with both stationary and non stationary sources, and secondly, a good efficiency and the theoretical entropy bound can be achieved by using this code. Arithmetic coding gives better results than the Huffman method. In fac...

We present the design and performance of a non-adaptive hardware system for data compression by arithmetic coding. The alphabet of the data source is the full 256-symbol ASCII character set, plus a non-ASCII end-of-file symbol. The key ideas of our system are ffl the non-arithmetic representation of the current interval width, which yields improved coding efficiency in the interval-width update, and ffl a retimed circuit for the code point update, which removes this step from the critical path of the system's operation. Through a further retiming, the lower bound on this circuit's clock period can be reduced to a constant, independent of its width in bits. We have implemented and tested the system on a reconfigurable coprocessor, which is constructed from commercially available field-programmable gate arrays and static RAM. This implementation compresses its input stream at better than 16 MBytes/sec. 1 Introduction Arithmetic coding is a well-known method for lossless data compressi...