## Unbounded Length Contexts for PPM (1995)

### Download From

IEEE### Download Links

- [cs.haifa.ac.il]
- [ftp.cs.waikato.ac.nz]
- [www1.oup.co.uk]
- [www.cs.waikato.ac.nz]
- [www.cs.waikato.ac.nz]
- [www.cs.waikato.ac.nz]
- [www.cs.waikato.ac.nz]
- DBLP

### Other Repositories/Bibliography

Venue: | The Computer Journal |

Citations: | 117 - 7 self |

### BibTeX

@INPROCEEDINGS{Cleary95unboundedlength,

author = {John G. Cleary and W. J. Teahan and Ian H. Witten},

title = {Unbounded Length Contexts for PPM},

booktitle = {The Computer Journal},

year = {1995},

pages = {52--61}

}

### Years of Citing Articles

### OpenURL

### Abstract

uses considerably greater computational resources (both time and space). The next section describes the basic PPM compression scheme. Following that we motivate the use of contexts of unbounded length, introduce the new method, and show how it can be implemented using a trie data structure. Then we give some results that demonstrate an improvement of about 6% over the old method. Finally, a recently-published and seemingly unrelated compression scheme [2] is related to the unbounded-context idea that forms the essential innovation of PPM*. 1 PPM: Prediction by partial match The basic idea of PPM is to use the last few characters in the input stream to predict the upcoming one. Models that condition their predictions on a few immediately preceding symbols are called "finite-context" models of order k, where k is the number of preceding symbols used. PPM employs a suite of fixed-order context models with different values of k

### Citations

1239 | A Universal Algorithm for Sequential Data Compression
- Ziv, Lempel
- 1977
(Show Context)
Citation Context ...the benchmark version [2]. This still achieves results superior to virtually all other compression methods, despite many attempts to better it. Other methods such as those based on Ziv--Lempel coding =-=[3, 4]-=- are more commonly used in practice, but their attractiveness lies in their relative speed rather than any superiority in compression---indeed, their compression performance generally falls distinctly... |

788 | Compression of individual sequences via variable-rate coding
- Ziv, Lempel
- 1978
(Show Context)
Citation Context ...ecome the benchmark version. This still achieves resuks superior to virtually all other compression methods, despite many attempts to better it. Other methods such as those based on Ziv-Lempel coding =-=[9]-=- are more commonly used in practice, but their attractiveness lles in their relative speed rather than any superiority in compression--indeed, their compression performauce generally falls distinctly ... |

700 |
Arithmetic coding for data compression
- Witten, Neal, et al.
- 1987
(Show Context)
Citation Context ...h context in the model are calculated from frequency counts which are updated adaptively, and the symbol that actually occurs is encoded relative to its predicted distribution using arithmetic coding =-=[6, 7]-=-. The maximum context length is a fixed constant, and it has been found that increasing it beyond about 5 does not generally improve compression [1, 2, 8]. The present paper 1 describes an algorithm, ... |

657 |
Text Compression
- Bell, Cleary, et al.
- 1990
(Show Context)
Citation Context ... attractiveness lles in their relative speed rather than any superiority in compression--indeed, their compression performauce generally falls distinctly below that of PPM in practical benchmark ests =-=[1]-=-. Prediction by' partial matching, or PPM, is a finite-context statistical modeling technique that can be viewed as blending together several fixed-order context models to predict the next character i... |

604 | A block-sorting lossless data compression algorithm
- Burrows, Wheeler
- 1994
(Show Context)
Citation Context ...mented using a trie data structure. Then we give some results that demonstrate an improvement of about 6% over the old method. Finally, a recently-published and seemingly unrelated conpression scheme =-=[2]-=- is related to the unbounded-context idea that forms the essential impovation of PPM*, 1 PPM: PI%EDICTION BY PAI%TIAL MATCH The basic idea of PPM is to use the last few characters in the input stream ... |

473 |
Algorithms
- Sedgewick
- 1983
(Show Context)
Citation Context ...ION ISSUES Using a PATRICIA-style trie. Substantial space can be saved in the context trie by collapsing non-branching sub-paths into single nodes, just like the standard PATRICIA trle data strncture =-=[7]-=-. For each collapsed node, only one branch emanates from it. In Figure 3 there are three such paths, two with the letters brac and the third with rac. Collapsing non-branching paths requires two extra... |

383 |
Prediction and entropy of printed English
- Shannon
- 1951
(Show Context)
Citation Context ...omputer model that can predict English text almost as well as humans. They performed experiments on the same text that Claude E. Shannon used in a famous experiment to estimate the entropy of English =-=[10]-=-, and found that performance was close to, and in some cases superior to, human-based results. It is also well-known in cryptography that removing redundancy is important prior to encryption to preven... |

362 | Data compression using adaptive coding and partial string matching - Cleary, Witten - 1984 |

348 | On-line construction of suffix tree
- Ukkonen
- 1995
(Show Context)
Citation Context ...ibed, inserting the next symbol into a PATRICIA trie can take time up to O(N ) (this worst case occurs when the input contains a string where the same symbol is repeated N times). As shown by Ukkonen =-=[15]-=-, the addition of `suffix THE COMPUTER JOURNAL, Vol. 40, No. 2/3, 1997 72 J. G. CLEARY AND W. J. TEAHAN [] a b c b c d r r a c a c ^ d r a c input string a b r a c a d a b r a [] [] [] [] [] trie poin... |

249 |
The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression
- Witten, Bell
- 1991
(Show Context)
Citation Context ...with the escape events. There is no sound theoretical basis for any particular choice in the absence of some a priori assumption on the nature of the symbol source; some alternatives are evaluated in =-=[8, 12]-=-. The method used in the example, commonly called `Method C', gives a count to the escape event equal to the number of different symbols that have been seen in the context so far [2]; thus, for exampl... |

181 |
A cache-base natural language model for speech recognition
- Kuhn, Mori
- 1990
(Show Context)
Citation Context ...classes of information that are not taken into account by PPM* (or by other related techniques). There is evidence, for example, that in English text there are words which show strong recency effects =-=[24]-=-. If the word occurs once then there is a high probability that it will occur again soon. It is unclear, at this time, what models and estimation techniques can effectively take advantage of such info... |

148 | Arithmetic coding revisited - Moffat, Neal, et al. - 1995 |

129 | Implementing the PPM data compression scheme
- Moffat
- 1990
(Show Context)
Citation Context ...sta,ndard in lossless compression of text throughout the past decade. The original algorithm was first published in 1984 by Cleary and Witten [3], and a series of improvements was described by Moffat =-=[6]-=-, culminating in a careful implementation, called PPMC, which has become the benchmark version. This still achieves resuks superior to virtually all other compression methods, despite many attempts to... |

66 |
Data Compression with Finite Windows
- Fiala, Greene
- 1989
(Show Context)
Citation Context ..., although convergence is very slow and in practice the method does not perform particularly well. Much effort has gone into devising improvements to the basic method, and one of the later ones, LZFG =-=[4]-=-, is not too far behind PPMC in compression performance (and greatly superior to it in speed). Very recently, a novel block-sorting algorithm has been described that achieves compression as good as co... |

55 | The design and analysis of efficient lossless data compression systems
- Howard
- 1993
(Show Context)
Citation Context ...-1) is used. Ve are presently using escape method C, which needs an extra "escape" count, equal o the number of branches, to be stored w(th every trio node. Other methods such as the ones pr=-=oposed in [5, 8]-=- my improve compression, although this remains to be investigated. Escaping is even more crucial to PPM* than it is to PPM. With a fixed maximum context length, use of the escape mechanism in PPM will... |

51 | Modelling English Text
- Teahan
- 1998
(Show Context)
Citation Context ...when it gives only one prediction. We have found in experiments that for such contexts the observed frequency of the novel characters is much lower than expected based on a uniform prior distribution =-=[13]-=-. This can be exploited by using such contexts for prediction. The strategy that we recommend is to choose the shortest deterministic context currently in the context list. If there is no deterministi... |

38 | Extended application of suffix trees to data compression
- Larsson
- 1996
(Show Context)
Citation Context ... 1 2 Order k = 1: # r 2 2 16 #bra # c 1 1 2 #a # b 2 2 7 # Esc 5 5 16 # Esc 1 1 2 # c 1 1 7 rac # a 1 1 2 # d 1 1 7 Order k = -1: # Esc 1 1 2 # Esc 3 3 7 # # A 1 1/|A| b # r 2 2 3 # Esc 1 1 3 Larsson =-=[16]-=- shows that Ukkonen's algorithm can be extended so that only a window of the most recent M inputs is used. Given that there will be at most 2M - 1 nodes, this permits the total memory used to be fixed... |

37 | The entropy of English using PPMbased models - Teahan, Cleary - 1996 |

25 | Semantically motivated improvements for PPM variants
- Bunton
- 1997
(Show Context)
Citation Context ...s predicted distribution using arithmetic coding [6, 7]. The maximum context length is a fixed constant, and it has been found that increasing it beyond about 5 does not generally improve compression =-=[1, 2, 8]-=-. The present paper 1 describes an algorithm, PPM*, which exploits contexts of unbounded length. It reliably achieves compression superior to the benchmark PPMC version, although our current implement... |

15 | On-Line Stochastic Processes in Data Compression
- Bunton
- 1996
(Show Context)
Citation Context ...utation time. It is unclear whether there exist effective probability estimation techniques that can avoid this O(N ) scan time. The use of suffix pointers for PPM* was inspired by the work of Bunton =-=[8, 17]-=-, which applies suffix trees as a common representation for a large class of compression techniques including PPM and PPM*. 3.2.4. Encoding the counts Prediction in PPM* is based upon the frequencies ... |

15 |
A note on the DMC data compression scheme
- Bell, Moffat
- 1989
(Show Context)
Citation Context ...some finite length of preceding context). This includes a very wide range of compression techniques. DMC is a technique originally described in [19] and has been shown to be a finite-context model in =-=[20]-=-. The archetypical compression schemes that use unbounded contexts are LZ77 and LZ78, which parse the text into non-overlapping phrases from the input [3, 4]. These schemes are known to be asymptotica... |

11 | The Design and Analysis of Ecient Lossless Data Compression Systems - Howard - 1993 |

11 | Experiments on the zero frequency problem - Cleary, Teahan - 1995 |

10 | Implementing the ppm data compression scheme - Moat - 1990 |

5 | Data compression with nite windows - Fiala, Greene - 1989 |

4 | Dynamic Markov modelling -- a prediction technique - Horspool, Cormack - 1986 |

3 | Chinks in the armor of public key cryptosystems
- Wilson
- 1994
(Show Context)
Citation Context ...rmance was close to, and in some cases superior to, human-based results. It is also well-known in cryptography that removing redundancy is important prior to encryption to prevent statistical attacks =-=[11]-=-. It is important here that there are no models (human or otherwise) that are significantly better than the model used to remove the redundancy. 2. PPM: PREDICTION BY PARTIAL MATCH The basic idea of P... |

3 | Dynamic Markov modelling---a prediction technique - Horspool, Cormack - 1986 |

2 |
Towards understanding and improving escape probabilities
- Aberg, Shtarkov, et al.
- 1997
(Show Context)
Citation Context ...on [8] shows a number of significant improvements to the escape mechanism used in this paper, and further improves performance with a new information-theoretic state-selection technique.sAberg et al. =-=[22]-=- also investigate several ways of improving the escape probabilities in PPM. THE COMPUTER JOURNAL, Vol. 40, No. 2/3, 1997 UNBOUNDED LENGTH CONTEXTS FOR PPM 75 Bloom [23] reports a result of 2.19 bpc f... |

1 |
The zero-frequency problem: estimating thc probabilities of novel events in adaptive text compression
- Wittcn, Bell
- 1991
(Show Context)
Citation Context ...a ss_L_)=11.2bits Table 2: Encodings br three sample characters using the model in Table the absence of some a priori assumption on the nature of the symbol source; some alternatives are evaluated in =-=[8]. The meth-=-od used in the example, commonly catled "Method C," gives a count to the escape event equal to the number of different symbols that have been seen in the context so far [6]; thus, for exampl... |

1 | On-line stochastic process in data compression - Bloom - 1996 |

1 | Unbounded length contexts for PPM - Witten - 1995 |

1 | Modelling English text - J - 1997 |