This paper is concerned with algorithms for prediction of discrete sequences over a finite alphabet, using variable order Markov models. The class of such algorithms is large and in principle includes any lossless compression algorithm. We focus on six prominent prediction algorithms, including Context Tree Weighting (CTW), Prediction by Partial Match (PPM) and Probabilistic Suffix Trees (PSTs). We discuss the properties of these algorithms and compare their performance using real life sequences from three domains: proteins, English text and music pieces. The comparison is made with respect to prediction quality as measured by the average log-loss. We also compare classification algorithms based on these predictors with respect to a number of large protein classification tasks. Our results indicate that a “decomposed” CTW (a variant of the CTW algorithm) and PPM outperform all other algorithms in sequence prediction tasks. Somewhat surprisingly, a different algorithm, which is a modification of the Lempel-Ziv compression algorithm, significantly outperforms all algorithms on the protein classification problems.

Abstract not found

This paper is concerned with sequence classification using Markov chains when classification noise is included in the learning data. These models offer a direct generalization of a Multinomial Naive Bayes classifier by taking into account dependences between successive events up to a certain history length. Our study shows that smoothed Markov chains are very robust to classification noise. The relation between classification accuracy and test set perplexity, often used to measure prediction quality, is discussed. The influence of varying the model order is also studied from an experimental viewpoint. Experiments are conducted both on a gender classification task from spelling of first names and splicing region classification in DNA sequences. The first set of experiments also illustrate the superiority of smoothed Markov chains to classify noisy sequence over an automaton learning technique using boosting.

HaploRec: efficient and accurate large-scale reconstruction of haplotypes