## Change-Point Estimation Using New Minimum Message Length Approximations (2002)

### Cached

### Download Links

- [www.csse.monash.edu.au]
- [www.cs.monash.edu.au]
- [www.csse.monash.edu.au]
- DBLP

### Other Repositories/Bibliography

Venue: | Proc. PRICAI |

Citations: | 5 - 2 self |

### BibTeX

@INPROCEEDINGS{Fitzgibbon02change-pointestimation,

author = {Leigh J. Fitzgibbon and David L. Dowe and Lloyd Allison},

title = {Change-Point Estimation Using New Minimum Message Length Approximations},

booktitle = {Proc. PRICAI},

year = {2002},

pages = {244--254},

publisher = {Springer}

}

### OpenURL

### Abstract

This paper investigates the coding of change-points in the information-theoretic Minimum Message Length (MML) framework. Changepoint coding regions affect model selection and parameter estimation in problems such as time series segmentation and decision trees. The Minimum Message Length (MML) and Minimum Description Length (MDL78) approaches to change-point problems have been shown to perform well by several authors. In this paper we compare some published MML and MDL78 methods and introduce some new MML approximations called `MMLDc' and `MMLDF'. These new approximations are empirically compared with Strict MML (SMML), Fairly Strict MML (FSMML), MML68, the Minimum Expected Kullback-Leibler Distance (MEKLD) loss function and MDL78 on a tractable binomial changepoint problem.

### Citations

311 |
An Information Measure for Classification
- Wallace, Boulton
- 1968
(Show Context)
Citation Context ...1 G=2 I0 = 8.893094 I1b = 9.195562 G=1 G=2 � � � 1 15-1 φ � 1 6-1 φ I0 = 6.605682 I1b = 6.916063 � � 1 11-1 φs3.3 MML68 Change-Point Approximation Oliver, Baxter and co-workers have app=-=lied the MML68 [1]-=- estimator methodology to the segmentation problem with Gaussian segments [4] [11, chapter 9] [5, 6]. They have derived MML formulas for stating the change-point locations to an optimal precision inde... |

170 |
Modeling by the shortest data description. Automatica
- Rissanen
- 1978
(Show Context)
Citation Context ... compared the MMLD approximation both with and without the FSMML Boundary Rule (MMLDF and MMLDc respectively) with SMML, FSMML, MML68 (as described in Section 3.3), Minimum Description Length (MDL78) =-=[16]-=- and the Minimum Expected KL Distance loss function (MEKLD) [10]. We ran 10 4 trials for each K = 2..15 where we sampled from the prior and then generated data. Each method was given the data and the ... |

109 | An experimental and theoretical comparison of model selection methods
- Kearns, Mansour, et al.
- 1995
(Show Context)
Citation Context ...tating the change-point locations to an optimal precision independently of the segment parameters. The same method has been used [7] for the problem of finding change-points in noisy binary sequences =-=[14] - w-=-here it compared favourably with Akaike’s Information Criterion (AIC), Schwarz’s Bayesian Information Criterion (BIC), an MDL-motivated metric of Kearns et al. [14] and a more correct version of M... |

104 | Minimum message length and Kolmogorov complexity,” The Computer Journal,vol.42,no.4
- Wallace, Dowe
- 1999
(Show Context)
Citation Context ... minimises the expected message length: I1 = − � r(x) (log q(m(x)) + log f(x|m(x))) (3) x∈X The estimator which minimises I1 is called the Strict Minimum Message Length (SMML) estimator [2, page=-= 242] [9, 10, 3].-=- The construction of I1 is NPhard for most distributions. The only distributions that it has reportedly been constructed for are the binomial and trinomial (trinomial using a heuristic) [9] and N(µ, ... |

53 |
On Grouping for Maximum Homogeneity
- Fisher
- 1958
(Show Context)
Citation Context ...mise I1a by searching for the optimal partition of the parameterspace and the ˆ θ for each segment of the partition. Since I1a consists of a sum over independent partitions, we can use W. D. Fisher�=-=��s [13]-=- polynomial time dynamic programming algorithm 1 . We therefore seek the partition of changepoints and the estimates which minimise I1a. We allow the partition to contain models from different subspac... |

14 |
1998]: ‘Point Estimation using the Kullback-Leibler Loss Function and
- Dowe, Baxter, et al.
(Show Context)
Citation Context ... minimises the expected message length: I1 = − � r(x) (log q(m(x)) + log f(x|m(x))) (3) x∈X The estimator which minimises I1 is called the Strict Minimum Message Length (SMML) estimator [2, page=-= 242] [9, 10, 3].-=- The construction of I1 is NPhard for most distributions. The only distributions that it has reportedly been constructed for are the binomial and trinomial (trinomial using a heuristic) [9] and N(µ, ... |

12 | Minimum message length segmentation
- Oliver, Baxter, et al.
- 1998
(Show Context)
Citation Context ...� � 1 11-1 φs3.3 MML68 Change-Point Approximation Oliver, Baxter and co-workers have applied the MML68 [1] estimator methodology to the segmentation problem with Gaussian segments [4] [11, chapte=-=r 9] [5, 6]-=-. They have derived MML formulas for stating the change-point locations to an optimal precision independently of the segment parameters. The same method has been used [7] for the problem of finding ch... |

9 | The Kindest Cut: Minimum Message Length Segmentation
- Baxter, Oliver
- 1996
(Show Context)
Citation Context ...5682 I1b = 6.916063 � � 1 11-1 φs3.3 MML68 Change-Point Approximation Oliver, Baxter and co-workers have applied the MML68 [1] estimator methodology to the segmentation problem with Gaussian segm=-=ents [4]-=- [11, chapter 9] [5, 6]. They have derived MML formulas for stating the change-point locations to an optimal precision independently of the segment parameters. The same method has been used [7] for th... |

9 |
Finding cutpoints in noisy binary sequences - a revised empirical evaluation
- Viswanathan, Wallace, et al.
- 1999
(Show Context)
Citation Context ...gments [4] [11, chapter 9] [5, 6]. They have derived MML formulas for stating the change-point locations to an optimal precision independently of the segment parameters. The same method has been used =-=[7] for-=- the problem of finding change-points in noisy binary sequences [14] - where it compared favourably with Akaike’s Information Criterion (AIC), Schwarz’s Bayesian Information Criterion (BIC), an MD... |

7 | Minimum message length grouping of ordered data
- Fitzgibbon, Allison, et al.
- 2000
(Show Context)
Citation Context ...ous work on coding change-point parameters in the MML framework has resulted in analytical approximations which treat the change-point as a continuous parameter [4–7] or avoid stating them altogethe=-=r [8]-=-. These methods work well in practice. However, change-points are realized as discrete parameters since they partition a data sample, and in this paper we investigate new MML approximations which trea... |

6 |
Estimation and inference by compact encoding (with discussion
- Wallace, Freeman
- 1987
(Show Context)
Citation Context ...still give insight into the behaviour of the methods. We then describe two new approximations called MMLDc and MMLDF. These are practical methods that are motivated by SMML (in part), FSMML and MML87 =-=[2]-=-. In Section 5 we empirically compare these new approximations with SMML, FSMML and other existing methods. 2 Binomial Problem A Bernoulli trial is conducted with K independent coin tosses. The result... |

6 | Bayesian approaches to segmenting a simple time series
- Oliver, Forbes
- 1997
(Show Context)
Citation Context ...� � 1 11-1 φs3.3 MML68 Change-Point Approximation Oliver, Baxter and co-workers have applied the MML68 [1] estimator methodology to the segmentation problem with Gaussian segments [4] [11, chapte=-=r 9] [5, 6]-=-. They have derived MML formulas for stating the change-point locations to an optimal precision independently of the segment parameters. The same method has been used [7] for the problem of finding ch... |

3 |
Algorithmic and Combinatorial Problems in Strict Minimum Message Length Inference
- Farr, Wallace
- 1997
(Show Context)
Citation Context ... minimises the expected message length: I1 = − � r(x) (log q(m(x)) + log f(x|m(x))) (3) x∈X The estimator which minimises I1 is called the Strict Minimum Message Length (SMML) estimator [2, page=-= 242] [9, 10, 3].-=- The construction of I1 is NPhard for most distributions. The only distributions that it has reportedly been constructed for are the binomial and trinomial (trinomial using a heuristic) [9] and N(µ, ... |

3 | Minimum message length inductive inference - theory and applications - Baxter - 1996 |

3 |
Improved approximations in MML. Honours thesis
- Lam
- 2000
(Show Context)
Citation Context ...roximation to FSMML: MMLD Minimum Message Length approximation D (MMLD) can be thought of as a numerical approximation to FSMML. It was proposed by D. L. Dowe and has been investigated by his student =-=[15]-=-. MMLD is based on choosing a region R of the parameter space after observing some data. It was partly motivated by improving the Taylor expansion approximation of MML87 [2] while retaining invariance... |

2 |
PAKDD-98 Tutorial: Data Mining
- Wallace
- 1998
(Show Context)
Citation Context ..., point estimates start appearing to the left of the change-point parameter space. This asymmetry is explained by the choice of biases used. 3.2 Fairly Strict Minimum Message Length (FSMML) The FSMML =-=[12] estimator is an approximation-=- to SMML based on a partition of the parameter space. The FSMML expected message length is: I1a = − � q( ˆ θ) log q( ˆ θ) − � � ˆθ∈Θ ∗ ˆθ∈Θ ∗ where q( ˆ θ) is approximate... |