#### DMCA

## What Size Test Set Gives Good Error Rate Estimates? (1996)

Venue: | IEEE Trans PAMI |

Citations: | 38 - 5 self |

### Citations

2213 | Probability inequalities for sums of bounded random variables
- Hoeffding
- 1963
(Show Context)
Citation Context ...) where oe 2 is the variance of X, estimated, for instance, as 1 :soe 2 = 1 n \Gamma 1 n X i=1 (p \Gamma x i ) 2 : (8) Other tighter bounds have been proposed more recently by Chernoff [3], Hoeffding =-=[4]-=- and others for the Binomial distribution (see Appendix A). Those bounds are tighter that Chebychev 's inequality, but Chebychev's inequality is distribution independent. 1 The denominator (n \Gamma 1... |

471 |
Intoduction to the theory of statistics”,
- Mood, Graybill, et al.
- 1974
(Show Context)
Citation Context ... rates of particular recognizers on similar tasks are known, it is possible to estimate what reasonable size a test set should have. In this paper, we use fairly straightforward statistical arguments =-=[1]-=- to address that problem. The method has been designed to help in preparing the data for the first UNIPEN benchmark [2], but the results are fairly general and a broader applicability is expected. We ... |

252 | Some statistical issues in the comparison of speech recognition algorithms,” in
- Gillick, Cox
- 1989
(Show Context)
Citation Context ...ror rates of two recognizers is statistically significant. We first revert to the assumption that errors are i.i.d. The method used is very simple. We do not need the sophistication of McNemar's test =-=[9]-=- nor that of the method proposed in [10]. These two methods require counting the number of common errors of the two recognizers which are not known prior to testing. We will, however, introduce these ... |

52 |
The first census optical character recognition systems conference,
- Wilkinson, Geist, et al.
- 1992
(Show Context)
Citation Context ...r of test samples needed) is determined by the smallest error rate which is provided by the best recognizer. A survey of the handwriting recognition literature and of the results of recent benchmarks =-=[5, 6, 7]-=- indicates that the best recognizers of isolated handwritten characters will probably not have a character error rate lower than 1% (p = 0:01). For p = 0:01, we obtain: n ' 10; 000 characters (28) Per... |

20 |
UNIPEN project of on-line data exchange and benchmarks
- Guyon, Schomaker, et al.
- 1994
(Show Context)
Citation Context ... should have. In this paper, we use fairly straightforward statistical arguments [1] to address that problem. The method has been designed to help in preparing the data for the first UNIPEN benchmark =-=[2], but-=- the results are fairly general and a broader applicability is expected. We tackle the problem from the point of view of the benchmark organizer. Thus, our approach differs from the classical "hy... |

15 |
Writer independent and writer adaptive neural network for on-line character recognition
- Guyon, Henderson, et al.
- 1992
(Show Context)
Citation Context ...rsussp for data obtained from the NIST benchmark of OCR for isolated handwritten characters [5]. The "between-writer" standard deviation of those datasoe lies roughly between 0:5p andsp. In =-=reference [8] the autho-=-rs also report a "between-writer" standard deviation which is of the order of the mean. Therefore, we adopted the value: oe ' p (37) in our calculations. We know that q p=nw is a lower bound... |

13 |
Overview and synthesis of online cursive handwriting recognition techniques
- Guyon, Schenkel, et al.
- 1997
(Show Context)
Citation Context ...r of test samples needed) is determined by the smallest error rate which is provided by the best recognizer. A survey of the handwriting recognition literature and of the results of recent benchmarks =-=[5, 6, 7]-=- indicates that the best recognizers of isolated handwritten characters will probably not have a character error rate lower than 1% (p = 0:01). For p = 0:01, we obtain: n ' 10; 000 characters (28) Per... |

2 |
A measure of assymptotic efficientcy for tests of a hypothesis based on the sums of observations
- Chernoff
- 1952
(Show Context)
Citation Context ... 2ffn !sff ; (7) where oe 2 is the variance of X, estimated, for instance, as 1 :soe 2 = 1 n \Gamma 1 n X i=1 (p \Gamma x i ) 2 : (8) Other tighter bounds have been proposed more recently by Chernoff =-=[3]-=-, Hoeffding [4] and others for the Binomial distribution (see Appendix A). Those bounds are tighter that Chebychev 's inequality, but Chebychev's inequality is distribution independent. 1 The denomina... |

1 |
Local leaning algorithms
- Bottou, Vapnik
- 1992
(Show Context)
Citation Context ...ically significant. We first revert to the assumption that errors are i.i.d. The method used is very simple. We do not need the sophistication of McNemar's test [9] nor that of the method proposed in =-=[10]-=-. These two methods require counting the number of common errors of the two recognizers which are not known prior to testing. We will, however, introduce these methods in section 6 to do a posteriori ... |