## The psychometric function: I. Fitting, sampling, and goodness of fit (2001)

### Cached

### Download Links

Citations: | 106 - 13 self |

### BibTeX

@MISC{Wichmann01thepsychometric,

author = {Felix A. Wichmann and N. Jeremy Hill},

title = {The psychometric function: I. Fitting, sampling, and goodness of fit},

year = {2001}

}

### Years of Citing Articles

### OpenURL

### Abstract

The psychometric function relates an observer’s performance to an independent variable, usually some physical quantity of a stimulus in a psychophysical task. This paper, together with its companion paper (Wichmann & Hill, 2001), describes an integrated approach to (1) fitting psychometric functions, (2) assessing the goodness of fit, and (3) providing confidence intervals for the function’s parameters and other estimates derived from them, for the purposes of hypothesis testing. The present paper deals with the first two topics, describing a constrained maximum-likelihood method of parameter estimation and developing several goodness-of-fit tests. Using Monte Carlo simulations, we deal with two specific difficulties that arise when fitting functions to psychophysical data. First, we note that human observers are prone to stimulus-independent errors (or lapses). We show that failure to account for this can lead to serious biases in estimates of the psychometric function’s parameters and illustrate how the problem may be overcome. Second, we note that psychophysical data sets are usually rather small by the standards required by most of the commonly applied statistical tests. We demonstrate the potential errors of applying traditional c 2 methods to psychophysical data and advocate use of Monte Carlo resampling techniques that do not rely on asymptotic theory. We have made available the software to implement our methods. The performance of an observer on a psychophysical

### Citations

3487 | An Introduction to the Bootstrap
- Efron, RJ
- 1993
(Show Context)
Citation Context ...ulations addressing the issue raised in the respective introduction. NOTATION We adhere mainly to the typographic conventions frequently encountered in statistical texts (Collett, 1991; Dobson, 1990; =-=Efron & Tibshirani, 1993-=-). Variables are denoted by uppercase italic letters, and observed values are denoted by the corresponding lowercase letters—for example, y is a realization of the random variable Y. Greek letters are... |

1968 |
Generalized Linear Models
- McCullagh, Nelder
- 1989
(Show Context)
Citation Context ...tion at a given level of performance serves as a measure of the change in performance with changing stimulus intensity. Statistical estimation of parameters is routine in data modeling (Dobson, 1990; =-=McCullagh & Nelder, 1989-=-): In the context of fitting psychometric functions, probit analysis (Finney, 1952 , 1971) and a maximum-likelihood search method described by Watson (1979) are most commonly employed. Recently, Treut... |

1546 | Bayesian Data Analysis - Gelman, Carlin, et al. - 1997 |

1129 | Bootstrap methods: Another look at the jackknife - Efron - 1979 |

835 | The Jackknife, the Bootstrap and Other Resampling Plans - Efron - 1982 |

811 |
Applied Regression Analysis
- Draper, Smith
- 1981
(Show Context)
Citation Context ... datapoints and the corresponding model prediction—is frequently suggested as being one of the most effective ways of identifying an incorrect model in linear and nonlinear regression (Collett, 1991; =-=Draper & Smith, 1981-=-). Given that deviance is the appropriate summary statistic, it is sensible to base one’s further analyses on the deviance residuals, d. Each deviance residual d i is defined as the square root of the... |

314 | Numerical Recipes in C: The Art of Scientific Computing, 2nd ed - Press, Teukolsky, et al. - 1992 |

237 | A statistical distribution function of wide applicability - Weibull - 1951 |

236 | A leisurely look at the bootstrap, the jackknife, and cross-validation - Efron, Gong - 1983 |

152 |
Modelling binary data
- Collett
- 1991
(Show Context)
Citation Context ...sue, and second, a set of simulations addressing the issue raised in the respective introduction. NOTATION We adhere mainly to the typographic conventions frequently encountered in statistical texts (=-=Collett, 1991-=-; Dobson, 1990; Efron & Tibshirani, 1993). Variables are denoted by uppercase italic letters, and observed values are denoted by the corresponding lowercase letters—for example, y is a realization of ... |

144 |
An introduction to generalized linear models
- Dobson
- 1990
(Show Context)
Citation Context ..., a set of simulations addressing the issue raised in the respective introduction. NOTATION We adhere mainly to the typographic conventions frequently encountered in statistical texts (Collett, 1991; =-=Dobson, 1990-=-; Efron & Tibshirani, 1993). Variables are denoted by uppercase italic letters, and observed values are denoted by the corresponding lowercase letters—for example, y is a realization of the random var... |

117 |
Statistical significance testing and cumulative knowledge in psychology
- Schmidt
- 1996
(Show Context)
Citation Context ... instead of the true D*. The first two measures, P F and P M , are primarily useful for individual data sets. The latter two measures, DP RMS and DP max , provide useful information in meta-analyses (=-=Schmidt, 1996-=-), where models are assessed across several data sets. (In such analyses, we are interested in CPE errors even if the deviance value of one particular data set is not close to the tails of D: A system... |

86 | Probability summation over time - Watson - 1979 |

85 |
Introduction to mathematical statistics
- Hoel
- 1954
(Show Context)
Citation Context ...oximation to the c 2 will be reasonably good once the K individual binomial contributions to Pearson X 2 are well approximated by a normal—that is, as long as both n i p i � 5 and n i (1 � p i ) � 5 (=-=Hoel, 1984-=-). Even for only moderately high p values like .9, this already requires n i values of 50 or more, and p � .98 requires an n i of 250. No such simple criterion exists for deviance, however. The approx... |

81 | A vector-magnitude model of contrast detection - Quick - 1974 |

74 | Probit analysis, 3rd ed - Finney - 1971 |

69 |
The psychometric function
- Wichmann, Hill
- 2001
(Show Context)
Citation Context ...ychometric function relates an observer’s performance to an independent variable, usually some physical quantity of a stimulus in a psychophysical task. This paper, together with its companion paper (=-=Wichmann & Hill, 2001-=-), describes an integrated approach to (1) fitting psychometric functions, (2) assessing the goodness of fit, and (3) providing confidence intervals for the function’s parameters and other estimates d... |

34 | On the psychometric function for contrast detection - Nachmias - 1981 |

33 | Adaptive psychophysical procedures
- Treutwein
- 1995
(Show Context)
Citation Context ...mates. The adverse effect of nonstationary observer behavior (of which lapses are an example) on maximum-likelihood parameter estimates has been noted previously (Harvey, 1986; Swanson & Birch, 1992; =-=Treutwein, 1995-=-; cf. Treutwein & Strasburger, 1999). We show that the biases depend heavily on the sampling scheme chosen (by which we mean the pattern of stimulus values at which samples are taken) but that it can ... |

25 |
Bootstrap methods
- Hinkley
- 1988
(Show Context)
Citation Context ...op computing speeds. It is potentially well suited to the analysis of psychophysical data, because its accuracy does not rely on large numbers of trials, as do methods derived from asymptotic theory (=-=Hinkley, 1988-=-). We show that for the typically small K and N used in psychophysical experiments, assessing goodness of fit by comparing an empirically obtained statistic against its asymptotic distribution is not ... |

25 | Generalized linear models - Cullagh, Nelder, et al. - 1989 |

20 |
Efficient estimation of sensory thresholds
- Harvey
- 1986
(Show Context)
Citation Context ...ficant biases into the parameter estimates. The adverse effect of nonstationary observer behavior (of which lapses are an example) on maximum-likelihood parameter estimates has been noted previously (=-=Harvey, 1986-=-; Swanson & Birch, 1992; Treutwein, 1995; cf. Treutwein & Strasburger, 1999). We show that the biases depend heavily on the sampling scheme chosen (by which we mean the pattern of stimulus values at w... |

19 |
Statistical properties of forced-choice psychometric functions: Implications of probit analysis
- McKee, Klein, et al.
- 1985
(Show Context)
Citation Context ...es for some sampling schemes, even if the variable-l fitting regime is employed. Similar conclusions were reached by O’Regan and Humbert (1989) for N � 100 (K � 10; cf. Leek, Hanna, & Marshall, 1992; =-=McKee, Klein, & Teller, 1985-=-). This is further supported by the analysis of bootstrap sensitivity in our companion paper (Wichmann & Hill, 2001). GOODNESS OF FIT Background Assessing goodness of fit is a necessary component of a... |

18 | An Introduction to the Bootstrap - B, Tibshirani - 1993 |

13 |
Numerical Mathematics
- Hämmerlin, Hoffmann
- 1991
(Show Context)
Citation Context ...n as the true or reference distribution—otherwise, we would simply substitute errors arising from the inappropriate use of an asymptotic distribution for numerical errors incurred by our simulations (=-=Hämmerlin & Hoffmann, 1991-=-). One way to see whether D* has indeed approached D is to look at the convergence of several of the quantiles of D* with increasing B. For a large range of different values of N, K, and n i , we foun... |

13 | The psychometric function - A, Hill - 2001 |

10 |
Extracting thresholds from noisy psychophysical data
- Swanson, Birch
- 1992
(Show Context)
Citation Context ...into the parameter estimates. The adverse effect of nonstationary observer behavior (of which lapses are an example) on maximum-likelihood parameter estimates has been noted previously (Harvey, 1986; =-=Swanson & Birch, 1992-=-; Treutwein, 1995; cf. Treutwein & Strasburger, 1999). We show that the biases depend heavily on the sampling scheme chosen (by which we mean the pattern of stimulus values at which samples are taken)... |

10 | The jackknife. the bootstrap. and other resampling plans - unknown authors - 1982 |

9 |
Some aspects of modelling human spatial vision: Contrast discrimination. Unpublished doctoral dissertation
- Wichmann
- 1999
(Show Context)
Citation Context ...en residuals and model predictions, it is not a linear one. Figure 9A shows data from a visual masking experiment with K � 10 and ni � 50, together with the bestfitting Weibull psychometric function (=-=Wichmann, 1999-=-). Figure 9B shows a histogram of D* for B � 10,000 with the scaled c2 10-PDF. The two arrows below the deviance axis mark the two-sided 95% confidence interval [D*(.025), D* (.975)]. The deviance of ... |

8 |
Probit analysis, 2nd ed
- Finney
- 1964
(Show Context)
Citation Context ...nging stimulus intensity. Statistical estimation of parameters is routine in data modeling (Dobson, 1990; McCullagh & Nelder, 1989): In the context of fitting psychometric functions, probit analysis (=-=Finney, 1952-=- , 1971) and a maximum-likelihood search method described by Watson (1979) are most commonly employed. Recently, Treutwein and Strasburger (1999) have described a constrained generalized maximum-likel... |

8 | Model selection in science: The problem of language variance - Forster - 1999 |

8 | Bootstrap methods: another look at the jackknife - unknown authors - 1979 |

6 | Estimating psychometric functions in forced-choice situations: Significant biases found in threshold and slope estimations when small samples are used - O'Regan, Humbert - 1989 |

6 | Applied Regression Analysis - R, Smith - 1966 |

6 | Bootstrap methods - V - 1988 |

5 |
Estimation of psychometric functions from adaptive tracking procedures
- Leek, Hanna, et al.
- 1992
(Show Context)
Citation Context ...stimates of thresholds and slopes for some sampling schemes, even if the variable-l fitting regime is employed. Similar conclusions were reached by O’Regan and Humbert (1989) for N � 100 (K � 10; cf. =-=Leek, Hanna, & Marshall, 1992-=-; McKee, Klein, & Teller, 1985). This is further supported by the analysis of bootstrap sensitivity in our companion paper (Wichmann & Hill, 2001). GOODNESS OF FIT Background Assessing goodness of fit... |

5 |
Fitting the psychometric function
- Terutwein, Strasburger
- 1999
(Show Context)
Citation Context ...fect of nonstationary observer behavior (of which lapses are an example) on maximum-likelihood parameter estimates has been noted previously (Harvey, 1986; Swanson & Birch, 1992; Treutwein, 1995; cf. =-=Treutwein & Strasburger, 1999-=-). We show that the biases depend heavily on the sampling scheme chosen (by which we mean the pattern of stimulus values at which samples are taken) but that it can be corrected, at minimal cost in te... |

5 | Probit analysis. 3rd ed - J - 1971 |

5 | Tel l er - McKee, ein, et al. - 1985 |

5 | Adaptive psychophysical procedures - wein, B - 1995 |

5 | Probability summation over time - SON - 1979 |

4 | Efficient estimation of sensory thresholds - vey, O - 1986 |

4 | St r asbur ger , H - wein, B - 1999 |

3 | A leisurely look at the bootstrap, the jackknife, and cross-validation - on, B, et al. - 1983 |

3 | Probit analysis (2nd ed - ey, J - 1952 |

3 | Placement of observations for the efficient estimation of a psychometric function - La, F, et al. - 1996 |

3 | On the psychometric function for contrast detection - hmias, J - 1981 |

3 | Vet t er l - ess, H, et al. - 1992 |

3 | A Statistical Distribution Function of Wide Applicability - Weibul - 1951 |

2 |
Placement of observations for the efficient estimation of a psychometric function
- Lam, Mills, et al.
- 1996
(Show Context)
Citation Context ...they carry more information about the underlying function and thus allow more efficient estimation. Accordingly, Figure 4 shows that the precision of slope estimates is better for s7 than for s1 (cf. =-=Lam, Mills, & Dubno, 1996-=-). This issue is explored more fully in our companion paper (Wichmann & Hill, 2001). Finally, even for those sampling schemes that contain no sample points at performance levels above 80%, bias in thr... |