Citation Context

... we also introduce procedures for performing inference on the estimated summary statistics. In our simulations we find that “nonparametric” bootstrap and sign permutation tests perform adequately with each model, and are roughly comparable in sensitivity to the standard parametric model when model assumptions hold. Use of these models may be advantageous when testing effects that do not have clear parametric p-values, such as the distribution of maxima used in multiple comparisons correction (Nichols and Holmes, 2002), or for which parametric p-values are insensitive (such as mediation tests; Shrout and Bolger, 2002). A key point of this paper is that model misspecification can result in bias in addition to loss in power. This bias may inflate the Type I error rate beyond the nominal α level, so that p-values for the test are inaccurate. For example, a statistical parametric map thresholded at pb0.001 may actually only control the false positive rate at, for example, pb0.004. We find that even relatively minor model misspecification can result in substantial power loss. In light of our results, it seems important for studies that use a single canonical HRF or a highly constrained basis set to construct ma...

Citation Context

...odels in terms of power, bias and parameter confusability. Because virtually all fMRI studies in cognitive and affective neuroscience employ these models, the results bear on the interpretation of hemodynamic response estimates across a wide variety of psychological and neuroscientific studies. © 2008 Elsevier Inc. All rights reserved.IntroductionFunctional magnetic resonance imaging (fMRI) is based on studying the vascular response in the brain to neuronal activity and can be used to study mental activity. It is most commonly performed using blood oxygenation level-dependent (BOLD) contrast (Ogawa et al., 1992) to study local changes in deoxyhemoglobin concentration in the brain. The primary goal of fMRI research is to use information provided by the BOLD signal to make conclusions about the underlying unobserved neuronal activation. Therefore, the ability to accurately model the evoked hemodynamic response to a neural event plays an important role in the analysis of fMRI data. When analyzing the shape of the estimated hemodynamic response function (HRF), summary measures of psychological interest (e.g., amplitude, delay, and duration) can be extracted and used to infer information regarding the int...

Citation Context

... second derivative term.Inference We also seek to compare several techniques for performing population inference on the estimated amplitude. Let Hi be the estimated amplitude for subject i, i=1,….M, defined for hypothesis testing purposes to be the global extreme point for the HRF, i.e. either a minimum or a maximum. The goal is to test whether H significantly differs from 0 in the population. In this work we compare three statistical techniques: the standard summary statistics approach (Holmes and Friston, 1998), a bootstrap procedure (Efron and Tibshirani, 1993) and a sign permutation test (Nichols and Holmes, 2002). Each of these methods has received extensive attention in the S190 M.A. Lindquist et al. / NeuroImage 45 (2009) S187–S198neuroimaging literature, and is described in detail in Section C of the Appendix. Detecting model misspecification Each of the models presented in this paper differ in their ability to handle unexpected HRF shapes. Using an ill-fitting model will violate the assumptions (e.g., mean 0 noise) required for valid inference and even a small amount of mis-modeling can result in severe power loss and inflate the false positive rate beyond the nominal value. Due to the massive amo...

Citation Context

...signal of interest, without attempting to make a direct quantitative link to neuronal activity. Early studies presented events with large separation in time (e.g., visual stimuli every 20–30 s), so that task-evoked average BOLD responses could be recovered, and H, T, and W estimated directly. However, this design is highly inefficient, as very few stimuli can be presented in a session, and it has become common practice to present events rapidly enough so that the BOLD responses to different events overlap. The dominant analysis strategy is to assume that BOLD responses to events add linearly (Boynton et al., 1996) and use a set of smooth functions to model the underlying HRF. Choices of HRF models range from the use of a single canonical HRF, the use of a basis set of smooth functions (Friston et al., 1998a), the use of flexible basis sets such as finite impulse response models (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001), and nonlinear estimation of smooth reference functions with multiple parameters (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). These models all involve a simplified estimation of the BOLD HRF, which gives rise to th...

Citation Context

... larger in mis-modeled segments of the time series. Suppose r(i), i =1,…T are the whitened residuals and K(t) a kernel function. Let, Zw tð Þ = ∑ t + w−1 i=t r ið ÞK t−ið Þ ð6Þ be the moving average of w consecutive observations, starting at time t. Under the null hypothesis that the model is correct, Zw is mean 0 for all values of t. Thus a large value of Zw indicates that model mis-fit might be present and the statistic S=max Zw (t) measures the strongest evidence against the null hypothesis. Choosing a Gaussian kernel allows Gaussian random field theory to be used to determine the p-value (Worsley et al., 1992, 1996). The results can be used to detect population wide mis-modeling in a voxel, by calculating the test statistic Q = −2∑ i=1 M log pið Þ, where pi is the p-value for subject i. Under the null hypothesis of no effect, Q follows a chi-square distribution with 2M degrees of freedom. As a follow-up we have proposed techniques for determining whether there is task-related signal remaining in the residuals and for quantifying the amount of power-loss and bias directly attributable toFig. 2. Illustration of “ground truth” data used in the simulations. (A) A set of 25 squares we created based on ...

Citation Context

... that task-evoked average BOLD responses could be recovered, and H, T, and W estimated directly. However, this design is highly inefficient, as very few stimuli can be presented in a session, and it has become common practice to present events rapidly enough so that the BOLD responses to different events overlap. The dominant analysis strategy is to assume that BOLD responses to events add linearly (Boynton et al., 1996) and use a set of smooth functions to model the underlying HRF. Choices of HRF models range from the use of a single canonical HRF, the use of a basis set of smooth functions (Friston et al., 1998a), the use of flexible basis sets such as finite impulse response models (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001), and nonlinear estimation of smooth reference functions with multiple parameters (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). These models all involve a simplified estimation of the BOLD HRF, which gives rise to the second problem identified at the right side of Fig. 1. Not all models are equally good at capturing evoked changes in the true H, T, and W of the BOLDresponse. Evaluating the performance of thes...

Citation Context

....g. squares in the lower right hand corner). In contrast, the sFIR and IL models provide uniform control of the true positive rate (TPR) across each of the 25 squares. While, the TPR is slightly lower than the gamma based models in squares with minor model misspecification, both of these methods provide a clear improvement with increasing model misspecification. Experiment The results of the pain experiment are summarized in Figs. 6–7. The location of the slice used and an illustration of areas of interest(rdACC and S2, two brain regions known to process pain intensity (Ferretti et al., 2003; Peyron et al., 2000) are shown in Fig. 6. In Fig. 7A, we show results obtained after estimating the height parameter on the 12 high-pain trials for each participant using the TD model, and testing for a population effect using the summary statistics approach (pb0.01). In addition, we show a map of model misspecification in Fig. 7B. Here red corresponds to values with increased mis-modeling. In particular note the relatively large amount of mis-modeling present in the regions corresponding to S2. Figs. 7C, D shows results obtained after fitting the height parameter using the sFIR and IL models, and testing for a p...

Citation Context

...every time-point within a given window of time following stimulation in every cognitive event type modeled (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001). Thus, the model is able to estimate an HRF of arbitrary shape for each event type in every voxel of the brain. There are a number of models that fall somewhere between these two extremes. A popular choice is to use a combination of the canonical HRF and its derivatives with respect to time and dispersion (Friston et al., 1998b; Henson et al., 2002). Other approaches include the use of basis sets composed of principal components (Aguirre et al., 1998; Woolrich et al., 2004), cosine functions (Zarahn, 2002), radial basis functions (Riera et al., 2004), spline basis sets, a Gaussian model (Rajapakse et al., 1998) and spectral basis functions (Liao et al., 2002). Also, a number of researchers have used nonlinear fitting of a canonical function with free parameters for magnitude and onset/ peak delay (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). In general, the more basis functions used in a linear model or the more free parameters in a nonlinear one, the more flexible the model is in me...

Citation Context

...used in neuroimaging. Herewe assume thatH is the samplemean and sH the sample standard deviation of the M amplitude estimates (H1, H2, …HM). Using these values we calculate the test statistic: t = H sH= ffiffiffiffiffi M p ðC1Þ p-values are obtained by comparing the results with a t-distribution with M−1 degrees of freedom. There are a few important issues to keep in mind when applying the summary statistic approach. First, the method assumes constant within-subject variation of the height estimates across subjects, though this assumption can be relaxed in a manner similar to that outlined by Beckmann et al. (2003). However, in this work we simply discuss the classic summary statistics approach with the caveat that the conclusions are only valid if the within-subject variance is homogenous across subjects. A second issue is that p-values are only valid when the Hi are normally distributed. This will typically be true when performing inference directly on the β values obtained from least-squares (canonical HRF term only or FIR basis set). However, when using the derivative boost, sFIR, NL or IL approach assuming that the amplitudes follow a normal distribution may not necessarily be reasonable. (ii) Boot...

Citation Context

.... However, this design is highly inefficient, as very few stimuli can be presented in a session, and it has become common practice to present events rapidly enough so that the BOLD responses to different events overlap. The dominant analysis strategy is to assume that BOLD responses to events add linearly (Boynton et al., 1996) and use a set of smooth functions to model the underlying HRF. Choices of HRF models range from the use of a single canonical HRF, the use of a basis set of smooth functions (Friston et al., 1998a), the use of flexible basis sets such as finite impulse response models (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001), and nonlinear estimation of smooth reference functions with multiple parameters (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). These models all involve a simplified estimation of the BOLD HRF, which gives rise to the second problem identified at the right side of Fig. 1. Not all models are equally good at capturing evoked changes in the true H, T, and W of the BOLDresponse. Evaluating the performance of these models is the focus of the current paper. Thus, in sum, the nature of the underlying ...

Citation Context

...he observed data, and ε is a vector of unexplained error values. For most statistical analysis the use of a LTI system is considered a reasonable assumption that provides for valid statistical inference. Therefore, in this work we assume an LTI system, and our main focus will be finding flexible models for the impulse function in the LTI system, i.e. the HRF. A number of models, varying greatly in their flexibility, have been suggested in the literature. In the most rigid model, the shape of the HRF is completely fixed and the height (i.e., amplitude) of the response alone is allowed to vary (Worsley and Friston, 1995). By contrast, one of the most flexible models, a finite impulse response (FIR) basis set, contains one free parameter for every time-point within a given window of time following stimulation in every cognitive event type modeled (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001). Thus, the model is able to estimate an HRF of arbitrary shape for each event type in every voxel of the brain. There are a number of models that fall somewhere between these two extremes. A popular choice is to use a combination of the canonical HRF and its derivatives with respect to time and dispersion (Fri...

Citation Context

...straints on the interpretability of parameter estimates. S188 M.A. Lindquist et al. / NeuroImage 45 (2009) S187–S198dependent, and is not constant over time (Logothetis, 2003). Second, the hemodynamic response is sluggish (i.e., there is hysteresis) and, when it does reflect neuronal/glial activity, it integrates this activity across time. Thus, an increase in the duration of neuronal activity could result in increases in both the amplitude (H) and duration (W) of the evoked BOLD response. Third, the BOLD response is itself a nonlinear integrator, as the vascular response saturates over time (Friston et al., 2000; Vazquez et al., 2006; Wager et al., 2005), further complicating matters. In sum, there is not always a clear relationship between neuronal/glial activity changes and parameters of the evoked BOLD response. The second part of the problem depicted in Fig. 1 is whether the statistical model of the HRF recovers the truemagnitude, time to peak, and width of the response. That is, do changes in the estimate of the height correspond to equivalent changes in the true magnitude of the BOLD response? While the second issue may seem easy to resolve, as we show here, both the use of multiple regression ...

Citation Context

... The dominant analysis strategy is to assume that BOLD responses to events add linearly (Boynton et al., 1996) and use a set of smooth functions to model the underlying HRF. Choices of HRF models range from the use of a single canonical HRF, the use of a basis set of smooth functions (Friston et al., 1998a), the use of flexible basis sets such as finite impulse response models (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001), and nonlinear estimation of smooth reference functions with multiple parameters (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). These models all involve a simplified estimation of the BOLD HRF, which gives rise to the second problem identified at the right side of Fig. 1. Not all models are equally good at capturing evoked changes in the true H, T, and W of the BOLDresponse. Evaluating the performance of these models is the focus of the current paper. Thus, in sum, the nature of the underlying BOLD physiology limits the ultimate interpretability of the parameter estimates in terms of neuronal and metabolic function, but modeling task-evoked BOLD responses is useful, and is in fact critical for inference in virtually ...

Citation Context

...fiffiffiffiffiffi β 2 1 + β 2 2 + β 2 3 q ð5Þ where β3 is the regression parameter corresponding to the second derivative term.Inference We also seek to compare several techniques for performing population inference on the estimated amplitude. Let Hi be the estimated amplitude for subject i, i=1,….M, defined for hypothesis testing purposes to be the global extreme point for the HRF, i.e. either a minimum or a maximum. The goal is to test whether H significantly differs from 0 in the population. In this work we compare three statistical techniques: the standard summary statistics approach (Holmes and Friston, 1998), a bootstrap procedure (Efron and Tibshirani, 1993) and a sign permutation test (Nichols and Holmes, 2002). Each of these methods has received extensive attention in the S190 M.A. Lindquist et al. / NeuroImage 45 (2009) S187–S198neuroimaging literature, and is described in detail in Section C of the Appendix. Detecting model misspecification Each of the models presented in this paper differ in their ability to handle unexpected HRF shapes. Using an ill-fitting model will violate the assumptions (e.g., mean 0 noise) required for valid inference and even a small amount of mis-modeling can resul...

Citation Context

...ower and validity (Lindquist and Wager, 2007; Loh et al., 2008). The issue of interpretability is complex, and the problem can be divided into two parts, shown in Fig. 1. The first relates to whether changes in physiological, metabolic-level parameters (e.g. magnitude, delay, and duration of evoked changes in neuronal/glial activity) directly translate into changes in corresponding parameters of the HRF, such as H, T, and W. These physiological parameters are often assumed to be neural in origin as they have been shown to correlate highly with measures of extracellular post-synaptic activity (Logothetis, 2003), but they also have glial components (Schummers et al., 2008). However, this part of the problem is complicated for several reasons. First, the neural response to a given stimulus is complex, taskFig. 1. The relationship between neural activity, evoked changes in the BOLD response, and estimated parameters. Solid lines indicate expected relationships, and dashed lines indicate relationships that complicate interpretation of the estimated parameters. As an example, for task-induced changes in estimated time-to-peak to be interpretable in terms of the latency of neural firing, the estimated tim...

Citation Context

...efficient, as very few stimuli can be presented in a session, and it has become common practice to present events rapidly enough so that the BOLD responses to different events overlap. The dominant analysis strategy is to assume that BOLD responses to events add linearly (Boynton et al., 1996) and use a set of smooth functions to model the underlying HRF. Choices of HRF models range from the use of a single canonical HRF, the use of a basis set of smooth functions (Friston et al., 1998a), the use of flexible basis sets such as finite impulse response models (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001), and nonlinear estimation of smooth reference functions with multiple parameters (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). These models all involve a simplified estimation of the BOLD HRF, which gives rise to the second problem identified at the right side of Fig. 1. Not all models are equally good at capturing evoked changes in the true H, T, and W of the BOLDresponse. Evaluating the performance of these models is the focus of the current paper. Thus, in sum, the nature of the underlying BOLD physiology limits the ultimate interpret...

Citation Context

...re collinear. It is also easier and statistically more powerful to interpret differences between task conditions on a single parameter such as height than it is to test for differences in multiple parameters — conditional, of course, on the interpretability of those parameter estimates. Together these problems suggest using a single, canonical HRF and it does in fact offer optimal power if the shape is specified exactly correctly. However, the shape of the HRF varies as a function of both task and brain region, and any fixed model will be misspecified for much of the brain (Birn et al., 2001; Handwerker et al., 2004; Marrelec et al., 2003; Wager et al., 2005). If the model is incorrectly specified, statistical power will decrease, and the results may be invalid and biased. In addition, using a single canonical HRF does not provide a way to assess latency and duration—in fact, differences between conditions in response latency will be confusedfor differences in amplitude (Calhoun et al., 2004; Lindquist and Wager, 2007). HRF models In this work we study seven HRF models in a series of simulation studies and an application to real data. We briefly introduce each model below, but leave a more detailed descr...

Citation Context

...f the most flexible models, a finite impulse response (FIR) basis set, contains one free parameter for every time-point within a given window of time following stimulation in every cognitive event type modeled (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001). Thus, the model is able to estimate an HRF of arbitrary shape for each event type in every voxel of the brain. There are a number of models that fall somewhere between these two extremes. A popular choice is to use a combination of the canonical HRF and its derivatives with respect to time and dispersion (Friston et al., 1998b; Henson et al., 2002). Other approaches include the use of basis sets composed of principal components (Aguirre et al., 1998; Woolrich et al., 2004), cosine functions (Zarahn, 2002), radial basis functions (Riera et al., 2004), spline basis sets, a Gaussian model (Rajapakse et al., 1998) and spectral basis functions (Liao et al., 2002). Also, a number of researchers have used nonlinear fitting of a canonical function with free parameters for magnitude and onset/ peak delay (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). In general, the more basis functions used...

Citation Context

...led (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001). Thus, the model is able to estimate an HRF of arbitrary shape for each event type in every voxel of the brain. There are a number of models that fall somewhere between these two extremes. A popular choice is to use a combination of the canonical HRF and its derivatives with respect to time and dispersion (Friston et al., 1998b; Henson et al., 2002). Other approaches include the use of basis sets composed of principal components (Aguirre et al., 1998; Woolrich et al., 2004), cosine functions (Zarahn, 2002), radial basis functions (Riera et al., 2004), spline basis sets, a Gaussian model (Rajapakse et al., 1998) and spectral basis functions (Liao et al., 2002). Also, a number of researchers have used nonlinear fitting of a canonical function with free parameters for magnitude and onset/ peak delay (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). In general, the more basis functions used in a linear model or the more free parameters in a nonlinear one, the more flexible the model is in measuring the parameters of interest. However, including more parameters also means more potential for e...

Citation Context

...here β3 is the regression parameter corresponding to the second derivative term.Inference We also seek to compare several techniques for performing population inference on the estimated amplitude. Let Hi be the estimated amplitude for subject i, i=1,….M, defined for hypothesis testing purposes to be the global extreme point for the HRF, i.e. either a minimum or a maximum. The goal is to test whether H significantly differs from 0 in the population. In this work we compare three statistical techniques: the standard summary statistics approach (Holmes and Friston, 1998), a bootstrap procedure (Efron and Tibshirani, 1993) and a sign permutation test (Nichols and Holmes, 2002). Each of these methods has received extensive attention in the S190 M.A. Lindquist et al. / NeuroImage 45 (2009) S187–S198neuroimaging literature, and is described in detail in Section C of the Appendix. Detecting model misspecification Each of the models presented in this paper differ in their ability to handle unexpected HRF shapes. Using an ill-fitting model will violate the assumptions (e.g., mean 0 noise) required for valid inference and even a small amount of mis-modeling can result in severe power loss and inflate the false positiv...

Citation Context

...n a given window of time following stimulation in every cognitive event type modeled (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001). Thus, the model is able to estimate an HRF of arbitrary shape for each event type in every voxel of the brain. There are a number of models that fall somewhere between these two extremes. A popular choice is to use a combination of the canonical HRF and its derivatives with respect to time and dispersion (Friston et al., 1998b; Henson et al., 2002). Other approaches include the use of basis sets composed of principal components (Aguirre et al., 1998; Woolrich et al., 2004), cosine functions (Zarahn, 2002), radial basis functions (Riera et al., 2004), spline basis sets, a Gaussian model (Rajapakse et al., 1998) and spectral basis functions (Liao et al., 2002). Also, a number of researchers have used nonlinear fitting of a canonical function with free parameters for magnitude and onset/ peak delay (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). In general, the more basis functions used in a linear model or the more free parameters in a nonlinear one, the more flexible the model is in measuring the parameters o...

Citation Context

..., 2008). The issue of interpretability is complex, and the problem can be divided into two parts, shown in Fig. 1. The first relates to whether changes in physiological, metabolic-level parameters (e.g. magnitude, delay, and duration of evoked changes in neuronal/glial activity) directly translate into changes in corresponding parameters of the HRF, such as H, T, and W. These physiological parameters are often assumed to be neural in origin as they have been shown to correlate highly with measures of extracellular post-synaptic activity (Logothetis, 2003), but they also have glial components (Schummers et al., 2008). However, this part of the problem is complicated for several reasons. First, the neural response to a given stimulus is complex, taskFig. 1. The relationship between neural activity, evoked changes in the BOLD response, and estimated parameters. Solid lines indicate expected relationships, and dashed lines indicate relationships that complicate interpretation of the estimated parameters. As an example, for task-induced changes in estimated time-to-peak to be interpretable in terms of the latency of neural firing, the estimated time-to-peak must vary only as a function of changes in neural fi...

Citation Context

...ity if regressors are collinear. It is also easier and statistically more powerful to interpret differences between task conditions on a single parameter such as height than it is to test for differences in multiple parameters — conditional, of course, on the interpretability of those parameter estimates. Together these problems suggest using a single, canonical HRF and it does in fact offer optimal power if the shape is specified exactly correctly. However, the shape of the HRF varies as a function of both task and brain region, and any fixed model will be misspecified for much of the brain (Birn et al., 2001; Handwerker et al., 2004; Marrelec et al., 2003; Wager et al., 2005). If the model is incorrectly specified, statistical power will decrease, and the results may be invalid and biased. In addition, using a single canonical HRF does not provide a way to assess latency and duration—in fact, differences between conditions in response latency will be confusedfor differences in amplitude (Calhoun et al., 2004; Lindquist and Wager, 2007). HRF models In this work we study seven HRF models in a series of simulation studies and an application to real data. We briefly introduce each model below, but le...

Citation Context

...s design is highly inefficient, as very few stimuli can be presented in a session, and it has become common practice to present events rapidly enough so that the BOLD responses to different events overlap. The dominant analysis strategy is to assume that BOLD responses to events add linearly (Boynton et al., 1996) and use a set of smooth functions to model the underlying HRF. Choices of HRF models range from the use of a single canonical HRF, the use of a basis set of smooth functions (Friston et al., 1998a), the use of flexible basis sets such as finite impulse response models (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001), and nonlinear estimation of smooth reference functions with multiple parameters (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). These models all involve a simplified estimation of the BOLD HRF, which gives rise to the second problem identified at the right side of Fig. 1. Not all models are equally good at capturing evoked changes in the true H, T, and W of the BOLDresponse. Evaluating the performance of these models is the focus of the current paper. Thus, in sum, the nature of the underlying BOLD physiology limit...

Citation Context

...neuroimaging literature, and is described in detail in Section C of the Appendix. Detecting model misspecification Each of the models presented in this paper differ in their ability to handle unexpected HRF shapes. Using an ill-fitting model will violate the assumptions (e.g., mean 0 noise) required for valid inference and even a small amount of mis-modeling can result in severe power loss and inflate the false positive rate beyond the nominal value. Due to the massive amount of data, performing model diagnostics is challenging, and only limited attention has been given to this problem (e.g., Luo and Nichols, 2003). We have recently introduced a procedure (Loh et al., 2008) that uses model residuals to identify voxels where model misfit (e.g. mis-specification of onset, duration, or response shape) may be present. The key idea is that residuals will be systematically larger in mis-modeled segments of the time series. Suppose r(i), i =1,…T are the whitened residuals and K(t) a kernel function. Let, Zw tð Þ = ∑ t + w−1 i=t r ið ÞK t−ið Þ ð6Þ be the moving average of w consecutive observations, starting at time t. Under the null hypothesis that the model is correct, Zw is mean 0 for all values of t. Thus a...

Citation Context

...easier and statistically more powerful to interpret differences between task conditions on a single parameter such as height than it is to test for differences in multiple parameters — conditional, of course, on the interpretability of those parameter estimates. Together these problems suggest using a single, canonical HRF and it does in fact offer optimal power if the shape is specified exactly correctly. However, the shape of the HRF varies as a function of both task and brain region, and any fixed model will be misspecified for much of the brain (Birn et al., 2001; Handwerker et al., 2004; Marrelec et al., 2003; Wager et al., 2005). If the model is incorrectly specified, statistical power will decrease, and the results may be invalid and biased. In addition, using a single canonical HRF does not provide a way to assess latency and duration—in fact, differences between conditions in response latency will be confusedfor differences in amplitude (Calhoun et al., 2004; Lindquist and Wager, 2007). HRF models In this work we study seven HRF models in a series of simulation studies and an application to real data. We briefly introduce each model below, but leave a more detailed description for Section A of...

Citation Context

... that task-evoked average BOLD responses could be recovered, and H, T, and W estimated directly. However, this design is highly inefficient, as very few stimuli can be presented in a session, and it has become common practice to present events rapidly enough so that the BOLD responses to different events overlap. The dominant analysis strategy is to assume that BOLD responses to events add linearly (Boynton et al., 1996) and use a set of smooth functions to model the underlying HRF. Choices of HRF models range from the use of a single canonical HRF, the use of a basis set of smooth functions (Friston et al., 1998a), the use of flexible basis sets such as finite impulse response models (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001), and nonlinear estimation of smooth reference functions with multiple parameters (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). These models all involve a simplified estimation of the BOLD HRF, which gives rise to the second problem identified at the right side of Fig. 1. Not all models are equally good at capturing evoked changes in the true H, T, and W of the BOLDresponse. Evaluating the performance of thes...

Citation Context

...r estimates. S188 M.A. Lindquist et al. / NeuroImage 45 (2009) S187–S198dependent, and is not constant over time (Logothetis, 2003). Second, the hemodynamic response is sluggish (i.e., there is hysteresis) and, when it does reflect neuronal/glial activity, it integrates this activity across time. Thus, an increase in the duration of neuronal activity could result in increases in both the amplitude (H) and duration (W) of the evoked BOLD response. Third, the BOLD response is itself a nonlinear integrator, as the vascular response saturates over time (Friston et al., 2000; Vazquez et al., 2006; Wager et al., 2005), further complicating matters. In sum, there is not always a clear relationship between neuronal/glial activity changes and parameters of the evoked BOLD response. The second part of the problem depicted in Fig. 1 is whether the statistical model of the HRF recovers the truemagnitude, time to peak, and width of the response. That is, do changes in the estimate of the height correspond to equivalent changes in the true magnitude of the BOLD response? While the second issue may seem easy to resolve, as we show here, both the use of multiple regression models and presentation of stimuli rapidly ...

Citation Context

...wer if the shape is specified exactly correctly. However, the shape of the HRF varies as a function of both task and brain region, and any fixed model will be misspecified for much of the brain (Birn et al., 2001; Handwerker et al., 2004; Marrelec et al., 2003; Wager et al., 2005). If the model is incorrectly specified, statistical power will decrease, and the results may be invalid and biased. In addition, using a single canonical HRF does not provide a way to assess latency and duration—in fact, differences between conditions in response latency will be confusedfor differences in amplitude (Calhoun et al., 2004; Lindquist and Wager, 2007). HRF models In this work we study seven HRF models in a series of simulation studies and an application to real data. We briefly introduce each model below, but leave a more detailed description for Section A of the Appendix. The first model under consideration is SPMs canonical HRF (Here denoted GAM), which consists of a linear combination of two gamma functions (Eq. (A1) in the Appendix). To increase its ability to fit responses that are shifted in time or have extended activation durations, we also consider models using the canonical HRF plus its temporal deriva...

Citation Context

...an important role in the analysis of fMRI data. When analyzing the shape of the estimated hemodynamic response function (HRF), summary measures of psychological interest (e.g., amplitude, delay, and duration) can be extracted and used to infer information regarding the intensity, onset latency, and duration of the underlying brain metabolic activity. To date most fMRI studies have been primarily focused on estimating the amplitude of evoked HRFs across different tasks. However, there is a growing interest in studying the time-to-peak and duration of activation as well (Bellgowan et al., 2003; Formisano and Goebel, 2003; Richter et al., 2000). The onset and peak latencies of the5 Amsterdam Ave, 10th Floor, . quist). rights reserved.HRF can provide information about the timing of activation for various brain areas and the width provides information about the duration of activation. However, questions remain regarding which methods for obtaining estimates of these parameters are most efficient while giving rise to the least amount of bias and misspecification. In this paper, we focus on estimation of response amplitude/height (H), time-to-peak (T), and full-width at half-max (W) in the HRF as potential measure...

Citation Context

...to a neural event plays an important role in the analysis of fMRI data. When analyzing the shape of the estimated hemodynamic response function (HRF), summary measures of psychological interest (e.g., amplitude, delay, and duration) can be extracted and used to infer information regarding the intensity, onset latency, and duration of the underlying brain metabolic activity. To date most fMRI studies have been primarily focused on estimating the amplitude of evoked HRFs across different tasks. However, there is a growing interest in studying the time-to-peak and duration of activation as well (Bellgowan et al., 2003; Formisano and Goebel, 2003; Richter et al., 2000). The onset and peak latencies of the5 Amsterdam Ave, 10th Floor, . quist). rights reserved.HRF can provide information about the timing of activation for various brain areas and the width provides information about the duration of activation. However, questions remain regarding which methods for obtaining estimates of these parameters are most efficient while giving rise to the least amount of bias and misspecification. In this paper, we focus on estimation of response amplitude/height (H), time-to-peak (T), and full-width at half-max (W) in ...

Citation Context

...bitrary shape for each event type in every voxel of the brain. There are a number of models that fall somewhere between these two extremes. A popular choice is to use a combination of the canonical HRF and its derivatives with respect to time and dispersion (Friston et al., 1998b; Henson et al., 2002). Other approaches include the use of basis sets composed of principal components (Aguirre et al., 1998; Woolrich et al., 2004), cosine functions (Zarahn, 2002), radial basis functions (Riera et al., 2004), spline basis sets, a Gaussian model (Rajapakse et al., 1998) and spectral basis functions (Liao et al., 2002). Also, a number of researchers have used nonlinear fitting of a canonical function with free parameters for magnitude and onset/ peak delay (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). In general, the more basis functions used in a linear model or the more free parameters in a nonlinear one, the more flexible the model is in measuring the parameters of interest. However, including more parameters also means more potential for error in estimating them, fewer degrees of freedom, and decreased power and validity if regressors are collinear...

Citation Context

... as the shift increases. The IL model and the smooth FIR model did not show large biases, and the ILmodel showedby far the least amount of confusability of all the models that were examined. Both these methods are clearly able to handle even large amounts of model misspecification and uncertainty about the exact timing of the onset and duration of activation. However, for situations when the exact timing and duration of activation are not known a priori (e.g. certain studies of emotion and stress) we recommend using alternative methods based on changepoint analysis (Lindquist and Wager, 2008; Lindquist et al., 2007). In this work we also introduce procedures for performing inference on the estimated summary statistics. In our simulations we find that “nonparametric” bootstrap and sign permutation tests perform adequately with each model, and are roughly comparable in sensitivity to the standard parametric model when model assumptions hold. Use of these models may be advantageous when testing effects that do not have clear parametric p-values, such as the distribution of maxima used in multiple comparisons correction (Nichols and Holmes, 2002), or for which parametric p-values are insensitive (such as med...

Citation Context

...retability of parameter estimates. S188 M.A. Lindquist et al. / NeuroImage 45 (2009) S187–S198dependent, and is not constant over time (Logothetis, 2003). Second, the hemodynamic response is sluggish (i.e., there is hysteresis) and, when it does reflect neuronal/glial activity, it integrates this activity across time. Thus, an increase in the duration of neuronal activity could result in increases in both the amplitude (H) and duration (W) of the evoked BOLD response. Third, the BOLD response is itself a nonlinear integrator, as the vascular response saturates over time (Friston et al., 2000; Vazquez et al., 2006; Wager et al., 2005), further complicating matters. In sum, there is not always a clear relationship between neuronal/glial activity changes and parameters of the evoked BOLD response. The second part of the problem depicted in Fig. 1 is whether the statistical model of the HRF recovers the truemagnitude, time to peak, and width of the response. That is, do changes in the estimate of the height correspond to equivalent changes in the true magnitude of the BOLD response? While the second issue may seem easy to resolve, as we show here, both the use of multiple regression models and presentatio...

Citation Context

...tion in every cognitive event type modeled (Glover, 1999; Goutte et al., 2000; Ollinger et al., 2001). Thus, the model is able to estimate an HRF of arbitrary shape for each event type in every voxel of the brain. There are a number of models that fall somewhere between these two extremes. A popular choice is to use a combination of the canonical HRF and its derivatives with respect to time and dispersion (Friston et al., 1998b; Henson et al., 2002). Other approaches include the use of basis sets composed of principal components (Aguirre et al., 1998; Woolrich et al., 2004), cosine functions (Zarahn, 2002), radial basis functions (Riera et al., 2004), spline basis sets, a Gaussian model (Rajapakse et al., 1998) and spectral basis functions (Liao et al., 2002). Also, a number of researchers have used nonlinear fitting of a canonical function with free parameters for magnitude and onset/ peak delay (Kruggel and von Cramon, 1999; Kruggel et al., 2000; Lindquist and Wager, 2007; Miezin et al., 2000). In general, the more basis functions used in a linear model or the more free parameters in a nonlinear one, the more flexible the model is in measuring the parameters of interest. However, including mo...

Citation Context

... (H), time-to-peak (T), and full-width at half-max (W) in the HRF as potential measures of response magnitude, latency and duration of neuronal activity. Ideally, the parameters of the HRF should be directly interpretable in terms of changes in neuronal activity, and should be estimated so that statistical power is maximized. An accurate estimate of the HRF shape may help prevent both false positive and negative results from arising due to ill-fitting constrained statistical models, as even minor amounts of mis-modeling can lead to severe loss in power and validity (Lindquist and Wager, 2007; Loh et al., 2008). The issue of interpretability is complex, and the problem can be divided into two parts, shown in Fig. 1. The first relates to whether changes in physiological, metabolic-level parameters (e.g. magnitude, delay, and duration of evoked changes in neuronal/glial activity) directly translate into changes in corresponding parameters of the HRF, such as H, T, and W. These physiological parameters are often assumed to be neural in origin as they have been shown to correlate highly with measures of extracellular post-synaptic activity (Logothetis, 2003), but they also have glial components (Schumme...

Citation Context

...modification that adjusts the percentiles to correct for bias and skewness. For more details we refer the interested reader to Efron and Tibshirani (1993). It is important to note that the bootstrap procedure is designed to estimate the sample standard error of a statistic and can therefore be used to construct confidence intervals. The bootstrap distribution is not calculated with a specific null hypothesis in mind and for this we need to use a permutation test. Also, for the specific problem of FWEcontrol using the max distribution of a large number of tests with small M, it has been shown (Troendle et al., 2004) that the Bootstrap can be unstable and permutation tests are to be preferred. (iii) Sign permutation test The final method, the sign permutation procedure (Nichols and Holmes, 2002), is another non-parametric test that is valid for each ofthe models under consideration. The testing procedure can be described as follows: 1. Randomly permute the sign of each value of Hi, i.e. take a resample (x1, x2, …xM) where xi = Hi withprobability 0:5 −Hi withprobability 0:5 : 2. Calculate the sample mean x for each resample. 3. Repeat steps 1 and 2 a total of B (e.g. 5000–10,000) times and use the collecti...

Citation Context

...progressively less accurate as the shift increases. The IL model and the smooth FIR model did not show large biases, and the ILmodel showedby far the least amount of confusability of all the models that were examined. Both these methods are clearly able to handle even large amounts of model misspecification and uncertainty about the exact timing of the onset and duration of activation. However, for situations when the exact timing and duration of activation are not known a priori (e.g. certain studies of emotion and stress) we recommend using alternative methods based on changepoint analysis (Lindquist and Wager, 2008; Lindquist et al., 2007). In this work we also introduce procedures for performing inference on the estimated summary statistics. In our simulations we find that “nonparametric” bootstrap and sign permutation tests perform adequately with each model, and are roughly comparable in sensitivity to the standard parametric model when model assumptions hold. Use of these models may be advantageous when testing effects that do not have clear parametric p-values, such as the distribution of maxima used in multiple comparisons correction (Nichols and Holmes, 2002), or for which parametric p-values are...

Citation Context

...f response amplitude/height (H), time-to-peak (T), and full-width at half-max (W) in the HRF as potential measures of response magnitude, latency and duration of neuronal activity. Ideally, the parameters of the HRF should be directly interpretable in terms of changes in neuronal activity, and should be estimated so that statistical power is maximized. An accurate estimate of the HRF shape may help prevent both false positive and negative results from arising due to ill-fitting constrained statistical models, as even minor amounts of mis-modeling can lead to severe loss in power and validity (Lindquist and Wager, 2007; Loh et al., 2008). The issue of interpretability is complex, and the problem can be divided into two parts, shown in Fig. 1. The first relates to whether changes in physiological, metabolic-level parameters (e.g. magnitude, delay, and duration of evoked changes in neuronal/glial activity) directly translate into changes in corresponding parameters of the HRF, such as H, T, and W. These physiological parameters are often assumed to be neural in origin as they have been shown to correlate highly with measures of extracellular post-synaptic activity (Logothetis, 2003), but they also have glial ...