## What to do about missing values in time series cross-section data (2009)

### Cached

### Download Links

Citations: | 18 - 5 self |

### BibTeX

@MISC{Honaker09whatto,

author = {James Honaker and Gary King},

title = {What to do about missing values in time series cross-section data },

year = {2009}

}

### OpenURL

### Abstract

Applications of modern methods for analyzing data with missing values, based primarily on multiple imputation, have in the last half-decade become common in American politics and political behavior. Scholars in this subset of political science have thus increasingly avoided the biases and inefficiencies caused by ad hoc methods like listwise deletion and best guess imputation. However, researchers in much of comparative politics and international relations, and others with similar data, have been unable to do the same because the best available imputation methods work poorly with the time-series cross section data structures common in these fields. Weattempttorectify this situation with three related developments. First, we build a multiple imputation model that allows smooth time trends, shifts across cross-sectional units, and correlations over time and space, resulting in far more accurate imputations. Second, we enable analysts to incorporate knowledge from area studies experts via priors on individual missing cell values, rather than on difficult-to-interpret model parameters. Third, because these tasks could not be accomplished within existing imputation algorithms, in that they cannot handle as many variables as needed even in the simpler cross-sectional data for which they were designed, we also develop a new algorithm that substantially expands the range of computationally feasible data types and sizes for which multiple imputation can be used. These developments also make it possible to implement the methods introduced here in freely available open source software that is considerably more reliable than existing algorithms. We develop an approach to analyzing data with

### Citations

2613 |
Times Series Analysis
- Hamilton
- 1994
(Show Context)
Citation Context ...deal with TSCS data, analyzing the resulting multiply imputed data set still requires the same attention that one would give to TSCS problems as if the data had been fully observed (see, for example, =-=Hamilton, 1994-=-; Beck and Katz, 1995). A A Generalized Version of Data Augmentation Priors within EM A.1 Notation As in the body of the paper, elements of the missingness matrix, M, are 1 when missing and 0 when obs... |

2295 | Economic Growth
- Sala-i-Martin
- 1995
(Show Context)
Citation Context ...ship. 6.2 Explaining Economic Growth For our second example we reestimate key results from Baum and Lake (2003), who are interested in the effect of democracy on economic growth, both directly (as in =-=Barro, 1997-=-), and indirectly through its intermediate effects on female life expectancy and female secondary education. We reproduce their recursive regression system of linear specifications, using our imputati... |

974 |
Multiple Imputation for Nonresponse in Surveys
- Rubin
- 1987
(Show Context)
Citation Context ...e times and combining results is routinely and transparently handled by a variety of statistical analysis software. As a result, after careful imputation, analysts can ignore the missingness problem (=-=Rubin, 1987-=-; King, Honaker, Joseph and Scheve, 2001). Existing multiple imputation methods work well for up to 30–40 variables from sample surveys and other data with similar rectangular, exchangeable, nonhierar... |

907 |
Analysis of Panel Data
- Hsiao
- 1986
(Show Context)
Citation Context ... analyzing the resulting multiply imputed data set still requires the same attention that one would give to TSCS problems as if the data had been fully observed (see, for example, Beck and Katz 1995; =-=Hsiao 2003-=-). Appendix A. Generalized Version of Data Augmentation Priors within EM Notation As in the body of the paper, elements of the missingness matrix, M, are 1 when missing and 0 when observed. For notati... |

554 |
Analysis of Incomplete Multivariate Data
- Schafer
- 1997
(Show Context)
Citation Context ...he likelihood and so represent no new information even though they enable the analyst to avoid listwise deleting any unit that is not fully observed on all variables. 5sfor categorical or mixed data (=-=Schafer, 1997-=-; Schafer and Olsen, 1998). All the innovations in this paper would easily apply to these more complicated alternative models, but we keep to the simpler normal case here. Furthermore, as long as the ... |

451 | Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion - DEMPSTER, LAIRD, et al. - 1977 |

432 | Why do more open economies have bigger governments - Rodrik - 1998 |

310 | Making the Most of Statistical Analyses: Improving Interpretation and Presentation - King, Tomz, et al. - 2000 |

237 | Analyzing Incomplete Political Science Data: An Alternative Algorithm for Multiple Imputation.” American Political Science Review 95(1):49–69
- King, Honaker, et al.
- 2001
(Show Context)
Citation Context ... 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 handled by special purpose statistical analysis software. As a result, after careful imputation, analysts can ignore the missingness problem (=-=King et al. 2001-=-; Rubin 1987). Commonly used multiple imputation methods work well for up to 30–40 variables from sample surveys and other data with similar rectangular, nonhierarchical properties, such as from surve... |

223 |
Statistical analysis with missing data (2nd ed
- Little, Rubin
- 2002
(Show Context)
Citation Context ...techniques routinely produce biased and inefficient inferences, standard errors, and confidence intervals, and they are almost uniformly dominated by appropriate multiple imputation-based approaches (=-=Little and Rubin, 2002-=-). 1 1 King et al. (2001) show that, with the average amount of missingness evident in political science articles, using listwise deletion under the most optimistic of assumptions causes estimates to ... |

145 |
Varieties of selection bias
- Heckman
- 1991
(Show Context)
Citation Context ...lling in observations and then deleting some rows from the data matrix, is too difficult to do properly; and although methods of analysis adapted to the swiss cheese in its original form exist (e.g., =-=Heckman 1990-=-; King et al. 2004), they are mostly not available for missing data scattered across both dependent and explantory variables. Instead, what multiple imputation does is to fill in the holes in the data... |

116 | Greed and Grievance - Collier, Hoeffler - 2004 |

89 | Anke Hoeffler, 2004. ‘Greed and Grievance - Collier |

84 |
What to do (and not to do) with Time-Series-CrossSection Data
- Beck
- 1999
(Show Context)
Citation Context ...ata, analyzing the resulting multiply imputed data set still requires the same attention that one would give to TSCS problems as if the data had been fully observed (see, for example, Hamilton, 1994; =-=Beck and Katz, 1995-=-). A A Generalized Version of Data Augmentation Priors within EM A.1 Notation As in the body of the paper, elements of the missingness matrix, M, are 1 when missing and 0 when observed. For notational... |

75 | Predictive Model Selection
- Laud, Ibrahim
- 1995
(Show Context)
Citation Context ...g (2007), putting priors on observations and then finding the implied prior on coefficients has appeared in work on prior elicitation (see Gill and Walker, 2005; Ibrahim and Chen, 1997; Kadane, 1980; =-=Laud and Ibrahim, 1995-=-; Weiss, Wang and Ibrahim, 1997), predictive inference (West, Harrison and Migon, 1985; Tsutakawa and Lin, 1986; Tsutakawa, 1992b), wavelet analysis (Jefferys et al., 2001), and logistic (Clogg et al.... |

68 | Primary Commodity Exports and Civil War - Fearon - 2005 |

65 | Multiple imputation for multivariate missing-data problems: A data analyst’s perspective
- Schafer, Olsen
- 1998
(Show Context)
Citation Context ...nd so represent no new information even though they enable the analyst to avoid listwise deleting any unit that is not fully observed on all variables. 5sfor categorical or mixed data (Schafer, 1997; =-=Schafer and Olsen, 1998-=-). All the innovations in this paper would easily apply to these more complicated alternative models, but we keep to the simpler normal case here. Furthermore, as long as the imputation model contains... |

56 |
Multiple-imputation inferences with uncongenial sources of input (disc
- Meng
- 1994
(Show Context)
Citation Context ...rthermore, as long as the imputation model contains at least as much information as the variables in the analysis model, using an analysis model that is neither normal nor linear generates no biases (=-=Meng, 1994-=-). In fact, the two-step nature of multiple imputation has two advantages over “optimal” one-step approaches. First, including variables or information in the imputation model not needed in the analys... |

46 | 2008): Demographic Forecasting - Girosi, King |

45 |
A missing information principle: Theory and Applications
- Orchard, A
- 1972
(Show Context)
Citation Context ... the posterior density of imputed values for priors of different strengths. As � 9 Although the first applications of the EM algorithm were for missing data problems (Dempster, Laird, and Rubin 1977; =-=Orchard and Woodbury 1972-=-), its use and usefulness has expanded to many maximum-likelihood applications (McLachlan and Krishan 2008), and as the conventional M-step is a likelihood maximization EM is considered a maximum-like... |

41 | Making the Most of Statistical Analyses - King, Tomz, et al. - 2000 |

38 |
The Invisible Hand of Democracy
- Lake, Baum
- 2001
(Show Context)
Citation Context ...hers sometimes discard information by aggregating covariates into five-or ten-year averages, losing variation on the dependent variable within the averages (see for example, Iversen and Soskice 2006; =-=Lake and Baum 2001-=-; Moene and Wallerstein 2001; and Timmons 2005, respectively). Obviously this procedure can reduce the number of observations on the dependent variable by 80 or 90%, limits the complexity of possible ... |

37 | Is Democracy Good for the Poor - Ross - 2006 |

36 |
The Dangers of Extreme Counterfactuals,” Political Analysis, forthcoming, copy at http://gking.harvard.edu/files/abs/counterft-abs.shtml
- King, Zeng
- 2006
(Show Context)
Citation Context ...olynomials work better for interpolation than extrapolation, and so missing values at the end of a series will have larger confidence intervals, but the degree of model dependence may be even larger (=-=King and Zeng, 2006-=-).) Since trends over time in one unit may not be related to other units, when using this option we also include interactions of the polynomials with the crosssectional unit. When the polynomial of ti... |

35 |
Bootstrap for imputed survey data
- Shao, Sitter
- 1996
(Show Context)
Citation Context ... with a bootstrapping algorithm. Creative applications of bootstrapping have been developed for several application-specific missing data problems (Rubin and Schenker, 1986; Rubin, 1994; Efron, 1994; =-=Shao and Sitter, 1996-=-; Lahlrl, 2003), but to our knowledge the technique has not been used to develop and implement a general purpose multiple imputation algorithm. The result is conceptually simple and easy to implement.... |

32 | The Political Economy of Growth: Democracy and Human Capital - Baum, Lake - 2003 |

29 |
Enhancing the Validity and
- King, Murray, et al.
- 2004
(Show Context)
Citation Context ...vations and then deleting some rows from the data matrix, is too difficult to do properly; and although methods of analysis adapted to the swiss cheese in its original form exist (e.g., Heckman 1990; =-=King et al. 2004-=-), they are mostly not available for missing data scattered across both dependent and explantory variables. Instead, what multiple imputation does is to fill in the holes in the data using a predictiv... |

28 | A new perspective on priors for generalized linear models - Bedrick, Christensen, et al. - 1996 |

26 |
Multiple imputation of industry and occupation codes in census public-use samples using Bayesian logistic regression
- CLOGG, RUBIN, et al.
- 1991
(Show Context)
Citation Context ...h weights for the pseudo-observations translated from the variance of the prior hyperparameter, and then running the same algorithm as if there were no priors (Bedrick, Christensen, and Johnson 1996; =-=Clogg et al. 1991-=-; Tsutakawa 1992a). Empirical priors (as in Schafer 1997, 155) can be implemented as DAPs. Unfortunately, implementing priors at the observation-level solely via current DAP technology would not work ... |

26 |
Electoral Institutions and the Politics of Coalitions: Why Some Democracies Redistribute More than Others
- Iversen, Soskice
- 2006
(Show Context)
Citation Context ...etter procedure, researchers sometimes discard information by aggregating covariates into five- or ten-year averages, losing variation on the dependent variable within the averages (see, for example, =-=Iversen and Soskice 2006-=-; Lake and Baum 2001; Moene and Wallerstein 2001; and Timmons 2005, respectively). Obviously this procedure can reduce the number of observations on the dependent variable by 80 or 90%, limits the com... |

23 |
Missing data, imputation, and the bootstrap
- Efron
- 1994
(Show Context)
Citation Context ...erior density with a bootstrapping algorithm. Creative applications of bootstrapping have been developed for several application-specific missing data problems (Rubin and Schenker, 1986; Rubin, 1994; =-=Efron, 1994-=-; Shao and Sitter, 1996; Lahlrl, 2003), but to our knowledge the technique has not been used to develop and implement a general purpose multiple imputation algorithm. The result is conceptually simple... |

21 |
Armed conflict as a public health problem
- Murray, King, et al.
- 2002
(Show Context)
Citation Context ...n most countries vital registration systems do not operate during wartime, and mortality due to war, which is surely higher due to the direct and indirect consequences of the conflict, is unobserved (=-=Murray et al. 2002-=-). And a final example would be where we do not have much raw information about the level of a variable in a country, but we believe that it is similar to the observed data in a neighboring country. W... |

16 |
Models for discrete longitudinal data. 1st ed
- Molenberghs, Verbeke
- 2005
(Show Context)
Citation Context ...n missing data has come to tackling TSCS data would seem to be “repeated measures” designs, where clinical patients are observed over a small number of irregularly spaced time intervals (Little 1995; =-=Molenberghs and Verbeke 2005-=-). Missingness occurs principally in the dependent variable (the patient’s response to treatment) and largely due to attrition, leading to monotone missingness patterns. As attrition is often due to a... |

15 | Amelia II: A Program for Missing Data.” http://gking.harvard.edu/amelia - Honaker, King, et al. - 2007 |

11 | Elicited priors for Bayesian model specifications in political science research
- Gill, Walker
- 2005
(Show Context)
Citation Context ...approach introduced for hierarchical models in Girosi and King (2007), putting priors on observations and then finding the implied prior on coefficients has appeared in work on prior elicitation (see =-=Gill and Walker, 2005-=-; Ibrahim and Chen, 1997; Kadane, 1980; Laud and Ibrahim, 1995; Weiss, Wang and Ibrahim, 1997), predictive inference (West, Harrison and Migon, 1985; Tsutakawa and Lin, 1986; Tsutakawa, 1992b), wavele... |

11 |
Predictive and Structural Methods for Eliciting Prior Distributions
- Kadane
- 1980
(Show Context)
Citation Context ...Girosi and King (2007), putting priors on observations and then finding the implied prior on coefficients has appeared in work on prior elicitation (see Gill and Walker, 2005; Ibrahim and Chen, 1997; =-=Kadane, 1980-=-; Laud and Ibrahim, 1995; Weiss, Wang and Ibrahim, 1997), predictive inference (West, Harrison and Migon, 1985; Tsutakawa and Lin, 1986; Tsutakawa, 1992b), wavelet analysis (Jefferys et al., 2001), an... |

10 | Predictive model selection for repeated measures random effects models using Bayes factors. Biometrics 53:159–169 - Weiss, Wang, et al. - 1997 |

9 | Primary Commodity Exports and Civil - Fearon - 2005 |

7 | Correcting for Selective Nonresponse in the National Longitudinal Survey of Youth Using Multiple Imputation’, The - Davey, Shanahan, et al. - 2001 |

6 | Do Economic Sanctions Destabilize Country Leaders - Marinov - 2005 |

5 |
Multiple Imputation for Interval Estimation for Simple Random Samples with Ignorable Nonresponse
- Rubin, Schenker
- 1986
(Show Context)
Citation Context ...cess of drawing µ and Σ from their posterior density with a bootstrapping algorithm. Creative applications of bootstrapping have been developed for several application-specific missing data problems (=-=Rubin and Schenker, 1986-=-; Rubin, 1994; Efron, 1994; Shao and Sitter, 1996; Lahlrl, 2003), but to our knowledge the technique has not been used to develop and implement a general purpose multiple imputation algorithm. The res... |

5 |
Economic Growth”Cambridge
- Barro, Sala-i-Martin
- 1999
(Show Context)
Citation Context ...Democracies .373 .393 (.094) (.081) N 1966 5627 The table shows the effect of being a democracy on life expectancy and on the percentage enrolled in secondary education (with pvalues in parentheses). =-=Barro 1997-=-) and indirectly through its intermediate effects on female life expectancy and female secondary education. We reproduce their recursive regression system of linear specifications, using our imputatio... |

4 |
Prior Distribution for Item Response Curves
- Tsutakawa
- 1992
(Show Context)
Citation Context ...pseudo-observations to the data set, with weights for the pseudo-observations translated from the variance of the prior hyperparameter, and then running the same algorithm as if there were no priors (=-=Tsutakawa, 1992-=-a; Clogg et al., 1991; Bedrick, Christensen and Johnson, 1996). Empirical priors, discussed in Section 2, can be implemented as DAPs. 7 In addition to the formal approach introduced for hierarchical m... |

3 |
Missing Data, Imputation, and the Bootstrap
- Rubin
- 1994
(Show Context)
Citation Context ...om their posterior density with a bootstrapping algorithm. Creative applications of bootstrapping have been developed for several application-specific missing data problems (Rubin and Schenker, 1986; =-=Rubin, 1994-=-; Efron, 1994; Shao and Sitter, 1996; Lahlrl, 2003), but to our knowledge the technique has not been used to develop and implement a general purpose multiple imputation algorithm. The result is concep... |

3 |
The bootstrap. Handbook of Econometrics 5
- HOROWITZ
- 2000
(Show Context)
Citation Context ...to make the observations conditionally independent. Although we have implemented more sophisticated bootstrap algorithms for when conditional independence cannot be accomplished by adding covariates (=-=Horowitz 2001-=-), we have thus far not found them necessary in practice. 4 Extreme situations, such as small data sets with bootstrapped samples that happen to have constant values or collinearity, should not be dro... |

3 |
Modelling the dropout mechanism
- Little
- 1995
(Show Context)
Citation Context ... literature on missing data has come to tackling TSCS data would seem to be “repeated measures” designs, where clinical patients are observed over a small number of irregularly spaced time intervals (=-=Little 1995-=-; Molenberghs and Verbeke 2005). Mmissingness occurs principally in the dependent variable (the patient’s response to treatment) and largely due to attrition, leading to monotone missingness patterns.... |

2 | Understanding the Lee-Carter Mortality Forecasting Method.”. http://gking.harvard.edu/files/abs/lc-abs.shtml - Girosi, King - 2007 |

2 |
Data Augmentation Priors for Bayesian and
- Greenland, Christensen
- 2001
(Show Context)
Citation Context ...5; Tsutakawa and Lin, 1986; Tsutakawa, 1992b), wavelet analysis (Jefferys et al., 2001), and logistic (Clogg et al., 1991) and other generalized linear models (Bedrick, Christensen and Johnson, 1996; =-=Greenland and Christensen, 2001-=-; Greenland, 2001). 18sUnfortunately, implementing priors at the observation-level solely via current DAP technology would not work well for imputation problems. The first issue is that we will someti... |

2 | Ado About Nothing: A Comparion of Missing Data Methods and Software to Fit Incomplete Data Regression Models.” The American Statistician 61(1, February):79–90 - “Much |

2 |
Predictive Variable Selection for the Multivariate Linear Model.” Biometrics 53(June):465–478
- Ibrahim, Chen
- 1997
(Show Context)
Citation Context ... hierarchical models in Girosi and King (2007), putting priors on observations and then finding the implied prior on coefficients has appeared in work on prior elicitation (see Gill and Walker, 2005; =-=Ibrahim and Chen, 1997-=-; Kadane, 1980; Laud and Ibrahim, 1995; Weiss, Wang and Ibrahim, 1997), predictive inference (West, Harrison and Migon, 1985; Tsutakawa and Lin, 1986; Tsutakawa, 1992b), wavelet analysis (Jefferys et ... |