Results 1  10
of
49
Averaging estimators for regressions with a possible structural break. Econometric Theory 35
, 2009
"... This paper investigates selection and averaging of linear regressions with a possible structural break. Our main contribution is the construction of a Mallows criterion for the structural break model. We show that the correct penalty term is nonstandard and depends on unknown parameters, but it can ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
This paper investigates selection and averaging of linear regressions with a possible structural break. Our main contribution is the construction of a Mallows criterion for the structural break model. We show that the correct penalty term is nonstandard and depends on unknown parameters, but it can be approximated by an average of limiting cases to yield a feasible penalty with good performance. Following Hansen (2007, Econometrica 75, 1175–1189) we recommend averaging the structural break estimates with the nobreak estimates where the weight is selected to minimize the Mallows criterion. This estimator is simple to compute, as the weights are a simple function of the ratio of the penalty to the Andrews SupF test statistic. To assess performance we focus on asymptotic meansquared error (AMSE) in a local asymptotic framework. We show that the AMSE of the estimators depends exclusively on the parameter variation function. Numerical comparisons show that the unrestricted leastsquares and pretest estimators have very large AMSE for certain regions of the parameter space, whereas our averaging estimator has AMSE close to the infeasible optimum. 1.
Ensemble Learning
, 2011
"... This note presents a chronological review of the literature on ensemble learning which has accumulated over the past twenty years. The idea of ensemble learning is to employ multiple learners and combine their predictions. If we have a committee of M models with uncorrelated errors, simply by averag ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
This note presents a chronological review of the literature on ensemble learning which has accumulated over the past twenty years. The idea of ensemble learning is to employ multiple learners and combine their predictions. If we have a committee of M models with uncorrelated errors, simply by averaging them the average error of a model can be reduced by a factor of M. Unfortunately, the key assumption that the errors due to the individual models are uncorrelated is unrealistic; in practice, the errors are typically highly correlated, so the reduction in overall error is generally small. However, by making use of Cauchy’s inequality, it can be shown that the expected committee error will not exceed the expected error of the constituent models. In this article the literature in general is reviewed, with, where possible, an emphasis on both theory and practical advice, then a taxonomy is provided, and finally four ensemble methods are covered in greater detail: bagging, boosting (including AdaBoost), stacked generalization and the random subspace method. Ensemble Learning
BonferroniBased SizeCorrection for Nonstandard Testing Problems ∗
, 2011
"... We develop powerful new sizecorrection procedures for nonstandard hypothesis testing environments in which the asymptotic distribution of a test statistic is discontinuous in a parameter under the null hypothesis. Examples of this form of testing problem are pervasive in econometrics and complicate ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
We develop powerful new sizecorrection procedures for nonstandard hypothesis testing environments in which the asymptotic distribution of a test statistic is discontinuous in a parameter under the null hypothesis. Examples of this form of testing problem are pervasive in econometrics and complicate inference by making size difficult to control. This paper introduces two new sizecorrection methods that correspond to two different general hypothesis testing frameworks. They are designed to maximize the power of the underlying test while maintaining correct asymptotic size uniformly over the parameter space specified by the null hypothesis. The new methods involve the construction of critical values that make use of reasoning derived from Bonferroni bounds. The first new method provides a complementary alternative to existing sizecorrection methods, entailing substantially higher power for many testing problems. The second new method provides the first available asymptotically sizecorrect testing methodology for the general class of testing problems to which it applies. This class includes hypothesis tests on parameters after consistent model selection and tests on superefficient/hardthresholding estimators. We detail the construction and performance of the new tests in three specific examples: testing after conservative model selection, testing when a nuisance parameter may be on a boundary and testing after consistent model selection.
1 Corresponding author:
, 2010
"... ISSN 09247815Expected utility and catastrophic risk in a stochastic economyclimate model ∗ ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
ISSN 09247815Expected utility and catastrophic risk in a stochastic economyclimate model ∗
Robust Confidence Intervals in Nonlinear Regression under Weak Identification
, 2008
"... In this paper, we develop a practical procedure to construct confidence intervals (CIs) in a weakly identified nonlinear regression model. When the coefficient of a nonlinear regressor is small, modelled here as local to zero, the signal from the respective nonlinear regressor is weak, resulting in ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
In this paper, we develop a practical procedure to construct confidence intervals (CIs) in a weakly identified nonlinear regression model. When the coefficient of a nonlinear regressor is small, modelled here as local to zero, the signal from the respective nonlinear regressor is weak, resulting in weak identification of the unknown parameters within the nonlinear regression component. In such cases, standard asymptotic theory can provide a poor approximation to finitesample behavior and failure to address the problem can produce misleading inferences. This paper seeks to tackle this problem in complementary ways. First, we develop a local limit theory that provides a uniform approximation to the finitesample distribution irrespective of the strength of identification. Second, standard CIs based on conventional normal or chisquared approximations as well as subsampling CIs are shown to be prone to size distortions that can be severe. Third, a new con…dence interval (CI) is constructed that has good finitesample coverage probability. Simulation results show that when the nonlinear function is a BoxCox type transformation, the nominal 95 % standard CI and subsampling CI have asymptotic sizes of 53 % and 2.3%, respectively. In contrast, the robust CI has correct asymptotic size and a finitesample coverage probability of 93.4 % when sample size is 100.
A PlugIn Averaging Estimator for Regressions with Heteroskedastic Errors
, 2011
"... This paper proposes a novel model averaging estimator for the linear regression model with heteroskedastic errors. Unlike model selection which picks the single model among the candidate models, model averaging, on the other hand, incorporates all the information by averaging over all potential mode ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
This paper proposes a novel model averaging estimator for the linear regression model with heteroskedastic errors. Unlike model selection which picks the single model among the candidate models, model averaging, on the other hand, incorporates all the information by averaging over all potential models. The two main questions of concern are: (1) How do we assign the weights for candidate models? (2) What is the asymptotic distribution of the averaging estimator and how do wemake inference? Thispaperseeks to tackle these two problemsfrom afrequentist view. First, we derive the asymptotic distribution of the averaging estimator with fixed weights in a local asymptotic framework. The optimal weights are obtained by minimizing the asymptotic meansquared error (AMSE) of the averaging estimator. Second, we propose a plugin averaging estimator which selects the weights by minimizing the sample analog of the AMSE. The asymptotic distribution of the proposed estimator is derived. Third, we show that the confidence intervals based on normal approximations suffer from size distortions. We suggest a plugin method to construct the confidence interval which has good finitesample coverage probability. The simulation results show that the plugin averaging estimator performs favorably compared with other existing model selection and model averaging methods. As an empirical illustration, the proposed methodology is applied to estimate the effect of the studentteacher ratio on student achievement. We find that the insignificance of the studentteacher ratio variable from previous literature could be potentially explained by the fact of ignoring the model uncertainty.
Efficient shrinkage in parametric models
, 2011
"... This paper introduces shrinkage for general parametric models. We show how to shrink maximum likelihood estimators towards parameter subspaces defined by general nonlinear restrictions. We derive the asymptotic distribution and risk of a shrinkage estimator using a local asymptotic framework. We sho ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
This paper introduces shrinkage for general parametric models. We show how to shrink maximum likelihood estimators towards parameter subspaces defined by general nonlinear restrictions. We derive the asymptotic distribution and risk of a shrinkage estimator using a local asymptotic framework. We show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage estimator is strictly less than that of the MLE. This reduction holds globally in the parameter space. We show that the reduction in asymptotic risk is substantial, even for moderately large values of the parameters. The risk formula simplify in a very convenient way in the context of high dimensional models. We derive a simple bound for the asymptotic risk. We also provide a new large sample minimax efficiency bound. We use the concept of local asymptotic minimax bounds, a generalization of the conventional asymptotic minimax bounds. The difference is that we consider minimax regions that are defined locally to the parametric restriction, and are thus tighter. We show that our shrinkage estimator asymptotically achieves this local asymptotic minimax bound when the shrinkage dimension is high. This theory is a combination and extension of standard asymptotic efficiency theory (Hájek, 1972) and local minimax efficiency theory for Gaussian models (Pinsker, 1980).