Results 1  10
of
128
Large Sample Sieve Estimation of SemiNonparametric Models
 Handbook of Econometrics
, 2007
"... Often researchers find parametric models restrictive and sensitive to deviations from the parametric specifications; seminonparametric models are more flexible and robust, but lead to other complications such as introducing infinite dimensional parameter spaces that may not be compact. The method o ..."
Abstract

Cited by 185 (19 self)
 Add to MetaCart
Often researchers find parametric models restrictive and sensitive to deviations from the parametric specifications; seminonparametric models are more flexible and robust, but lead to other complications such as introducing infinite dimensional parameter spaces that may not be compact. The method of sieves provides one way to tackle such complexities by optimizing an empirical criterion function over a sequence of approximating parameter spaces, called sieves, which are significantly less complex than the original parameter space. With different choices of criteria and sieves, the method of sieves is very flexible in estimating complicated econometric models. For example, it can simultaneously estimate the parametric and nonparametric components in seminonparametric models with or without constraints. It can easily incorporate prior information, often derived from economic theory, such as monotonicity, convexity, additivity, multiplicity, exclusion and nonnegativity. This chapter describes estimation of seminonparametric econometric models via the method of sieves. We present some general results on the large sample properties of the sieve estimates, including consistency of the sieve extremum estimates, convergence rates of the sieve Mestimates, pointwise normality of series estimates of regression functions, rootn asymptotic normality and efficiency of sieve estimates of smooth functionals of infinite dimensional parameters. Examples are used to illustrate the general results.
Piecewise linear regularized solution paths,
 The Annals of Statistics,
, 2007
"... Abstract We consider the generic regularized optimization problemβ(λ) = arg min β L(y, Xβ) + λJ(β). Recently, ..."
Abstract

Cited by 140 (9 self)
 Add to MetaCart
(Show Context)
Abstract We consider the generic regularized optimization problemβ(λ) = arg min β L(y, Xβ) + λJ(β). Recently,
Quantile Regression  An introduction
, 2000
"... Quantile regression as introduced in Koenker and Bassett (1978) may be viewed as a natural extension of classical least squares estimation of conditional ..."
Abstract

Cited by 61 (2 self)
 Add to MetaCart
Quantile regression as introduced in Koenker and Bassett (1978) may be viewed as a natural extension of classical least squares estimation of conditional
Nonparametric quantile estimation
, 2006
"... In regression, the desired estimate of yx is not always given by a conditional mean, although this is most common. Sometimes one wants to obtain a good estimate that satisfies the property that a proportion, τ, of yx, will be below the estimate. For τ = 0.5 this is an estimate of the median. What ..."
Abstract

Cited by 55 (9 self)
 Add to MetaCart
(Show Context)
In regression, the desired estimate of yx is not always given by a conditional mean, although this is most common. Sometimes one wants to obtain a good estimate that satisfies the property that a proportion, τ, of yx, will be below the estimate. For τ = 0.5 this is an estimate of the median. What might be called median regression, is subsumed under the term quantile regression. We present a nonparametric version of a quantile estimator, which can be obtained by solving a simple quadratic programming problem and provide uniform convergence statements and bounds on the quantile property of our estimator. Experimental results show the feasibility of the approach and competitiveness of our method with existing ones. We discuss several types of extensions including an approach to solve the quantile crossing problems, as well as a method to incorporate prior qualitative knowledge such as monotonicity constraints. 1.
ℓ1 Trend Filtering
, 2007
"... The problem of estimating underlying trends in time series data arises in a variety of disciplines. In this paper we propose a variation on HodrickPrescott (HP) filtering, a widely used method for trend estimation. The proposed ℓ1 trend filtering method substitutes a sum of absolute values (i.e., ..."
Abstract

Cited by 51 (7 self)
 Add to MetaCart
(Show Context)
The problem of estimating underlying trends in time series data arises in a variety of disciplines. In this paper we propose a variation on HodrickPrescott (HP) filtering, a widely used method for trend estimation. The proposed ℓ1 trend filtering method substitutes a sum of absolute values (i.e., an ℓ1norm) for the sum of squares used in HP filtering to penalize variations in the estimated trend. The ℓ1 trend filtering method produces trend estimates that are piecewise linear, and therefore is well suited to analyzing time series with an underlying piecewise linear trend. The kinks, knots, or changes in slope, of the estimated trend can be interpreted as abrupt changes or events in the underlying dynamics of the time series. Using specialized interiorpoint methods, ℓ1 trend filtering can be carried out with not much more effort than HP filtering; in particular, the number of arithmetic operations required grows linearly with the number of data points. We describe the method and some of its basic properties, and give some illustrative examples. We show how the method is related to ℓ1 regularization based methods in sparse signal recovery and feature selection, and list some extensions of the basic method.
Quantile Regression Forests
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2006
"... Random Forests were introduced as a Machine Learning tool in Breiman (2001) and have since proven to be very popular and powerful for highdimensional regression and classification. For regression, Random Forests give an accurate approximation of the conditional mean of a response variable. It is sh ..."
Abstract

Cited by 47 (0 self)
 Add to MetaCart
Random Forests were introduced as a Machine Learning tool in Breiman (2001) and have since proven to be very popular and powerful for highdimensional regression and classification. For regression, Random Forests give an accurate approximation of the conditional mean of a response variable. It is shown here that Random Forests provide information about the full conditional distribution of the response variable, not only about the conditional mean. Conditional quantiles can be inferred with Quantile Regression Forests, a generalisation of Random Forests. Quantile Regression Forests give a nonparametric and accurate way of estimating conditional quantiles for highdimensional predictor variables. The algorithm is shown to be consistent. Numerical examples suggest that the algorithm is competitive in terms of predictive power.
Monotone Bspline Smoothing
 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
, 1996
"... Estimation of growth curves or item response curves often involves monotone data smoothing. Methods that have been studied in the literature tend to be either less flexible or more difficult to compute when constraints such as monotonicity are incorporated. Built upon the ideas of Ramsay (1988) and ..."
Abstract

Cited by 32 (3 self)
 Add to MetaCart
Estimation of growth curves or item response curves often involves monotone data smoothing. Methods that have been studied in the literature tend to be either less flexible or more difficult to compute when constraints such as monotonicity are incorporated. Built upon the ideas of Ramsay (1988) and Koenker, Ng and Portnoy (1994), we propose monotone Bspline smoothing based on L 1 optimization. It inherits the desirable properties of spline approximations and the computational efficiency of linear programs. The constrained fit is similar to the unconstrained estimate in terms of computational complexity and asymptotic rate of convergence. Through applications to some real and simulated data we show that the method is useful in a variety of applications. The basic ideas utilized in monotone smoothing can be useful in some other constrained function estimation problems.
COBS: Qualitatively Constrained Smoothing via Linear Programming
, 1999
"... this paper, we attempt to bring the problem of constrained spline smoothing to the foreground and describe the details of a constrained Bspline smoothing (COBS) algorithm that is being made available to Splus users. Recent work of He & Shi (1998) considered a special case and showed that the L ..."
Abstract

Cited by 31 (5 self)
 Add to MetaCart
this paper, we attempt to bring the problem of constrained spline smoothing to the foreground and describe the details of a constrained Bspline smoothing (COBS) algorithm that is being made available to Splus users. Recent work of He & Shi (1998) considered a special case and showed that the L 1 projection of a smooth function into the space of Bsplines provides a monotone smoother that is flexible, efficient and achieves the optimal rate of convergence. Several options and generalizations are included in COBS: it can handle small or large data sets either with user interaction or full automation. Three examples are provided to show how COBS works in a variety of realworld applications.
Penalized Triograms: Total Variation Regularization for Bivariate Smoothing, preprint
, 2002
"... Abstract. Hansen, Kooperberg, and Sardy (1998) introduced a family of continuous, piecewise linear functions defined over adaptively selected triangulations of the plane as a general approach to statistical modeling of bivariate densities, regression and hazard functions. These triograms enjoy a nat ..."
Abstract

Cited by 28 (4 self)
 Add to MetaCart
Abstract. Hansen, Kooperberg, and Sardy (1998) introduced a family of continuous, piecewise linear functions defined over adaptively selected triangulations of the plane as a general approach to statistical modeling of bivariate densities, regression and hazard functions. These triograms enjoy a natural affine equivariance that offers distinct advantages over competing tensor product methods that are more commonly used in statistical applications. Triograms employ basis functions consisting of linear “tent functions ” defined with respect to a triangulation of a given planar domain. As in knot selection for univariate splines, Hansen, et al adopt the regression spline approach of Stone (1994). Vertices of the triangulation are introduced or removed sequentially in an effort to balance fidelity to the data and parsimony. In this paper we explore a smoothing spline variant of the triogram model based on a roughness penalty adapted to the piecewise linear structure of the triogram model. We show that the proposed roughness penalty may be interpreted as a total variation penalty on the gradient of the fitted function. The methods are illustrated with two artificial examples and with an application to estimated quantile surfaces of land value in the Chicago metropolitan area. “Goniolatry, or the worship of angles,...” Pynchon (1997) 1.
Variable selection in quantile regression
 Statistics Sinica
, 2009
"... Abstract: After its inception in Koenker and Bassett (1978), quantile regression has become an important and widely used technique to study the whole conditional distribution of a response variable and grown into an important tool of applied statistics over the last three decades. In this work, we f ..."
Abstract

Cited by 25 (1 self)
 Add to MetaCart
Abstract: After its inception in Koenker and Bassett (1978), quantile regression has become an important and widely used technique to study the whole conditional distribution of a response variable and grown into an important tool of applied statistics over the last three decades. In this work, we focus on the variable selection aspect of penalized quantile regression. Under some mild conditions, we demonstrate the oracle properties of the SCAD and adaptiveLASSO penalized quantile regressions. For the SCAD penalty, despite its good asymptotic properties, the corresponding optimization problem is nonconvex and, as a result, much harder to solve. In this work, we take advantage of the decomposition of the SCAD penalty function as the difference of two convex functions and propose to solve the corresponding optimization using the Difference Convex Algorithm (DCA).