Results 1 
3 of
3
AdaBoost and Forward Stagewise Regression are FirstOrder Convex Optimization Methods
, 2013
"... Boosting methods are highly popular and effective supervised learning methods which combine weak learners into a single accurate model with good statistical performance. In this paper, we analyze two wellknown boosting methods, AdaBoost and Incremental Forward Stagewise Regression (FSε), by establi ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Boosting methods are highly popular and effective supervised learning methods which combine weak learners into a single accurate model with good statistical performance. In this paper, we analyze two wellknown boosting methods, AdaBoost and Incremental Forward Stagewise Regression (FSε), by establishing their precise connections to the Mirror Descent algorithm, which is a firstorder method in convex optimization. As a consequence of these connections we obtain novel computational guarantees for these boosting methods. In particular, we characterize convergence bounds of AdaBoost, related to both the margin and logexponential loss function, for any stepsize sequence. Furthermore, this paper presents, for the first time, precise computational complexity results for FSε. 1
3 AdaBoost and Forward Stagewise Regression are FirstOrder Convex Optimization Methods
, 2013
"... ar ..."
Sparse Random Features Algorithm as Coordinate Descent in Hilbert Space
"... In this paper, we propose a Sparse Random Features algorithm, which learns a sparse nonlinear predictor by minimizing an ℓ1regularized objective function over the Hilbert Space induced from a kernel function. By interpreting the algorithm as Randomized Coordinate Descent in an infinitedimensiona ..."
Abstract
 Add to MetaCart
(Show Context)
In this paper, we propose a Sparse Random Features algorithm, which learns a sparse nonlinear predictor by minimizing an ℓ1regularized objective function over the Hilbert Space induced from a kernel function. By interpreting the algorithm as Randomized Coordinate Descent in an infinitedimensional space, we show the proposed approach converges to a solution within ϵprecision of that using an exact kernel method, by drawingO(1/ϵ) random features, in contrast to the O(1/ϵ2) convergence achieved by current MonteCarlo analyses of Random Features. In our experiments, the Sparse Random Feature algorithm obtains a sparse solution that requires less memory and prediction time, while maintaining comparable performance on regression and classification tasks. Moreover, as an approximate solver for the infinitedimensional ℓ1regularized problem, the randomized approach also enjoys better convergence guarantees than a Boosting approach in the setting where the greedy Boosting step cannot be performed exactly. 1