Composite Objective Mirror Descent

Composite Objective Mirror Descent

Cached

Download Links

by John Duchi , Shai Shalev-shwartz , Yoram Singer , Ambuj Tewari

Citations:

BibTeX

@MISC{Duchi_compositeobjective,
    author = {John Duchi and Shai Shalev-shwartz and Yoram Singer and Ambuj Tewari},
    title = {Composite Objective Mirror Descent},
    year = {}
}

Bookmark

OpenURL

Abstract

We present a new method for regularized convex optimization and analyze it under both online and stochastic optimization settings. In addition to unifying previously known firstorder algorithms, such as the projected gradient method, mirror descent, and forwardbackward splitting, our method yields new analysis and algorithms. We also derive specific instantiations of our method for commonly used regularization functions, such as ℓ1, mixed norm, and trace-norm. 1

Citations

3200	Topics in Matrix Analysis - Horn, Johnson - 1994
2333	Convex Analysis - Rockafellar - 1970
2134	Convex Optimization - Boyd, Vandenberghe - 2004
556	The weighted majority algorithm - Littlestone, Warmuth - 1994
236	An iterative thresholding algorithm for linear inverse problems with a sparsity constraint - Daubechies, Defrise, et al. - 2004
189	Weighted sums of certain dependent random variables - Azuma - 1967
100	Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization - Recht, Fazel, et al. - 2008
91	Problem Complexity and Method Efficiency in Optimization - Nemirovski, Yudin - 1983
90	T.S.: Maximum-margin matrix factorization - Srebro, Rennie, et al. - 2005
83	On the generalization ability of on-line learning algorithms - Cesa-Bianchi, Conconi, et al. - 2004
73	Splitting algorithms for the sum of two nonlinear operators - Lions, Mercier - 1979
73	Sparse reconstruction by separable approximation - Wright, Nowak, et al. - 2009
71	Gradient methods for minimizing composite objective function - Nesterov - 2007
65	The tradeoffs of large scale learning - Bottou, Bousquet - 2008
65	Logarithmic regret algorithms for online convex optimization - Hazan, Agarwal, et al. - 2006
64	From online to batch learning - Littlestone - 1989
52	Fixed point and Bregman iterative methods for matrix rank minimization - Ma, Goldfarb, et al. - 2009
48	The robustness of the p-norm algorithms - Gentile - 2003
41	M.: Mirror descent and nonlinear projected subgradient methods for convex optimization - Beck, Teboulle
40	Primal-dual subgradient methods for convex problems - Nesterov - 2009
36	Sharp uniform convexity and smoothness inequalities for trace norms - Ball, Carlen, et al. - 1994
36	Efficient projections onto the ℓ1-ball for learning in high dimensions - Duchi, Shalev-Shwartz, et al. - 2008
31	SVM optimization: Inverse dependence on training set size - Shalev-Shwartz, Srebro - 2008
29	Dual averaging methods for regularized stochastic learning and online optimization - Xiao
26	Sparse online learning via truncated gradient - Langford, Li, et al.
18	Matrix Analysis. Cambridge University Press, Cambridge. Reduction of Mutation Rates - Horn, Johnson - 1985
14	Joint covariate selection for grouped classification - Obozinski, Taskar, et al. - 2007
13	The convex analysis of unitarily invariant matrix functions - Lewis - 1995
11	Online and Batch Learning using Forward-Backward Splitting - Duchi, Singer
10	Stochastic methods for ℓ 1 -regularized loss minimization - Shalev-Shwartz, Tewari
9	On the generalization ability of online strongly convex programming algorithms - Kakade, Tewari - 2009
2	Approximation accuracy, gradient methods, and error bound for structured convex optimization - Tseng - 2010