Worst-case Quadratic Loss Bounds for On-line Prediction of Linear Functions by Gradient Descent

Worst-case Quadratic Loss Bounds for On-line Prediction of Linear Functions by Gradient Descent (1993)

Cached

Download Links

[ftp.cse.ucsc.edu]

by Nicolo Cesa-Bianchi , Philip M. Long , Manfred K. Warmuth

Venue:	IEEE Transactions on Neural Networks
Citations:	29 - 12 self

Versions

Version History

Metadata Version 3

User correction supplied by SystemCorrections

Datum	Value	Source
TITLE	Worst-case Quadratic Loss Bounds for On-line Prediction of Linear Functions by Gradient Descent	user correction - Legacy Corrections
AUTHOR NAME	Nicolo Cesa-Bianchi	user correction - Legacy Corrections
AUTHOR NAME	Philip M. Long	user correction - Legacy Corrections
AUTHOR AFFIL	y	user correction - Legacy Corrections
AUTHOR NAME	Manfred K. Warmuth	user correction - Legacy Corrections
AUTHOR AFFIL	; z; Board of Studies in Computer and Information Sciences; University of California, Santa Cruz; Computer Science Department, Duke University; Computer Science Department, UC Santa Cruz	user correction - Legacy Corrections
AUTHOR ADDR	; Santa Cruz, CA 95064; DSI, Universit`a di Milano, Via Comelico 39, 20135 Milano (ITALY).; P.O. Box 90129, Durham, NC 27708 USA.; Santa Cruz, CA 95064 USA.	user correction - Legacy Corrections
ABSTRACT	this paper we study the performance of gradient descent when applied to the problem of on-line linear prediction in arbitrary inner product spaces. We show worst-case bounds on the sum of the squared prediction errors under various assumptions concerning the amount of a priori information about the sequence to predict. The algorithms we use are variants and extensions of on-line gradient descent. Whereas our algorithms always predict using linear functions as hypotheses, none of our results requires the data to be linearly related. In fact, the bounds proved on the total prediction loss are typically expressed as a function of the total loss of the best fixed linear predictor with bounded norm. All the upper bounds are tight to within constants. Matching lower bounds are provided in some cases. Finally, we apply our results to the problem of on-line prediction for classes of smooth functions. Keywords: prediction, Widrow-Hoff algorithm, gradient descent, smoothing, inner product spaces, computational learning theory, on-line learning, linear systems.	user correction - Legacy Corrections
YEAR	1993	user correction - Legacy Corrections
VENUE	IEEE Transactions on Neural Networks	INFERENCE
VENUE TYPE	JOURNAL	INFERENCE
PAGES	604--619	INFERENCE
VOLUME	7	INFERENCE
CITATIONS	13 found	ParsCit 1.0