Abstract:
This paper evaluates linear models for predicting the Digital Unix five-second host load average from 1 to 30 seconds into the future. A detailed statistical study of a large number of long, fine grain load traces from a variety of real machines leads to consideration of the Box-Jenkins models (AR, MA, ARMA, ARIMA), and the ARFIMA models (due to self-similarity.) We also consider a simple windowed-mean model. The computational requirements of these models span a wide range, making some more practical than others for incorporation into an online prediction system. We rigorously evaluate the predictive power of the models by running a large number of randomized testcases on the load traces and then data-mining their results. The main conclusions are that load is consistently predictable to a very useful degree, and that the simple, practical models such as AR are sufficient for host load prediction. We recommend AR(16) models or better for host load prediction. We implement an online host load prediction system around the AR(16) model and evaluate its overhead, finding that it uses miniscule amounts of CPU time and network bandwidth
Citations
|
686
|
The Art of Computer System Performance Analysis
– Jain
- 1991
|
|
343
|
Adaptive load sharing in homogeneous distributed systems
– Eager, Lazowska, et al.
- 1986
|
|
272
|
Architectural Support for Quality of Service for CORBA Objects,” Theory and Practice of Object Systems
– Zinky, Bakken, et al.
- 1997
|
|
255
|
Exploiting process lifetime distributions for load balancing
– Harchol-Balter, Downey
- 1997
|
|
194
|
Forecasting network performance to support dynamic scheduling using the network weather service
– Wolski
- 1997
|
|
163
|
The Common Object Request Broker Architecture: Core Specification, Revision 3.0
– Group
- 2002
|
|
154
|
Introduction to Time Series and Forecasting
– Brockwell, Davis
- 1996
|
|
124
|
An introduction to long-memory time series models and fractional differencing
– Granger, Joyeux
- 1980
|
|
115
|
Load-balancing heuristics and process behavior
– Leland, Ott
- 1986
|
|
95
|
Java Remote Method Invocation Specification
– Microsystems
- 1997
|
|
94
|
The Limited Performance Benefits of Migrating Active Processes for Load Sharing
– Eager, Lazowska, et al.
- 1988
|
|
86
|
Scheduling from the perspective of the application
– BERMAN, WOLSKI
- 1996
|
|
86
|
The Available Capacity of a Privately Owned Workstation Environment
– Mutka, Livny
- 1991
|
|
77
|
Statistical methods for data with long-range dependence
– Beran
- 1992
|
|
67
|
Space-time modelling with long-memory dependence: assessing Ireland’s wind power resource
– Haslett, Raftery
- 1989
|
|
65
|
Predicting the CPU availability of time-shared unix systems on the computational grid
– Wolski, Spring, et al.
|
|
43
|
An Extensible Toolkit for Resource Prediction in Distributed Systems
– Dinda, O’Hallaron
- 1999
|
|
40
|
O'Hallaron. An evaluation of linear models for host load prediction.In
– Dinda, R
- 1999
|
|
39
|
Time Series Models for Internet Traffic
– Basu, Mukherjee
- 1999
|
|
34
|
The performance of a service for network-aware applications
– Obraczka, Georghiu
- 1998
|
|
33
|
Preliminary report on the design of a framework for distributed visualization
– Aeschlimann, Dinda, et al.
- 1999
|
|
30
|
A time series model of long-term NSFNET backbone traffic
– Groschwitz, Polyzos
- 1994
|
|
26
|
The statistical properties of host load. Scientific Programming, 7(3,4
– DINDA
- 1999
|
|
23
|
A resource monitoring system for network-aware applications
– LOWEKAMP, MILLER, et al.
- 1998
|
|
16
|
The case for prediction-based best-effort real-time systems
– Dinda, Lowekamp, et al.
- 1999
|
|
15
|
On distributed scheduling using load prediction from past information. Abstracts published
– SAMADANI, KALTHOFEN
- 1996
|
|
11
|
Resource Signal Prediction and Its Application to Real-time Scheduling Advisors
– Dinda
- 2000
|
|
8
|
Fractal structures and processes
– BASSINGTHWAIGHTE, BEARD, et al.
- 1995
|
|
7
|
Fracdiff: Maximum likelihood estimation of the parameters of a fractionally differenced ARIMA(p; d; q) model. Computer Program
– FRALEY
- 1991
|
|
4
|
Fractional differencing, Biometrika 68
– HOSKING
- 1981
|
