As Internet services become more popular and pervasive, a critical problem that arises is managing the performance of services under extreme overload. This paper presents a set of techniques for managing overload in complex, dynamic Internet services. These techniques are based on an adaptive admission control mechanism that attempts to bound the 90th-percentile response time of requests flowing through the service. This is accomplished by internally monitoring the performance of the service, which is decomposed into a set of event-driven stages connected with request queues. By controlling the rate at which each stage admits requests, the service can perform focused overload management, for example, by filtering only those requests that lead to resource bottlenecks. We present two extensions of this basic controller that provide class-based service differentiation as well as application-specific service degradation. We evaluate these mechanisms using a complex Webbased e-mail service that is subjected to a realistic user load, as well as a simpler Web server benchmark.

This paper presents a method for admission control and request scheduling for multiply-tiered e-commerce Web sites, achieving both stable behavior during overload and improved response times. Our method externally observes execution costs of requests online, distinguishing different request types, and performs overload protection and preferential scheduling using relatively simple measurements and a straightforward control mechanism. Unlike previous proposals, which require extensive changes to the server or operating system, our method requires no modifications to the host O.S., Web server, application server or database. Since our method is external, it can be implemented in a proxy. We present such an implementation, called Gatekeeper, using it with standard software components on the Linux operating system. We evaluate the proxy using the industry standard TPC-W workload generator in a typical three-tiered e-commerce environment. We show consistent performance during overload and throughput increases of up to 10 percent. Response time improves by up to a factor of 14, with only a 15 percent penalty to large jobs.

Large data centers host several application environments (AEs) that are subject to workloads whose intensity varies widely and unpredictably. Therefore, the servers of the data center may need to be dynamically redeployed among the various AEs in order to optimize some global utility function. Previous approaches to solving this problem suffer from scalability limitations and cannot easily address the fact that there may be multiple classes of workloads executing on the same AE. This paper presents a solution that addresses these limitations. This solution is based on the use of analytic queuing network models combined with combinatorial search techniques. The paper demonstrates the effectiveness of the approach through simulation experiments. Both online and batch workloads are considered. 1.

Controlling the timing performance of a network server is a challenging problem. This paper presents a Queueing Model Based Feedback Control approach to keep the timing performance of a network server close to the service level specification. We show that in an instrumented Apache server, combining feedback control with a queueing model leads to better tracking of QoS specifications than with feedback control alone or queueing model based feed forward control alone. Network based server systems, e.g., Web servers, have now become an integral part of our information services infrastructure. Controlling the timing performance of each individual connection to a network server is a challenging

Abstract: We present an architecture and prototype implementation of a performance management system for cluster-based web services. The system supports multiple classes of web services traffic and allocates server resources dynamically so to maximize the expected value of a given cluster utility function in the face of fluctuating loads. The cluster utility is a function of the performance delivered to the various classes, and this leads to differentiated service. In this paper we will use the average response time as the performance metric. The management system is transparent: it requires no changes in the client code, the server code, or the network interface between them. The system performs three performance management tasks: resource allocation, load balancing, and server overload protection. We use two nested levels of management mechanism. The inner level centers on queuing and scheduling of request messages. The outer level is a feedback control loop that periodically adjusts the scheduling weights and server allocations of the inner level. The feedback controller is based on an approximate first-principles model of the system, with parameters derived from continuous monitoring. We focus on SOAP-based web services. We report experimental results that show the dynamic behavior of the system. 1.

The high cost of operating large computing installations has motivated a broad interest in reducing the need for human intervention by making systems self-managing. This paper explores the extent to which control theory can provide an architectural and analytic foundation for building self-managing systems, either from new components or layering on top of existing components. Further, we propose a deployable testbed for autonomic computing (DTAC) that we believe will reduce the barriers to addressing key research problems in autonomic computing. The initial DTAC architecture is described along with several problems that itcanbeusedtoinvestigate. 1

this paper, we describe an approach in which analytic performance models are combined with combinatorial search techniques to design controllers that run periodically (e.g., every few minutes) to determine the best possible configuration for the system given its workload. We first illustrate and motivate the ideas using a simulated multithreaded server. Then, we provide experimental results, obtained by using the techniques described here, to an actual Web server subject to a workload generated by SURGE

With the increasB us of "Virtual Machines (VMs as vehicles thatist.O1 applications running on the se. hos: it is neces sce to devis techniques that enable multipleVMs tos hare underlying resP---P.B both fairly and e#ciently.To that end, one common approach is to deploy complexresex.0 management techniques in the hos0PO infras.B11---5P.ss55.s0P , inthis paper, we advocate the us ofs5O1:---.B1:---0:. in theVMs themsem es bas- on feedback about resPR:P us05 and availability. ConsRR0. tly, we define a "Friendly" VM (FVM) to be a virtual machine that adjus--- its demand forsr.05 res5101.B s o that they are both e#ciently and fairly allocated to competing FVMs.0[ h properties areens222 us5[ one of many provably convergent control rules s uch as AIMD.By adoptingthis dis tributed application-bas--- approach to res50P2 management, itis not necesR.B to makeasP0:.B:2--- about the underlying resderly nor about the requirements of FVMs competing for thes resR---.B::--- odemonsB::--- the elegance ands implicity of our approach, wepres[ t a prototype implementation of our FVM framework inUs50O[ de Linux (UML)---an implementation thatcons[0O ofles than 500lines of code changes to UML.Wepres5 t an analytic, control-theoretic model of FVM adaptation, which es------:.B0R5 convergence andfairnes propertiesRR2. B properties areals backed up with experimental res0[R usR0 our prototype FVM implementation. 1.

The overwhelming success of the Web as a mechanism for facilitating information retrieval and for conducting business transactions has led to an increase in the deployment of complex enterprise applications. These applications typically run on Web Application Servers, which assume the burden of managing many tasks, such as concurrency, memory management, database access, etc., required by these applications. The performance of an Application Server depends heavily on appropriate configuration. Configuration is a difficult and error-prone task due to the large number of configuration parameters and complex interactions between them. We formulate the problem of finding an optimal configuration for a given application as a black-box optimization problem. We propose a Smart Hill-Climbing algorithm using ideas of importance sampling and Latin Hypercube Sampling (LHS). The algorithm is efficient in both searching and random sampling. It consists of estimating a local function, and then, hill-climbing in the steepest descent direction. The algorithm also learns from past searches and restarts in a smart and selective fashion using the idea of importance sampling. We have carried out extensive experiments with an online brokerage application running in a WebSphere environment. Empirical results demonstrate that our algorithm is more efficient than and superior to traditional heuristic methods. Categories and Subject Descriptors

Abstract. Properly optimizing the setting of configuration parameters can greatly improve performance, especially in the presence of changing workloads. This paper explores approaches to online optimization of the Apache web server, focusing on the MaxClients parameter (which controls the maximum number of workers). Using both empirical and analytic techniques, we show that MaxClients has a concave upward effect on response time and hence hill climbing techniques can be used to find the optimal value of MaxClients. We investigate two optimizers that employ hill climbing—one based on Newton’s Method and the second based on fuzzy control. A third technique is a heuristic that exploits relationships between bottleneck utilizations and response time minimization. In all cases, online optimization reduces response times by a factor of 10 or more compared to using a static, default value. The trade-offs between the online schemes are as follows. Newton’s method is well known but does not produce consistent results for highly variable data such as response times. Fuzzy control is more robust, but converges slowly. The heuristic works well in our prototype system, but it may be difficult to generalize because it requires knowledge of bottleneck resources and an ability to measure their utilizations. 1