In network (e.g., Web) servers, it is often desirable to isolate the performance of different classes of requests from each other. That is, one seeks to achieve that a certain minimal proportion of server resources are available for a class of requests, independent of the load imposed by other requests. Recent work demonstrates how to achieve this performance isolation in servers consisting of a single, centralized node; however, achieving performance isolation in a distributed, cluster based server remains a problem. This paper introduces a new abstraction, the cluster reserve, which represents a resource principal in a cluster based network server. We present a design and evaluate a prototype implementation that extends existing techniques for performance isolation on a single node server to cluster based servers. In our design, the dynamic cluster-wide resource management problem is formulated as a constrained optimization problem, with the resource allocations on individual machin...

We present a scalable architecture for content-aware request distribution in Web server clusters. In this architecture, a level-4 switch acts as the point of contact for the server on the Internet and distributes the incoming requests to a number of back-end nodes. The switch does not perform any content-based distribution. This function is performed by each of the back-end nodes, which may forward the incoming request to another back-end based on the requested content. In terms of scalability, this architecture compares favorably to existing approaches where a front-end node performs content-based distribution. In our architecture, the expensive operations of TCP connection establishment and hando are distributed among the back-ends, rather than being centralized in the front-end node. Only a minimal additional latency penalty is paid for much improved scalability. We have implemented this new architecture, and we demonstrate its superior scalability by comparing it to a system tha...

This paper presents a set of techniques for providing fine-grained failover of long-running connections across a distributed collection of replica servers, and is especially useful for fault-tolerant and load-balanced delivery of streaming media and telephony sessions. Our system achieves connection-level failover across both local- and wide-area server replication, without requiring a frontend transport- or application-layer switch. Our approach uses recently proposed end-to-end "connection migration" mechanisms for transport protocols such as TCP, combined with a soft-state session synchronization protocol between replica servers. The end

Efficiency and portability are usually conflicting objectives for cluster-based network servers that distribute the clients ’ requests across the cluster based on the actual content requested. Our work is based on the observation that this efficiency vs. portability tradeoff has not been discussed before in the literature. To fill this gap, in this paper we study this tradeoff in the context of an interesting class of content-based network servers, the locality-conscious servers, using modeling and experimentation. Our analytical model gauges the potential performance benefits of portable and non-portable localityconscious request distribution with respect to a traditional, locality-oblivious server, as a function of multiple parameters. Based on our experience with the model, we design and evaluate a portable, locality-conscious server. Experiments with our server, a nonportable server, and a traditional server validate and confirm our modeling results under several real workloads. Based on our modeling and experimental results, our main conclusion is that portability should be promoted in cluster-based network servers with low processor overhead communication, given its relatively low cost 15%) in terms of efficiency. For clusters with high processor overhead communication, efficiency should be the overriding concern, as the cost of portability can be very high (as high as 98 % on 32 nodes). We also conclude that user-level communication can be useful even for non-scientific applications such as network servers.

Web proxies and Content Delivery Networks (CDNs) are widely used to accelerate Web content delivery and to conserve Internet bandwidth. These caching agents are highly effective for static content, which is an important component of all Web-based services. This paper

We evaluate the feasibility of using Migratory TCP (M-TCP), a reliable connection-oriented transport layer protocol that supports connection migration, for building highly available Internet services. M-TCP can transparently migrate the server endpoint of a live connection and assists server applications in resuming service on migrated connections. M-TCP provides a generic solution for the problem of service continuity and availability in the face of connectivity failures.

Clusters of commodity computers are currently being used to provide the scalability required by several popular Internet services. In this paper we evaluate an efficient cluster-based WWW server, as a function of the characteristicsof the intra-cluster communication architecture. More specifically, we evaluate the impact of processor overhead, networkbandwidth, remote memory writes, and zero-copy data transfers on the performance of our server. Our experimental results with an 8-node cluster and four real WWW traces show that networkbandwidth affects the performanceof our server by only 6%. In contrast, user-level communication can improve performance by as much as 29%. Low processor overhead, remote memory writes, and zero-copyall make small contributions towardsthis overall gain. Tobe able to extrapolate fromour experimental results, we usean analytical model to assess the performance of our server under different workload characteristics, different numbers of cluster nodes, and higher performance systems. Our modeling results show that higher gains (of up to 55%) can be accrued for workloads with large working sets and next-generation servers running on large clusters. 1

We describe BLT, a tool for extracting full HTTP level as well as TCP level traces via packet monitoring. This paper presents the software architecture that allows us to collect traces continuously, online, and at any point in the network. The software has been used to extract extensive traces within AT

In this paper we use analytic modeling and simulation to evaluate network servers implemented on clusters of workstations. More specifically, we model the potential benefits of locality-conscious request distribution within the cluster and evaluate the performance of a cluster-based server (called L2S) we designed in light of our experience with the model. Our most important modeling results show that locality-conscious distribution on a 16-node cluster can increase server throughput with respect to a locality-oblivious server by up to 5-fold, depending on the average size of the files requested and on the size of the server's working set. Our simulation results demonstrate that L2S achieves throughput that is within 28% of the full potential of locality-conscious distribution on 16 nodes, outperforming and significantly outscaling the best-known locality-conscious server. Based on our results and on the fact that the files serviced by network servers are becoming larger and more numer...

As the World Wide Web experiences increasing commercial and mission-critical use, server systems are expected to deliver high and predictable performance. The phenomenal improvement in microprocessor speeds, coupled with the deployment of clusters of commodity workstations has enabled server systems to meet the continually increasing performance demands in a cost-effective and scalable manner. However, as the volume, variety and sophistication of services oered by server systems increase, eective support for providing dierentiated and predictable quality of service has also become important. For example, it is often desirable to dierentiate between the resources allocated to virtual web sites hosted on a server system so as to provide predictable performance to individual sites, regardless of the load imposed upon others. Server systems lack adequate support for providing predictable performance to hosted services in terms of metrics that are meaningful to server applications, such...