Popular Web sites can neither rely on a single powerful server nor on independent mirroredservers to support the ever increasing request load. Scalability and availability can be provided by distributed Web-server architectures that schedule client requests among the multiple server nodes in a user-transparent way. In this paper we will review the state of the art in load balancing techniques on distributed Web-server systems. We will analyze the efficiency and limitations of the various approaches and their tradeoff.

It is becoming increasingly common to construct network services using redundant resources geographically distributed across the Internet. Content Distribution Networks are a prime example. Such systems distribute client requests to an appropriate server based on a variety of factors---e.g., server load, network proximity, cache locality---in an effort to reduce response time and increase the system capacity under load. This paper explores the design space of strategies employed to redirect requests, and defines a class of new algorithms that carefully balance load, locality, and proximity. We use large-scale detailed simulations to evaluate the various strategies. These simulations clearly demonstrate the effectiveness of our new algorithms, which yield a 60-91% improvement in system capacity when compared with the best published CDN technology, yet user-perceived response latency remains low and the system scales well with the number of servers.

Efficient server selection algorithms reduce retrieval time for objects replicated on different servers and are an important component of Internet cache architectures. This paper empirically evaluates six clientside server selection algorithms. The study compares two statistical algorithms, one using median bandwidth and the other median latency, a dynamic probe algorithm, two hybrid algorithms, and random selection. The server pool includes a topologically dispersed set of United States state government web servers. Experiments were run on three clients in different cities and on different regional networks. The study examines the effects of time-of-day, client resources, and server proximity. Differences in performance highlight the degree of algorithm adaptability and the effect that network upgrades can have on statistical estimators. Dynamic network probing performs as well or better than the statistical bandwidth algorithm and the two probe-bandwidth hybrid algorithms. The statis...

A distributed multi-server Web site can provide the scalability necessary to keep up with growing client demand at popular sites. Load balancing of these distributed Web-server systems, consisting of multiple Web servers for document retrieval and a Domain name server (DNS) for address resolution, opens interesting new problems. In this paper, we investigate the effects of using a more active DNS which, as an atypical centralized scheduler, applies some scheduling strategy in routing the requests to the most suitable Web server. Unlike traditional parallel/distributed systems in which a centralized scheduler has full control of the system, the DNS controls only a very small fraction of the requests reaching the multi-server Web site. This peculiarity, especially in the presence of highly skewed load, makes it very difficult to achieve acceptable load balancing and avoid overloading some Web server. This paper adapts traditional scheduling algorithms to the DNS, proposes new policies, a...

In this technical report we propose a new method for improving the average response time of Web servers by cooperatively caching the results of requests for dynamic content. The work is motivated by our recent study of access logs from the Alexandria Digital Library server at UCSB, which demonstrates that approximately a 30 percent decrease in average response time could be achieved by caching dynamically generated content. We have developed a distributed Web server called Swala, in which the nodes cooperatively cache the results of CGI requests. We use a two-level cache table consistency protocol and a replicated global cache directory to maximize the system performance and minimize overhead in responding to dynamic Web requests. Our experiments show that the single-node performance of Swala without caching is comparable to the Netscape Enterprise server, that considerable speedups are obtained using caching, and that the cache hit rate is substantially higher with cooperative cache t...

With ever increasing Web traffic, a distributed multi-server Web site can provide scalability and flexibility to cope with growing client demands. Load balancing algorithms to spread the requests across multiple Web servers are crucial to achieve the scalability. Various domain name server (DNS) based schedulers have been proposed in the literature, mainly for multiple homogeneous servers. The presence of heterogeneous Web servers not only increases the complexity of the DNS scheduling problem, but also makes previously proposed algorithms for homogeneous distributed systems not directly applicable. This leads us to propose new policies, called adaptive TTL algorithms, that take into account of both the uneven distribution of client request rates and heterogeneity of Web servers to adaptively set the time-to-live (TTL) value for each address mapping request. Extensive simulation results show that these strategies are robust and effective in balancing load among geographically distribut...

In this paper, we investigate the issues involved in developing a scalable World Wide Web (WWW) server called SWEB on a cluster of workstations. The objective is to strengthen the processing capabilities of such a server in order to match huge demands in simultaneous access requests from the Internet, especially when these requests involve delivery of large digitized documents. The scheduling component of the system actively monitors the usages of CPU, disk I/O channels and the interconnection network to effectively distribute HTTP requests across processing units to exploit task and I/O parallelism. We analyze the maximum number of requests that can be handled by the system and present several experiments to examine the performance of this system. 1 Motivation The Scalable Web server (SWEB) project grew out of the needs of the Alexandria Digital Library (ADL) project at UCSB [A96]. Digital library systems, which provide the on-line retrieval and processing of digitized documents thro...

This paper considers policies for distributing requests in clustered Web servers, wherein multiple server machines are configured to function as a single high(er) performance Web server. We evaluate various load distribution policies with respect to both their ability to achieve good load balance (the primary goal) and also to their impact on the effectiveness of per-machine caching. Trace-driven simulation is employed, with workload traces from two heavily-loaded (3-8 million requests per day) commercial Web servers. Our results show that use of current state information is necessary in achieving good load balance only when the achievable per-request bandwidth is not strongly network or client limited. Use of current state information is not found to be necessary with respect to achieving good cache behaviour. Load distribution based on a static hashed assignment of the URL space is found to yield very similar cache performance to load distribution based on current cache contents. We also find that it is possible to achieve both good cache behaviour and good load balance, but it requires use of policies that take both objectives into consideration and that make use of information concerning current server loads.

Replication of information across a server cluster provides a promising way to support popular Web sites. However, a Web-server cluster requires some mechanism for the scheduling of requests to the most available server. One common approach is to use the local Domain Name Server (DNS) as a centralized dispatcher. The main problem is that WWW address caching mechanisms (although reducing network trac) only let this DNS dispatcher control a very small fraction of the requests reaching the Web-server cluster. The nonuniformity of the load from dierent client domains, and the high variability of real Web workload introduce additional degrees of complexity to the load balancing issue. These characteristics make existing scheduling algorithms for traditional distributed systems not applicable to control the load of Web-server clusters and motivate the research on entirely new DNS policies that require some system state information. We analyze various DNS dispatching policies under realistic situations where state information needs to be estimated with low computation and communication overhead so as to be applicable to a Web cluster architecture. In a model of realistic scenarios for the Web cluster, a large set of simulation experiments shows that, by incorporating the proposed state estimators into the dispatching policies, the eectiveness of the DNS scheduling algorithms can improve substantially, in particular if compared to the results of DNS algorithms not using adequate state information.

Most of the techniques used for increasing the availability of web services do not provide fault tolerance for requests being processed at the time of server failure. Other schemes require deterministic servers or changes to the web client. These limitations are unacceptable for many current and future applications of the Web. We have developed an efficient implementation of a client-transparent mechanism for providing fault-tolerant web service that does not have the limitations mentioned above. The scheme is based on a hot standby backup server that maintains logs of requests and replies. The implementation includes modifications to the Linux kernel and to the Apache web server, using their respective module mechanisms. We describe the implementation and present an evaluation of the impact of the backup scheme in terms of throughput, latency, and CPU processing cycles overhead.