This paper presents a new persistent data management layer designed to simplify cluster-based Internet service construction. This self-managing layer, called a distributed data structure (DDS), presents a conventional single-site data structure interface to service authors, but partitions and replicates the data across a cluster. We have designed and implemented a distributed hash table DDS that has properties necessary for Internet services (incremental scaling of throughput and data capacity, fault tolerance and high availability, high concurrency, consistency, and durability). The hash table uses two-phase commits to present a coherent view of its data across all cluster nodes, allowing any node to service any task. We show that the distributed hash table simplies Internet service construction by decoupling service-specic logic from the complexities of persistent, consistent state management, and by allowing services to inherit the necessary service properties from the DDS rather ...

We describe the design, implementation and performance of a Web server accelerator which runs on an embedded operating system and improves Web server performance by caching data. The accelerator resides in front of one or more Web servers. Our accelerator can serve up to 5000 pages/second from its cache on a 200 MHz PowerPC 604. This throughput is an order of magnitude higher than that which would be achieved by a high-performance Web server running on similar hardware under a conventional operating system such as Unix or NT. The superior performance of our system results in part from its highly optimized communications stack. In order to maximize hit rates and maintain updated caches, our accelerator provides an API which allows application programs to explicitly add, delete, and update cached data. The API allows our accelerator to cache dynamic as well as static data. We analyze the SPECweb96 benchmark, and show that the accelerator can provide high hit ratios and excellent performa...

Service composition enables flexible creation of new services by assembling independent service components. We are focused on the scenario where such composition takes place across the wide-area Internet. We envision independent providers deploying and managing service instances and portal providers composing them to quickly enable new applications in nextgeneration networks.

Future scalable, high throughput, and high performance applications are likely to execute on platforms constructed by clustering multiple autonomous distributed servers, with resource access governed by agreements between the owners and users of these servers. As an example, application service providers (ASPs) can pool their resources together according to pre-specified sharing agreements to provide better services to their customers. Such systems raise several new resource management challenges, chief amongst which is the enforcement of agreements to ensure that, despite the distributed nature of both requests and resources, user requests only receive a predetermined share of the aggregate resource and that the resources of a participant are not misused. Current solutions only enforce such agreements at a coarse granularity and in a centralized fashion, limiting their applicability for general workloads. This paper presents an architecture for the distributed enforcement of resource sharing agreements. Our approach exploits a uniform application-independent representation of agreements, and combines it with efficient time-window based coordinated queuing algorithms running on multiple nodes. We have successfully implemented this general strategy in two different network layers: a layer-7 HTTP redirector and a layer-4 packet redirector, which redirect connection requests from distributed clients to a cluster of distributed servers. Our measurements of both implementations verify that our approach is general and effective: different client groups receive service commensurate with their agreements. 1 1

This paper focuses on an experimental analysis of the performance and scalability of cluster-based web servers. We carry out the comparative studies using two experimental platforms, namely, a hardware testbed consisting of 16 PCs, and a trace-driven discrete-event simulator. Dispatcher and web server service times used in the simulator are determined by carrying out a set of experiments on the testbed. The simulator is validated against stochastic queuing models and the testbed. Experiments on the testbed are limited by the hardware configuration, but our complementary approach allows us to carry out scalability study on the validated simulator. The three dispatcherbased scheduling algorithms analyzed are: round robin scheduling, least connected based scheduling, and least loaded based scheduling. The least loaded algorithm is used as the baseline (upper performance bound) in our analysis and the performance metrics include average waiting time, average response time, and average web server utilization. A synthetic trace generated by the workload generator called SURGE, and a public-domain France Football World Cup 1998 trace are used. We observe that the round robin algorithm performs much worse in comparison with the other two algorithms for low to medium workload. However, as the request arrival rate increases the performance of the three algorithms converge with the least connected algorithm approaching the baseline algorithm as at a much faster rate than the round robin. The least connected algorithm performs well for medium to high workload. At very low load the average waiting time is two to six times higher than the baseline algorithm but the absolute value between these two waiting times is very small. 1.

This paper presents the design of a novel system architecture, Virtual Service Grid (VSG), for delivering high performance network services. The VSG is based on the concept of the Virtual Service which provides location, replication, and fault transparency to clients accessing remotely deployed high-end services. One of the novel features of the Virtual Service is the ability to self-scale in response to client demand. The VSG exploits network- and serviceinformation to make adaptive dynamic replica selection, creation, and deletion decisions. We describe the VSG architecture, middleware, and replica management policies. We have deployed the VSG on a wide-area Internet testbed to evaluate its performance. The results indicate that the VSG can deliver efficient performance for a wide-range of client workloads, both in terms of reduced response time, and in utilization of system resources. A large array of wide-area application technologies for distributed high-performance computing including scientific problem-solving environments [7][8][18][19][23], computational Grids [12][14], and peer-to-peer environments [9][16] are emerging. These technologies are beginning to support a model in which diverse services are deployed in the network and made available to users and applications. In this paper, we focus on “high-end ” services,

We describe the design, implementation and performance of a high-performance Web server accelerator which runs on an embedded operating system and improves Web server performance by caching data. It can serve Web data at rates an order of magnitude higher than that which would be achieved by a high-performance Web server running on similar hardware under a conventional operating system such as Unix or NT. The superior performance of our system results in part from its highly optimized communications stack. In order to maximize hit rates and maintain updated caches, our accelerator provides an API which allows application programs to explicitly add, delete, and update cached data. The API allows our accelerator to cache dynamic as well as static data. We describe how our accelerator can be scaled to multiple processors to increase performance and availability. The basic design alternatives include a content router or a TCP router (without content routing) in front of a set of Web cache accelerator nodes, with the cache memory distributed across the accelerator nodes. Content-based routing reduces cache node CPU cycles but can make the front-end router a bottleneck. With the TCP router, a request for a cached object may initially be sent to the wrong cache node; this results in larger cache node CPU cycles, but can provide a higher aggregate throughput, because the TCP router becomes a bottleneck at a higher throughput than the content router. We quantify the throughput ranges in which different designs are preferable. We also examine a combination of content-based and TCP routing techniques. In addition, we present statistics from critical deployments of our accelerator for improving performance at highly accessed Sporting and Event Web sites hosted by IBM. 2002 Elsevier...

As a result of current resource provisioning schemes in Internet services, servers end up less than 50 % utilized almost all the time. At this level of utilization, the servers ’ energy efficiency is less than half their efficiency at peak utilization. A solution to this problem could be consolidating workloads into fewer servers and turning others off. However, services typically resist doing so. A major reason is the fear of slow response times during re-activation in handling traffic spikes. Another reason is that services want to maximize the amount of main memory available for data caching across the server cluster. In this paper, we propose an approach that does not completely shutdown idle servers and allows free memory space to be used for cooperative data caching. Specifically, we make two key contributions. First, we propose to send servers to a new “barely-alive ” power state, instead of turning them off after consolidation. Our barely-alive servers allow remote accesses to their main memories even when all processing cores have been turned off. Second, we design a distributed middleware that accommodates barely-alive servers and is capable of dynamically re-sizing the amount of cache space across the cluster to the minimum required to respect the service’s service-level agreement (SLA). Any memory that is not in use by the middleware can be used by applications. Our trace-driven simulations of a server cluster using our middleware and barely-alive servers show very encouraging results. 1.

Web services on the Internet are increasingly relying on personalized content that is dynamically generated by server application code and customized for users. Delivering such personalized content increases computational load on servers and does not fit into the current Internet web caching model leading to an increase in user latency and bandwidth consumption. In this paper, we propose DynCoDe, a novel architecture for efficient delivery of personalized web services that integrates the distribution, caching and generation of personalized content. In the DynCoDe architecture, resource intensive processes for content generation and reusable content components are pushed to the network edges increasing server scalability and content availability while reducing user latency and backbone Internet traffic. We evaluate DynCoDe under real world network conditions and show significant improvements in bandwidth consumption, user latency and server scalability.