We study several transparent techniques for scaling dynamic content web sites, and we evaluate their relative impact when used in combination. Full transparency implies strong data consistency as perceived by the user, no modifications to existing dynamic content site tiers and no additional programming effort from the user or site administrator upon deployment.

This paper introduces a self-configuring architecture for scaling the database tier of dynamic content web servers. We use a unified approach to load and fault management based on dynamic data replication and feedback-based scheduling. While replication provides scaling and high availability, feedback scheduling dynamically allocates tasks to commodity databases across workloads in response to peak loads or failure conditions thus providing quality of service. By augmenting the feedback loop with state awareness, we avoid oscillations in resource allocation.

The database tier of dynamic content servers at large Internet sites is typically hosted on centralized and expensive hardware. Recently, research prototypes have proposed using database replication on commodity clusters as a more economical scaling solution. In this paper, we propose using database replication to support multiple applications on a shared cluster. Our system dynamically allocates replicas to applications in order to maintain application-level performance in response to either peak loads or failure conditions. This approach allows unifying load and fault management functionality. The main challenge in the design of our system is the time taken to add database replicas. We present replica allocation policies that take this time delay into account and also design an efficient replica addition method that has minimal impact on other applications. We evaluate our dynamic replication system on a commodity cluster with two standard benchmarks: the TPC-W e-commerce benchmark and the RUBIS auction benchmark. Our evaluation shows that dynamic replication requires fewer resources than static partitioning or full overlap replication policies and provides over 90 % latency compliance to each application under a range of load and failure scenarios.

Middleware-based database replication protocols require few or no changes in the database engine. Thus, they are more portable and flexible than kernel-based protocols, but have coarser-grain information about transaction access data, resulting in reduced concurrency and increased aborts. This paper proposes conflict-aware load-balancing techniques to increase the concurrency and reduce the abort rate of middleware-based replication protocols. Our algorithms assign transactions to replicas so that the number of conflicting transactions executing on distinct servers is reduced and the processing load is equitably distributed over the servers. Experimental evaluation using a prototype of our system running the TPC-C benchmark showed that aborts can be reduced with no penalty in response time.

Abstract. We consider the use of a database cluster for Application Service Provider (ASP). In the ASP context, applications and databases can be update-intensive and must remain autonomous. In this paper, we describe the Leganet system which performs freshness-aware transaction routing in a database cluster. We use multi-master replication and relaxed replica freshness to increase load balancing. Our transaction routing takes into account freshness requirements of queries at the relation level and uses a cost function that takes into account the cluster load and the cost to refresh replicas to the required level. We implemented the Leganet prototype on a 11-node Linux cluster running Oracle8i. Using experimentation and emulation up to 128 nodes, our validation based on the TPC-C benchmark demonstrates the performance benefits of our approach.

The need for high availability and performance in data management systems has been fueling a long running interest in database replication from both academia and industry. However, academic groups often attack replication problems in isolation, overlooking the need for completeness in their solutions, while commercial teams take a holistic approach that often misses opportunities for fundamental innovation. This has created over time a gap between academic research and industrial practice. This paper aims to characterize the gap along three axes: performance, availability, and administration. We build on our own experience developing and deploying replication systems in commercial and academic settings, as well as on a large body of prior related work. We sift through representative examples from the last decade of open-source, academic, and commercial database replication systems and combine this material with case studies from real systems deployed at Fortune 500 customers. We propose two agendas, one for academic research and one for industrial R&D, which we believe can bridge the gap within 5-10 years. This way, we hope to both motivate and help researchers in making the theory and practice of middleware-based database replication more relevant to each other.

Middleware-based database replication approaches have emerged in the last few years as an alternative to traditional database replication implemented within the database kernel. A middleware approach enables third party vendors to provide high availability solutions, a growing practice nowadays in the software industry. However, middleware solutions often lack scalability and exhibit a number of consistency and performance issues. The reason is that in most cases the middleware has to handle the database as a black box, and hence, cannot take advantage of the many optimizations implemented in the database kernel. Thus, middleware solutions often reimplement key functionality but cannot achieve the same efficiency as a kernel implementation. Reflection has been proposed during the last decade as a fruitful paradigm to separate non-functional aspects from functional ones, simplifying software development and maintenance whilst fostering reuse. However, fully reflective databases are not feasible due to the high cost of reflection. Our claim is that by exposing some minimal database functionality through a lightweight reflective interface, efficient and scalable middleware database replication can be attained. In this paper we explore a wide variety of such lightweight reflective interfaces and discuss what kind of replication algorithms they enable. We also discuss implementation alternatives for some of these interfaces and evaluate their performance.

This paper introduces a self-configuring architecture for on-demand resource allocation to applications in a shared database cluster. We use a unified approach to load and fault management based on data replication and reactive replica provisioning. While data replication provides scaling and high availability, reactive provisioning dynamically allocates additional replicas to applications in response to peak loads or failure conditions, thus providing per application performance. We design an efficient method for data migration when joining a new replica to a running application that allows for the quick addition of replicas with minimal disruption of transaction processing. Furthermore, by augmenting the adaptation feedback loop with awareness of the delay introduced by the data migration process in our replicated system, we avoid oscillations in resource allocation. We investigate our transparent database provisioning mechanisms in the context of multitier dynamic content Web servers. We dynamically expand/contract the respective allocations within the database tier for two different applications, the TPC-W e-commerce benchmark and the RUBIS online auction benchmark. We demonstrate that our techniques provide quality of service under different load and failure scenarios. Categories and Subject Descriptors: H.2.4 [Database Management]: Systems—Query processing; H.2.7 [Database Management]: Database Administration—Logging and recovery

Recent industry trends towards reducing the costs of ownership in large data centers emphasize the need for database system techniques for both automatic performance tuning and efficient resource usage. The goal is to host several database applications on a shared server farm, including scheduling multiple applications on the same physical server or even within a single database engine, while meeting each application’s service level agreement. Automatic provisioning of database servers to applications and virtualization techniques, such as, live virtual machine migration have been proposed as useful tools to address this problem. In this paper we argue that by allocating entire server boxes and migrating entire application stacks in cases of server overload, these solutions are too coarse-grained for many overload situations. Hence, they may result in resource usage inefficiency, performance penalties, or both. We introduce an outlier detection algorithm which zooms in to the fine-grained query contexts which are most affected by an environment change and/or where a perceived overload problem is likely to originate from. We show that isolating these query contexts through either memory quota enforcements or fine-grained load balancing across different database replicas of their respective applications allows us to alleviate resource interference in many cases of overload. 1

Abstract. Grid technologies have matured over the last few years. This level of maturity is especially true in the field of scientific computing in which grids have become the main infrastructure for scientific problem solving. Due to its success, the use of grid technology rapidly finds its introduction into other fields. One of such fields is enterprise computing in which grids are seen as a new architecture for data centers. In this paper, we describe the vision of enterprise grids, current scientific achievements that will leverage this vision, and challenges ahead. 1