As Internet services become ubiquitous, the selection and management of diverse server platforms now affects the bottom line of almost every firm in every industry. Ideally, such cross-platform management would yield high performance at low cost, but in practice, the performance consequences of such decisions are often hard to predict. In this paper, we present an approach to guide cross-platform management for real-world Internet services. Our approach is driven by a novel performance model that predicts application-level performance across changes in platform parameters, such as processor cache sizes, processor speeds, etc., and can be calibrated with data commonly available in today’s production environments. Our model is structured as a composition of several empirically observed, parsimonious sub-models. These sub-models have few free parameters and can be calibrated with lightweight passive observations on a current production platform. We demonstrate the usefulness of our cross-platform model in two management problems. First, our model provides accurate performance predictions when selecting the next generation of processors to enter a server farm. Second, our model can guide platform-aware load balancing across heterogeneous server farms. 1

The typical workload in a database system consists of a mixture of multiple queries of different types, running concurrently and interacting with each other. Hence, optimizing performance requires reasoning about query mixes and their interactions, rather than considering individual queries or query types. In this paper, we show the significant impact that query interactions can have on workload performance. We present a new approach based on planning experiments and statistical modeling to capture the impact of query interactions. This approach requires no prior assumptions about the internal workings of the database system or the nature or cause of query interactions, making it portable across systems. As a concrete demonstration of the potential of capturing, modeling, and exploiting query interactions, we develop a novel interaction-aware query scheduler that targets report-generation workloads in Business Intelligence (BI) settings. Under certain assumptions, the schedule found by this scheduler is within a constant factor of optimal. An experimental evaluation with TPC-H queries on IBM DB2 demonstrates that our scheduler consistently outperforms (up to 4x) conventional schedulers that do not account for query interactions.

As the complexity of IT systems increases, performance management and capacity planning become the largest and most difficult expenses to control. New methodologies and modeling techniques that explain large-system behavior and help predict their future performance are now needed to effectively tackle the emerging performance issues. With the multi-tier architecture paradigm becoming an industry standard for developing scalable client-server applications, it is important to design effective and accurate performance prediction models of multi-tier applications under an enterprise production environment and a real workload mix. To accurately answer performance questions for an existing production system with a real workload mix, we design and implement a new capacity planning and anomaly detection tool, called R-Capriccio, that is based on the following three components: i) a Workload Profiler that exploits locality in existing enterprise web workloads and extracts a small set of most popular, core client transactions responsible for the majority of client requests in the system; ii) a Regression-based Solver that is used for deriving the CPU demand of each core transaction on a given hardware; and iii) an Analytical Model that is based on a network of queues that models a multi-tier system. To validate R-Capriccio, we conduct a detailed case study using the access logs from two heterogeneous production servers that represent customized client accesses to a popular and actively used HP Open View Service Desk application.

Complex system software allows a variety of execution conditions on system configurations and workload properties. This paper explores a principled use of reference executions—those of similar execution conditions from the target—to help identify the symptoms and causes of performance anomalies. First, to identify anomaly symptoms, we construct change profiles that probabilistically characterize expected performance deviations between target and reference executions. By synthesizing several single-parameter change profiles, we can scalably identify anomalous reference-totarget changes in a complex system with multiple execution parameters. Second, to narrow the scope of anomaly root cause analysis, we filter anomaly-related low-level system metrics as those that manifest very differently between target and reference executions. Our anomaly identification approach requires little expert knowledge or detailed models on system internals and consequently it can be easily deployed. Using empirical case studies on the Linux I/O subsystem and a J2EE-based distributed online service, we demonstrate our approach’s effectiveness in identifying performance anomalies over a wide range of execution conditions as well as multiple system software versions. In particular, we discovered five previously unknown performance anomaly causes in the Linux 2.6.23 kernel. Additionally, our preliminary results suggest that online anomaly detection and system reconfiguration may help evade performance anomalies in complex online systems.

Abstract — The typical workload in a database system consists of a mixture of multiple queries of different types, running concurrently and interacting with each other. Hence, optimizing performance requires reasoning about query mixes and their interactions, rather than considering individual queries or query types. In this paper, we use such a reasoning approach to develop a query scheduler. We treat the database system as a black box and experimentally build a model to estimate the performance of different query mixes. Our scheduler uses this model to decide which query mixes to schedule, with the goal of maximizing throughput. We experimentally demonstrate the effectiveness of our scheduler using queries from the TPC-H benchmark on DB2. I.

Many emerging distributed applications require the realtime processing of large amounts of data that are being updated continuously. Distributed stream processing systems offer a scalable and efficient means of in-network processing of such data streams. However, the large scale and the distributed nature of such systems, as well as the fluctuation of their load render it difficult to ensure that distributed stream processing applications meet their Quality of Service demands. We describe a decentralized framework for proactively predicting and alleviating hot-spots in distributed stream processing applications in real-time. We base our hot-spot prediction techniques on statistical forecasting methods, while for hot-spot alleviation we employ a non-disruptive component migration protocol. The experimental evaluation of our techniques, implemented in our Synergy distributed stream processing middleware over PlanetLab, using a real stream processing application operating on real streaming data, demonstrates high prediction accuracy and substantial performance benefits. 1.

Abstract — Configuring virtual machine parameters in a virtualized server environment is a challenging task for IT operators. For instance, while current management products can dynamically size virtual machines in order to maintain resource utilization targets, default target values are rarely ideal for individual workloads and operators find it difficult to decide appropriate values for the targets. This paper presents AutoParam, a tool that provides application-level performance guarantees by automatically determining system-level parameters such as the utilization targets and the sizes of the virtual servers hosting individual tiers of multi-tier applications. AutoParam is based on synthesis of a feed-forward transaction-mix-based queueing model and feedback control loops. We describe the integration of AutoParam with the Xen virtualization system, and present empirical results showing that AutoParam effectively adjusts sizes of Xen virtual machine containers to maintain mean transaction response time at a desired level in spite of variable workloads. We compared AutoParam to four other designs, and show that it provides more reasonable tradeoffs between application performance and resource efficiency as well as more robust dynamic properties. I.

Database workloads consist of mixes of queries that run concurrently and interact with each other. In this paper, we demonstrate that query interactions can have a significant impact on database system performance. Hence, we argue that it is important to take these interactions into account when characterizing workloads, designing test cases, or developing performance tuning algorithms for database systems. To capture and model query interactions, we propose using an experimental approach that is based on sampling the space of possible interactions and fitting statistical models to the sampled data. We discuss using such an approach for database testing and tuning, and we present some opportunities and research challenges. 1.

Online instructional applications, social networking sites, Wiki-based web sites, and other emerging web applications that rely on end users for the generation of web content are increasingly popular. However, these collaborative web applications are still absent from the benchmark suites commonly used in the evaluation of online systems. This paper argues that collaborative web applications are unlike traditional online benchmarks, and therefore warrant a new class of benchmarks. Specifically, request behaviors in collaborative web applications are determined by contributions from end users, which leads to qualitatively more diverse server-side resource requirements and execution patterns compared to traditional online benchmarks. Our arguments stem from an empirical examination of WeB-WorK — a widely-used collaborative web application that allows teachers to post math or physics problems for their students to solve online. Compared to traditional online benchmarks (like TPC-C, SPECweb, and RUBiS), WeBWorK requests are harder to cluster according to their resource consumption, and they follow less regular patterns. Further, we demonstrate that the use of a WeBWorK-style benchmark would probably have led to different results in some recent research studies concerning request classification from event chains and type-based resource usage prediction. 1

Large cloud-computing datacenters now host a wide range of business applications, ranging from e-commerce websites to search engines to data mining. Increasingly, these datacenters use renewable energy from wind turbines