. Current I/O benchmarks suffer from several chronic problems: they quickly become obsolete, they do not stress the I/O system, and they do not help in understanding I/O system performance. We propose a new approach to I/O performance analysis. First, we propose a self-scaling benchmark that dynamically adjusts aspects of its workload according to the performance characteristic of the system being measured. By doing so, the benchmark automatically scales across current and future systems. The evaluation aids in understanding system performance by reporting how performance varies according to each of five workload parameters. Second, we propose predicted performance, a technique for using the results from the self-scaling evaluation to quickly estimate the performance for workloads that have not been measured. We show that this technique yields reasonably accurate performance estimates and argue that this method gives a far more accurate comparative performance evaluation than tradition...

. The increasing number of advanced database management systems offered on the market requires tools to quickly assess their performance and to assure their quality. Performance measurement involves running a set of representative workloads, such as benchmarks, and quality assurance, which involves extensive testing. The Software Testpilot 1 described in this paper greatly simplifies both jobs by enabling a compact specification of the workload search space, a flexible mechanism to interact with a system under study, and a fast algorithm to expose the performance bottlenecks or software instabilities. 1 Introduction Database technology for single processor architectures has matured from file processing by a large number of individual programs to sound data models and data manipulation schemes applicable in many environments. The DBMSs are delivered as portable, single processor implementations with many techniques to obtain good performance. However, new (prototype) systems...