Abstract — Understanding the behavior of emerging workloads is important for designing next generation microprocessors. For addressing this issue, computer architects and performance analysts build benchmark suites of new application domains and compare the behavioral characteristics of these benchmark suites against well-known benchmark suites. Current practice typically compares workloads based on microarchitecture-dependent characteristics generated from running these workloads on real hardware. There is one pitfall though with comparing benchmarks using microarchitecture-dependent characteristics, namely that completely different inherent program behavior may yield similar microarchitecture-dependent behavior. This paper proposes a methodology for characterizing benchmarks based on microarchitecture-independent characteristics. This methodology minimizes the number of inherent program characteristics that need to be measured by exploiting correlation between program characteristics. In fact, we reduce our 47dimensional space to an 8-dimensional space without compromising the methodology’s ability to compare benchmarks. The important benefits of this methodology are that (i) only a limited number of microarchitecture-independent characteristics need to be measured, and (ii) the resulting workload characterization is easy to interpret. Using this methodology we compare 122 benchmarks from 6 recently proposed benchmark suites. We conclude that some benchmarks in emerging benchmark suites are indeed similar to benchmarks from well-known benchmark suites as suggested through a microarchitecture-dependent characterization. However, other benchmarks are dissimilar based on a microarchitecture-independent characterization although a microarchitecture-dependent characterization suggests the opposite to be true. I.

A large fraction of the data that will stored and accessed in future systems is expected to be unstructured, in the form of images, audio files, etc. Therefore, it is very important to design future I/O subsystems to provide efficient storage, and access to these vast and continuously growing repositories of unstructured data. To facilitate system design and evaluation, we first need benchmarks that capture the processing and I/O access characteristics of applications that operate on unstructured data. In this paper, we present an unstructured data processing benchmark suite that we have developed. We provide detailed descriptions of the workloads in the benchmark suite and discuss the larger space of application characteristics that each of them capture. Keywords: Benchmarks, unstructured data, I/O. 1

Characterizing and understanding emerging workload behavior is of vital importance to ensure next generation microprocessors perform well on their anticipated future workloads. This paper compares a number of benchmark suites from emerging application domains, such as bio-informatics (BioPerf), biometrics (BioMetricsWorkload) and multimedia (MediaBench II), against generalpurpose workloads represented by SPEC CPU2000 and CPU2006. Although these benchmark suites have been characterized before, prior work did not capture the benchmark suites ’ inherent (microarchitecture-independent) behavior, nor did they provide a phase-level characterization. In this paper, we characterize these existing and emerging general-purpose and domain-specific benchmark suites in terms of their inherent phase-level behaviors. Our characterization methodology identifies prominent phase behaviors across benchmarks and visualizes them in terms of their key microarchitecture-independent characteristics. From this analysis, next to revealing a detailed picture of the distinct phase-level behaviors in these benchmark suites, we also obtain a number of interesting high-level insights. For example, SPEC CPU2006 turns out to be the benchmark suite with the largest workload space coverage, i.e., it covers the largest set of distinct behaviors in the workload space. Also, our analysis provides experimental evidence supporting the intuitive understanding that domainspecific benchmark suites cover a narrower range in the workload space than general-purpose benchmark suites. Finally, the BioPerf bio-informatics benchmark suite exhibits a large fraction unique behaviors not observed in the general-purpose benchmark suites, substantially more so than the other domain-specific benchmark suites, Bio-MetricsWorkload and MediaBench II. 1