Citation Context

...ime [11], [16], [61]. Siegmund et al. [50] and Guo et al. [20] predict how configuration options influence performance, Trubiani et al. [53] consider uncertainty in performance modeling, Malik et al. =-=[33]-=- detect deviations in load tests, and Lu and Song [51] investigate design points in performance statistical debugging. Other techniques generate performance tests [8], [10], [18], [48], [62], detect p...

Citation Context

...ols and infrastructures for load testing (e.g., Apache JMeter [1] and LoadRunner [2]). There are also an increasing number of research works done recently in the area of load testing (e.g., [3], [6], =-=[7]-=-, [10], [12]). Load testing will become more important, as more services (e.g., Microsoft Office 365 and Salesforce.com) are being offered in the cloud to millions or even billions of users. This tech...

Citation Context

...es [7, 39] focus on how to easily manipulate and present these subpaths to the user. Other techniques detect performance problems that manifest as anomalous behavior [44,45], deviations in load tests =-=[24]-=-, or performance regressions [33]. For all these techniques, the performance problems need to manifest in the observed runs. In contrast, SUNCAT provides information to the developer to help reason ab...

Citation Context

...h on testing nonfunctional attributes have garnered much attention since the advent of the object-oriented paradigm. In performance testing, system throughput metrics can highlight degradation errors =-=[121]-=-, [124], as when a server fails to respond when a number of requests are sent simultaneously. A case study by Weyuker and Vokolos showed how a process with excessive CPU usage caused service delays an...

Citation Context

...ared statistically, and a deviation report is generated. This approach is limited by the granularity of the log files and also suffers from the same issues as the statistical techniques. Malik et al. =-=[39]-=- suggested to reduce the number of counters generated by performance regression testing to a manageable number that can be analysed by performance experts and is known as application signature. The nu...

Citation Context

...to detect performance regressions. In particular, they build a control chart for every performance counter and examine the violation ratio of the same performance counter for a new test. Malik et al. =-=[23, 24]-=- propose approaches that cluster performance counters using Principal Component Analysis (PCA). Each component generated by PCA is mapped to performance counters by a weight value. The weight value me...

Citation Context

...ensemble of classifiers [37]. When an SLO violation is detected, the most relevant classifier is selected from the ensemble to report the counters that correlate with the particular SLO violation. In =-=[26]-=-, Malik et al. presented a wrapper-based supervised approach, which detects performance problems. These supervised techniques require counters to explicitly exceed certain thresholds (e.g., SLOs). In ...

Citation Context

...performance in identifying performance counters thatsare sensitive to minute changes in both workload andsenvironment as compared to many other supervised andsunsupervised machine learning techniques =-=[6]-=-. We provide ansoverview of the PCA based performance counter selectionstechnique in this paper. Further details are discussed in oursprevious work [7]. Basically, the high level goal of using PCAsin ...