DMCA
Are Test Cases Needed? Replicated Comparison between Exploratory and Test-Case-Based Software Testing
| Citations: | 2 - 0 self |
BibTeX
@MISC{Itkonen_aretest,
author = {Juha Itkonen and Mika V. Mäntylä},
title = {Are Test Cases Needed? Replicated Comparison between Exploratory and Test-Case-Based Software Testing},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Manual software testing is a widely practiced verification and validation method that is unlikely to fade away despite the advances in test automation. In the domain of manual testing, many practitioners advocate exploratory testing (ET), i.e., creative, experience-based testing without predesigned test cases, and they claim that it is more efficient than testing with detailed test cases. This paper reports a replicated experiment comparing effectiveness, efficiency, and perceived differences between ET and test-case-based testing (TCT) using 51 students as subjects, who performed manual functional testing on the jEdit text editor. Our results confirm the findings of the original study: 1) there is no difference in the defect detection effectiveness between ET and TCT, 2) ET is more efficient by requiring less design effort, and 3) TCT produces more falsepositive defect reports than ET. Based on the small differences in the experimental design, we also put forward a hypothesis that the effectiveness of the TCT approach would suffer more than ET from time pressure. We also found that both approaches had distinctive issues: in TCT, the problems were related to correct abstraction levels of test cases, and the problems in ET were related to test design and logging of the test execution and results. Finally, we recognize that TCT has other benefits over ET in managing and controlling testing in large organizations.
Keyphrases
test-case-based software testing test case needed test case detailed test case original study time pressure test-case-based testing manual testing replicated experiment small difference jedit text editor exploratory testing large organization manual functional testing test execution tct approach distinctive issue defect detection effectiveness experimental design manual software falsepositive defect report validation method test automation abstraction level many practitioner design effort
