Testing Heuristics: We Have It All Wrong (1995) [92 citations — 1 self]
Popular Tags
No tags have been applied to this document.
BibTeX | Add To MetaCart
author = {J.N. Hooker},
title = {Testing Heuristics: We Have It All Wrong},
journal = {Journal of Heuristics},
year = {1995},
volume = {1},
pages = {33--42}
}
Bookmarks
OpenURL
Abstract:
The competitive nature of most algorithmic experimentation is a source of problems that are all too familiar to the research community. It is hard to make fair comparisons between algorithms and to assemble realistic test problems. Competitive testing tells us which algorithm is faster but not why. Because it requires polished code, it consumes time and energy that could be spent doing more experiments. This paper argues that a more scientific approach of controlled experimentation, similar to that used in other empirical sciences, avoids or alleviates these problems. We have confused research and development; competitive testing is suited only for the latter. Most experimental studies of heuristic algorithms resemble track meets more than scientific endeavors. Typically an investigator has a bright idea for a new algorithm and wants to show that it works better, in some sense, than known algorithms. This requires computational tests, perhaps on a standard set of benchmark p...
