Exploring Realistic Program Behavior
Modern test case generation techniques can automatically achieve high code coverage. If they operate on the unit level, they run the risk of generating nonsensical inputs, which, when causing failures, are painful to identify and eliminate. Running a unit test generator on five open source Java programs, the authors found that all of the 181 reported failures were false failures - that is, indicating a problem in the generated test case rather than the program. By generating test cases at the GUI level, their EXSYST prototype can avoid such false failures by construction. In their evaluation, it achieves higher coverage than search-based test generators at the unit level; yet, every failure can be shown to be caused by a real sequence of input events.