A Human Study of Fault Localization Accuracy
Localizing and repairing defects are critical software engineering activities. Not all programs and not all bugs are equally easy to debug, however. They present formal models, backed by a human study involving 65 participants (from both academia and industry) and 1830 total judgments, relating various software- and defect-related features to human accuracy at locating errors. Their study involves example code from Java textbooks, helping one to control for both readability and complexity. They find that certain types of defects are much harder for humans to locate accurately. For example, humans are over five times more accurate at locating "Extra statements" than "Missing statements" based on experimental observation.