Institute of Electrical & Electronic Engineers
Malware researchers rely on the observation of malicious code in execution to collect datasets for a wide array of experiments, including generation of detection models, study of longitudinal behavior, and validation of prior research. For such research to reflect prudent science, the work needs to address a number of concerns relating to the correct and representative use of the datasets, presentation of methodology in a fashion sufficiently transparent to enable reproducibility, and due consideration of the need not to harm others. In this paper, the authors study the methodological rigor and prudence in 36 academic publications from 2006 - 2011 that rely on malware execution. 40% of these papers appeared in the 6 highest-ranked academic security conferences.