Characterizing the Differences Between Pre-And Post-Release Versions of Software
Many software producers utilize beta programs to predict post-release quality and to ensure that their products meet quality expectations of users. Prior work indicates that software producers need to adjust predictions to account for usage environments and usage scenarios differences between beta populations and post-release populations. However, little is known about how usage characteristics relate to field quality and how usage characteristics differ between beta and post-release. In this paper, the authors examine application crash, application hang, system crash, and usage information from millions of Windows users to examine the effects of usage characteristics differences on field quality (e.g. which usage characteristics impact quality), examine usage characteristics differences between beta and post-release (e.g. do impactful usage characteristics differ), and report experiences adjusting field quality predictions for Windows.