User interface test suite vendor Applitools published a report Wednesday contending that companies face substantial costs for even minuscule visual bugs in software.
Developers, QA teams, and tech leaders often concern themselves with eliminating under-the-hood bugs that have an impact on app functionality. The report suggests that this focus on functional testing comes at the cost of UI and other visual bugs, which is bad news: The visual aspect of an app is the first thing users see.
Visual bugs are common and costly, the report said. The average company ends up with nine visual bugs per software release, needs 12 hours to resolve each one, and ends up paying a total of $36,426 per release cycle to get everything fixed. 30% of business, however, are in much worse shape. Those businesses end up with around 22 visual bugs per release, need 22 hours to fix each one, and end up shelling out $143,989 to do so.
The report states that the average company will release 108 visual bugs over the course of a year, which assumes one release per month. That totals $437,000 spent on fixing UI and other visual issues, all of which amounts to lost money.
What is to be done?
As with many problems in the modern software world, shared responsibilities and divided roles can save time, but they can also lead to problems that no one is able to fully claim responsibility for.
For visual bugs, the report found responsibility to be lacking: Engineering, testers, product teams, and marketing respondents were evenly split on who was responsible for visual bugs. In this case, the report concludes, everyone is responsible, hence no one is responsible.
Add to that the problem of the sheer number of visual elements that most companies are responsible for and you have yet another ingredient in a disastrous recipe. The average company has over 90,000 screens it is responsible for, with the largest 30% of companies having over 600,000 accessible at any given time.
It isn’t reasonable, the report said, for humans to be able to completely manage visuals on their web sites and web apps: There simply isn’t enough time in the day.
The conclusion that Applitools draws is that automated visual testing is a necessity for being able to eliminate the large number of visual bugs that slip through the cracks.
Given the proliferation of other forms of automated testing it makes sense for specific tools to be used in testing UIs and other visual elements. Applitools found that automated visual testing increases coverage anywhere from 25% to 65%, shortens release cycles by up to 40%, and improves release quality by 260% (in terms of the number of visual bugs released on average).
With that in mind it’s also important to remember who is releasing this report: Applitools itself produces the software it is saying companies should use to eliminate visual errors. If your organization has found that UI and other visual bugs are common, or at the very least consistent, across releases it’s a good idea to consider a visual testing automation tool.
It’s also important for IT leaders to try finding an in-house solution before spending money on new software, particularly addressing problems of responsibility. Figuring out who should be addressing visual and UI bugs is essential in eliminating them, which can’t be done by testing automation alone.