How Diffblue uses AI to automate unit testing for Java applications

Commentary: Software keeps getting more complicated to build. Robots might be the answer.

aijobs.jpg

Image: JIRAROJ PRADITCHAROENKUL, Getty Images/iStockphoto

Let's do some math. Developers are critically important to every business, and they're not cheap. According to the Bureau of Labor Statistics, the median annual salary for a software developer in the US is close to $110,000. In San Francisco, it's closer to $145,000, where entry-level developers can command $100,000. Meanwhile, developer tools are typically priced using the size of development team (seats) as a proxy. Production platforms, on the other hand, are typically charged by the size of the environment. 

See the opportunity? An investment in developer efficiency gives smart companies enormous leverage on that investment. Small groups of developers can deploy massive systems. 

Unfortunately, the average developer spends up to half of her time writing tests on code before pushing it through the CI/CD process. It's a tedious, manual task that, while critical to the business, consumes developer time that could otherwise be spent building. 

But maybe, just maybe, artificial intelligence (AI) can help.

Putting the machines to work

Diffblue, a UK-based startup spun out of the University of Oxford, certainly believes so. On Tuesday Diffblue announced general availability of its flagship Diffblue Cover and Diffblue Cover: Community Edition, a free version created for developers using IntelliJ, the most popular IDE for enterprise Java. Using AI, Diffblue automates writing Java unit tests with up to 80% coverage in a human-readable form that automatically maintains the tests as the code evolves, even on applications with millions of lines of code.

Oh, and it does this 10X to 100X faster than humans. Not bad.

SEE: 10 ways to prevent developer burnout (free PDF) (TechRepublic)

The irony, as Diffblue CEO Mathew Lodge pointed out in an interview, is how late the software industry is in embracing AI to improve software development, given how we've used AI to automate and disrupt so many other industries--from retail, travel, transportation, manufacturing, and more.

Lodge said Diffblue researchers took advantage of the machine learning strategy that powered AlphaGo, Alphabet subsidiary DeepMind's software program that beat the world champion player of Go. While the company is starting with a Java solution (by far the most popular language in the Global 2000 where companies invest heavily in productivity tools), its technology can also be used to automate testing for most programming languages such as Python, JavaScript, and C#, among others. 

Among the first customers to roll out Diffblue's solution is Goldman Sachs (with an annual IT budget larger than many countries' GDP). Using Diffblue's AI on one module with an important backend system, Goldman Sachs was able to expand existing unit test coverage from 36% to 72% in less than 24 hours, a feat that would have required more than eight days of developer time if done manually. Developer time savings? 90%. 

Lodge argued that systems are becoming so complex and large that huge portions of the construction, delivery, and maintenance of the software supply chain are just better handled by machines than by humans. In this perspective, he's joined by Tyler Jewell, an investor who has written some of the best analysis on developer tooling. According to Jewell:

Software systems are more complicated due to tool and programming language fragmentation, API sprawl, microservices, and the layering of big data, analytics, ML, and reactive user experiences into projects. Along with this growing complexity, continuous delivery has shortened the feedback loop while providing deeper insights to developers for each change. Machines are better equipped than humans to act on continuous feedback and recommend changes for extremely complex systems.

As Jewell continued, "[D]emand for digital services [has] outstrip[ped] the supply of engineers to build them, creating opportunities for vendors who could improve developer productivity." 

A robotic symphony

Diffblue is not alone in seeing this as a huge market opportunity. Driven in large part by the emerging DevOps trend, developer, operations, and security solutions are increasingly shifting into pre-production, and further into the hands of developers who are building the modules prior to their orchestration (for examples look at Cyral for security, Vercel for JavaScript and React front-end frameworks, and Diffblue for unit-test automation). 

This larger shift aligns with the industry's adoption of containers (and usually cloud) in a new world of digital transformation and cloud native. Certainly the spectacular quarterly earnings and growth reported by many public SaaS companies in the middle of a global pandemic suggest what the category winners will look like in the future. Today, the median SaaS/cloud public markets EV/next 12 month revenue ratio is a mind-boggling 15x. The 2020 global Information Technology industry will reach $5.2 trillion, of which $624B is software. And developers drive an ever-increasing share of the software market.

Autonomous software development will be a big part of the future of computing. Diffblue is only the beginning of the AI disruption of the software industry.

Disclosure: I work for AWS, but the views expressed herein are mine and don't reflect those of my employer.

Also see