The new Drupal service I built on my EC2 machine is a basic model with no optional extras. Even though it is simple, this service will be functionally tested by the customer.

I am not the best person to run functional tests. Being an infrastructure guy, I don’t have to know much about the business pain a customer is trying to ease. I don’t have to get involved in UAT (User Acceptance Testing) for business applications. I don’t need a functional testing toolkit and I don’t need to follow a UAT methodology. A business expert must confirm the service meets a business need.

The new owner of a cloud service wants to make sure she is getting what she paid for. She will run functional tests to make sure the service meets the specifications. Checking a new service against a list of requirements requires people and manual processes so it doesn’t fit too well with computers and automated processes – but there is a little help available from cloud services.

Functional testing

Functional testing finds out if the business requirements have been met, not the technical requirements. With functional testing, only the client interface is checked, not the technical stack that makes it go. If a project is running out of money, functional testing may be a simple check that the spinning logo is present. If a project is run by a government department, testing may involve a long-term program of recruiting teams of business experts, training them, measuring their interaction with the new service and analysing results.

The customer does not need to know the fine details of non-functional testing. She wants to see a signed security testing report from an independent vendor to prove a cloud provider is secure, but she is not interested in how a hypervisor is examined for rootkits.

It may seem that functional testing of Internet services is an ideal cloud activity. An array of testing clients could constantly beaver away at services all over the Internet. Unfortunately, functional testing is not an activity that lends itself to cloud automation. If an HR administrator adds an employee account to a new federated identity product, he wants to make sure the employee’s details don’t mysteriously disappear from the old phone book, payroll and training products. It’s a requirement that is not easily turned into a repeatable process, let alone a commodity.

Cloud-based testing services

Cloud service vendors are not shouting about their testing capabilities. You won’t find the word testing in GigaOM’s list of the top 50 cloud innovators.

Software development testing is easily automated and big changes are coming. 2012 is the year when code developers everywhere will be sent loving e-mails by PaaS vendors, offering to relieve them of their testing workload. Build/test/deploy suites like Cloudbees DEVCloud and Redhat Openshift promise to take much of the testing off developers’ hands.

Functional testing isn’t so lucky. Some common activities have been factored out into SaaS offerings, such as response time measurement and content checking. Hopefully, this area will get more attention in future. The more work that is offloaded to automated testers, the more time the subject matter expert can spend making sure a new app is actually usable.

A cross-browser compatibility test

One advantage of the cloud is the ease of getting to a service with many different types of devices. This means a service has to look great on both a laptop and a smartphone.

One of the front-end tests that customers get excited about is cross-browser compatibility. Two testing service providers who check compatibility are testingbot.com and saucelabs.com. I know they both have something to do with Selenium (an open source testing application) and cloud services. I have installed Selenium for customers, and I have built cloud services, but that does not qualify me to check compatibility. I am happy to leave cross-browser tests to these professionals.