Mobile app testing for fun and profit

Mobile app testing can be complex but with the right testing platform choice and mobile application management can be quite effective.

Mobile app testing technologies are growing in importance as more developers and enterprises focus on a mobile-first strategy. Here are more details about two mobile app testing platforms, and how mobile application management (MAM) can play a role in mobile app testing.

Perfecto Mobile: mobile app testing in the cloud

Perfecto Mobile is a mobile app testing platform with a presence in major data centers around the globe. Chris Willis, CMO, said, "what we have at the base of everything is physical devices. Lots of them. Thousands of them in every configuration possible. Over that, we lay the controlled environment, so the access to networks and carriers."

There's also a software layer in the Perfecto testing environment that enables functional testing, performance testing, synthetic monitoring and customer support, Willis said.

"So what we provide is devices for, not only, functional tasks, but automation and performance and monitoring as well,"said David Coe, sales engineer for Perfecto Mobile. The Perfecto Mobile testing platform hosts actual mobile devices in the cloud for testing. The company can bring a variety of device, across different carriers.

Coe said, "They [the phones] have carrier contracts. We do not jailbreak or hack these phones in any method or way. Even after you're done with these, we can send them back to you, to our customers, and they can re-insert them into the population for their employees if they want to."

BlazeMeter: Mobile app recorders

BlazeMeter is another mobile app testing platform. It provides performance, testing, app forms for web, mobile and CPI testing, said Ophir Prusak, chief evangelist for BlazeMeter. He said BlazeMeter is a single platform and that differentiates them from the competition.

Prusak said Blazemeter is based on the JMeter open source project that's an alternative to the popular Load Runner testing tool. He said BlazeMeter is to JMeter what GitHub is to Git.

BlazeMeter services can be used as a service through an application programming interface (API) or integration with a continuous integration testing and APM tools.

Prusak said customers use BlazeMeter for performance testing help after trying to use simulators for testing native mobile apps and hardly efficient for recording user activity.

"If you're doing testing for a mobile website you can more or less use the same tools as you would when just testing out a normal website with your browser," Prusak said. "Ultimately, it should still work with your browser, and there are plug-ins and extensions which work with today's browsers which you can modify HTTP headers or even the resolution and make the backend still think you're connecting on a mobile device."

"I'm aware of a plethora of different solutions and all of them require either jail-breaking the device or installing software on your computer and then pointing your phone or device to your computer and using that as a proxy," Prusak said. He sees these solutions as rife with issues, inefficient, and too complex.

BlazeMeter took this testing issue as a challenge to make a testing platform which made recording the actual activity of recording mobile device activity easy for anybody regardless of that device type and play the that back en masse so that it replicates 50, 500 or 5,000 users, Prusak said.

App47: MAM can be integral to app testing

Chris Schroeder, CEO and co-founder of App47, a mobile application management platform, raised the following questions as to how a MAM fits into mobile app testing:

1. What if you were building 10 apps, and five of those apps were upgrades to a previous version?

2. What if those apps only worked on a specific OS level or a specific device?

3. What if you needed to get four of those apps to a group of 50 testers and ensure they were testing the current version of your test build?

"With a MAM solution in place, you have the ability to 'publish' a test build to a branded app store right from your development environment," Schroeder said. "The MAM solution manages both users and the apps that they may access. In addition, it knows all about the devices that are registered for each user so that the app store will only present test builds that fit that device or OS reducing confusion and improving the overall user experience."

Schroeder said a MAM solution can enforce app level policies encouraging users/testers to upgrade to the latest app eliminating the "testing the wrong version" issues that often arises.

"MAM solutions should provide a rich set of data from the actual use of your app," Schroeder said. "This includes event logging such as what is the user doing with the app? Are they actually using that new design idea that everyone wanted but eats up valuable screen real estate?"

MAM can be a useful source of performance metrics during testing by showing how the app is performing for the user. This is how fast the app is processing and displaying the data, not how fast your web service is delivering data to the app, Schroeder said.

Crash data needs reporting and a good MAM solution allows you to search this information so you can troubleshoot these problems down to the device and user level.

"Once you have analyzed user behavior, app performance, and crash logs, developers will have a full 360-degree view of how best to improve the app before going live," he said. "Now it will be a snap to update your app, immediately publish the next test build and ensure your users are on that current version supplying meaningful data so that the 'you want it released by when?' development timelines are met."

"The primary consideration in integrating a development environment to a MAM solution surrounds the level of automation," Schroeder said.

"For high volume development shops where multiple apps are being built and tested simultaneously, an automated release from dev to test is recommended, because MAM vendors already have built-in connectors to major development platforms, he said.

For others, simply uploading builds and setting the policy and active version attributes is a simple task. The MAM solution takes care of notifications and policy enforcement so that this process is a set it and forget process for the developer.

Mobile app testing and your enterprise

Keep in mind; I'm only touching the surface of the topic in this article. Perfecto Mobile and BlazeMeter are but two examples of mobile app testing platforms on the market. My one reaction to delving into the mobile app testing industry is its complexity. We can expect to see mobile app testing evolve to keep pace with mobile app development.

See also:

10 low-cost ways to develop and improve applications

Ten Answers Regarding Mobile App Testing

Mobile Testing: Follow these Five Steps to Simplify and Accelerate App Testing

Deliver Mobile Apps: A Lifecycle Approach to Mobile Testing and Development