Simple split-testing for ads in Android apps

Split-tests (or A/B tests) will help you determine which ad settings or ad networks get the most revenue, as well as other things, for your Android app. Here's how to set up a split-test.

I've been a big fan of split-testing, also known as A/B testing, since before I released Android apps. It's important to know the difference that your choices can make, because some things you can only find out by testing.

Split-testing is used throughout web and mobile development, but not every developer is comfortable with it. Split-testing doesn't have to be difficult, however—you don't even need to employ an expensive or complicated service.

I've already covered the importance of testing and measuring ad performance for your Android apps. Now I'm going to show how to set up a quick and simple split test that won't cost anything.

What is split-testing?

Split-testing is the practice of performing a controlled test to measure which input gets the best results. In terms of ads in Android apps, this means finding which ad settings or ad networks get the best revenue.

There are other things to measure, such as click-through rate (the percentage of views that result in a click) and eCPM (effective cost-per-mille, or thousand impressions). Read more on cost-per-click and cost-per-action mobile ads if you are unfamiliar with the terms.

Example code and explanation

I previously covered different ways to show ads in Android apps. This sample code can be used for either XML (layout) based ad implementations or with code-instantiated ads. I will focus on the XML-based implementation for clarity. First, we're going to create two different layouts—one with each ad type.

Variant 1 (main_1.xml):

<?xml version="1.0" encoding="utf-8"?>

Variant 2 (main_2.xml):

The second variant is identical, except it replaces android:layout_alignParentTop with android:layout_alignParentBottom; this moves the ad from the top to the bottom. Use a different ad unit ID, and you can now compare the results. In addition, your app should have content filling the rest of the screen for both layouts. The content should be identical to make this test meaningful.

Next, we'll set the activity up to randomly select between the two (you're not limited to just two).

Add this to
// Randomly decide which layout variant to show
final Random random = new Random(System.currentTimeMillis());
final int selector = random.nextInt(2); // [0, 2)

if(selector == 0) {
    Log.i(this.toString(), "Loading layout 1");
} else {
    Log.i(this.toString(), "Loading layout 2");

When your activity loads, it will randomly decide which layout to use, bringing the selected ad layout.

You can use this for a number of things—testing different ad placements (top/bottom), different ad networks, or even more complex factors such as refresh rates, demographic information, or other ad settings.

Where to go from here

The above solution won't cost you a dime, is easy to maintain and change, and gets you started quickly. You can see the results by watching the two ad units (which should have similar impression counts), and you can directly compare the revenue. Pick a winner after sufficient time and run with it!

You'll eventually outgrow this mechanism. As I continued to research ad networks, I have changed my setup a number of times. To confuse the issue further, not all users update the app, which leaves old versions of my split-test out there. This all results in the impression counts no longer matching, and the test being harder to evaluate.

There are a number of options to help manage these tests in a more controlled manner, and by the time your have enough traffic to outgrow the setup I described, you will probably have enough ad income to justify spending a little more time investigating.

Some ad mediators and ad networks can perform split testing for you on their servers, which will make it easier to implement and maintain these tests. Other ad mediators will automatically allocate ads based on their measurements without any intervention from you.

Whichever method you choose, keep testing and go with what performs best in your Android apps!

About Tim Mackenzie

Tim Mackenzie, author of the Android Income Series books, is a software engineer that escaped the cubicle world at a large company to go solo with Android app development. He uses this freedom to teach others how to make money with Android apps. Visi...

Editor's Picks

Free Newsletters, In your Inbox