A game developer describes his successful experiment of using Facebook to beta test his app.
There are many stages to the lifecycle of a mobile app - Concept, Design, Development, Testing, Publishing, and Marketing. One thing we probably do not do enough of is thoroughly test our application before publishing it. Let's face it, testing is difficult. There are lots of mobile devices, each with different hardware characteristics, and lots of different OS versions to compound the problem. So, on my most recent project I wondered if Facebook could help.
Note: This article originally published in our iOS App Builder blog in June 2013.
The development of my new game, Compulsive, was almost finished and I decided to ask my online friends if they could help try it out before I published. I couldn't believe that over 100 people responded saying they were willing to help out.
My problem now shifted from needing testers to how was I going to manage this entire process? I decided to create a private Facebook group called, "TMSOFT Elite Users," and then sent invites to everyone on that thread to join. This way all communications could be private between me and the testers. And I'm sure my family would appreciate that my wall and their news feeds wouldn't get blasted with stuff they don't care about. This also allowed everyone to communicate and post questions to the group and not just myself. It worked really well and became a thriving community with questions, suggestions, and bugs being posted to the forum.
The most tedious part of the process was asking everyone for their unique device identifier (UDID) so I could register their device for ad hoc iOS builds. This allows users to install apps that aren't already published on the App Store. There is a useful app called UDID by Harrison Apps that allows users to get that information from their device and then post it to the forum. I would then take all of those IDs and copy it into an Excel document for upload to Apple.
I quickly ran out of test devices because Apple only allows you to register up to 100 and I already had 60 allocated even though most of them were no longer being used. Apple lets you clear unused UUIDs at the annual renewal of your developer membership but that was a few months ago for me. I sent a quick email to Apple support to see if they would make an exception and allow me to delete some devices and they responded the next day that they granted me a one-time exception. I was then able to delete all my old devices and free up enough space for everyone to participate in the beta.
If I could do the whole UDID process over, I would have created a shared Google Doc spreadsheet and just had everyone fill in their name, device model, and UUID. That would have greatly simplified the process and been less work for me. Make sure to download the sample spreadsheet file that Apple provides so that it is formatted correctly and won't be rejected when you upload.
I was able to get my Beta 1 build posted to Google Drive and provided a download link to everyone in the forum so they could install the game using iTunes. Once my beta users got the game installed on their device, the forum came alive with lots of posts, screenshots, and even videos. My first major bug rolled in:
In the game you have to group four or more same color tiles together to trigger an explosion and remove those pieces from the board. As pointed out from Shelli's screenshot that did not happen when she joined five blue tiles together. This gave me some really good information, especially how the board was configured when the bug occurred. This made it much easier to figure out how to recreate the bug and ultimately fix it.
Another user posted a video of their experience with the game, and it wasn't great. They felt like it was too difficult to grab the tiles, and his video really helped to show that. This would require a major redesign under the hood, and I initially didn't want to do this because it would most likely (and did) create new issues. But in the end, it was the right thing to do, as the game play experience greatly improved.
There were countless other bugs and feature suggestions that were shared in the community. One of my favorites was a request to listen to your own music while playing the game. This was suggested by numerous users, which motivated me to come up with an elegant solution. Now when you launch Compulsive, it will detect if you already have music playing and if so temporary disable the in-game music until the your music finishes. That's much better than having to change the setting to not play music, go back to your music app, restart it, and then return to Compulsive to play a game. Those little touches are just an example of how my game really benefited from this experience.
My Facebook experiment was so successful that it will become part of my business process before I publish any new apps. The next time you're getting ready to launch a new app you might try leveraging Facebook for something other than sharing cat photos. I felt a lot more confident launching Compulsive v1.0 knowing it had been thoroughly reviewed and enjoyed by an amazing group of elite users.
Compulsive is a free download for Android and iOS devices.