There are numerous reviews about Android security apps. But, Michael Kassner does not remember any mention of actual testing. Why is that?
It's been my good fortune to have TechRepublic writer and software engineer William Francis collaborate with me on several articles about Android security -- our latest, "Does Android's permission system really work?"
I mention this for a reason. In the comment section of each article, more than a few members questioned whether the malware-scanning portion of their Android security app was actually working -- not one example of any captured malcode.
That did not sit well. Security pundits -- I include myself -- have been stressing the need for security apps on all Internet-capable devices, including smartphones. I surely do not want to be yelling fire when there is none.
After a bit of head-scratching, I came up with several hypotheses:
- There is no malware.
- The bad guys aren't up to speed yet.
- Phone technology is good enough without security apps.
- Security apps are not catching malware.
It was easy to resolve the first two:
- There is malware in the wild dedicated to Android firmware (this just released).
- I can't speak for the bad guys (says my mentor).
What about the last two? I decided to test my hypotheses, just like the pros. To that end, I dug out my university book on testing methodology. For a split second, I wondered about it being written in 1970. Nah, it couldn't have changed that much.
After some page-flipping, it all came back -- including how complicated accurate testing really is. Do not fear, I have a plan.
I called William -- damn good plan if you ask me. I explained my foray into "testing methodology" and why I needed his help. His comment stopped me cold. "I'd love to run your tests Michael, just as soon as you get me the Android malcode."
For some weird reason, I immediately envisioned myself as this cloak and dagger spy whose mission was to purloin contraband from the "digital underworld". Rudely, William interrupted my moment with, "Well?"
‘I'll figure something out."
Thinking clearer the next day, I decided to leave my James Bond kit in the closet, where it was safe. An even smarter decision was to contact Adrienne Porter Felt, an extremely-talented Ph.D. candidate at Berkeley. She understands everything Android. In fact, Adrienne has helped William and me on past articles.
I called Adrienne. She said no problem -- but... Knowing where this was going, I interrupted: "Don't worry, William will be handling the malware. And, I will let you know what we find." I swear I heard a sigh of relief.
That problem solved, I was ready to test. Or, so I thought.
Android's many faces
I quickly learned how many different Android platform versions and different Android phones exist today. That begs the question: Which phones and, more importantly, which versions of Android do we test? I went back to my book on testing methodology for the answer. Do not fear, I have another plan.
I called William. After a few minutes I hinted at my shrewd proposition by asking "What phones do you normally test with?"
"I have two, a HTC Hero running Android 2.1 and a Nexus One running Android 2.3.4."
I suggested, "Let's use those two."
William saw where I was going and agreed. We know it's not what NIST would do, but that's not what we're about. On a good note, William explained why it's a good choice -- in particular, the Nexus One:
"The Nexus One is a ‘developer' device created by HTC in partnership with Google. It's unlocked and carrier-independent. One of several devices created with developers in mind. It's immune to OS fragmentation, mfg/carrier-added bloatware, and on a short list of devices that app developers should use to create a test baseline. All said and done, the Nexus One is generic as an Android phone can get, making it the ideal choice when it comes to any sort of testing."
I knew that.
William and I decided to answer my third conjecture as to whether "phone technology is good enough without security apps" first.
I'd like to introduce you to Android.DogoWar:
"A trojan horse on the Android platform that sends SMS texts to all contacts on the device. It is a repackaged version of a game application called Dog Wars, which can be downloaded from a third-party market and must be manually installed."
William thought this would be a good choice because it is a fairly new piece of malcode -- discovered August 15, 2011 -- and one that is least likely to permanently damage his phones.
William set about his task. All I could do was wait. Finally, William emailed me:
"I installed the infected software and shortly thereafter the Rabies service (malware package) began wreaking havoc on my phones."
Scratch my third hypothesis -- the technology is vulnerable. Now onto the last hypothesis, "Security apps are not catching malware."
Which security apps?
William and I went back and forth about which security apps to test. And, to be honest, I'm still not sure why we picked the ones we did, but here they are along with their respective sales pitch.
AVG Antivirus for Smartphones & Tablets: Detects harmful SMS and apps automatically. Anti-Virus Free is a security suite which protects your phone from viruses, malware, and exploits in real-time.
Lookout Mobile Security: Protect your phone with award-winning security & antivirus from Lookout. Get Lookout for free. Antivirus, Phone Locator, Data Backup and more.
McAfee Mobile Security: A powerful combination of McAfee VirusScan Mobile, McAfee WaveSecure, and McAfee SiteAdvisor for Android. The solution protects your mobile device if it's lost or stolen, backs up and restores your personal data, safeguards against mobile viruses and spyware, and lets you safely surf the mobile web.
Norton Mobile Security Lite: Protects your mobile device against loss, theft and malware. Norton lends its anti-malware, anti-virus, and security expertise to mobile. Your life and your important stuff are on your phone. Keep it safe with Norton Mobile Security Lite for Android.
Trend Micro Mobile Security-Personal Edition: The best protection for your digital mobile life. It protects your Android device from loss, malicious apps, and web threats.
All our chosen security apps had paid versions. We tested the free versions (AVG, Lookout, and Norton) and trial versions (McAfee and Trend Micro).
William devised several tests, starting with the malcode DogoWar -- mentioned earlier. Each test was run twice on both phones. Also, only one security app was installed on each phone at any given time. We learned early on that having multiple installed apps produced inaccurate test results.
What you see below is the description of each test, followed by a graph of how each of the security apps fared.DogoWar: When the phone is rebooted, a malicious service named com.dogbite.Doghouse is started, which starts the service com.dogbite.Rabies. This service determines all contacts that have a phone number and sends them an SMS. Then, the malcode sends a hard-coded SMS to 73822, enrolling the victim in free SMS service from People for the Ethical Treatment of Animals.
To catch this, security apps will have to pick out the rogue service hidden in the legitimate code. Current computer anti-virus software attempts this using heuristics. Smartphone security apps might detect this, but limited onboard resources make it doubtful.
Kept my promise
I sent Adrienne, our Android mentor, the above test results and she offered the following comment:
"It's possible that they (security apps) are not looking for the re-packaging of DogoWar because DogoWar is only known to have been distributed with 1 application; maybe they use more sophisticated techniques for malware that is known to have trojanized a large set of apps."
That was enough of a hint for William. He set up the next batch of tests using malcode called DroidDeluxe.DroidDeluxe: A root exploit for Android-based phones. DroidDeluxe silently roots the phone to gain access to user credentials.
Realizing that William has become quite intimate with each of the security apps, I asked him for his take on each:AVG:
- The app scan process is slow compared to the others.
- AVG does not auto-scan the SD chip when file systems are mounted.
- AVG had false positives seemingly based on package name alone.
- Detected malware only at installation.
- Failed to detect re-packaged malware.
- AVG appears to only look for malware package names
- Doesn't appear to detect threats until malcode is installed.
- Really clean user interface.
- Had some false positives.
- Did not rely on package-name recognition, as it also caught repackaged malware.
- Invasive. Requires registration for a 7 day trial, plus SMS verification to your phone and the phone number of one or more friends.
- No false positives caught all known viruses plus the repackaged malware.
- Non-intrusive setup but a little confusing.
- Only detected one of the four real threats.
- No false positives.
- Recently completely re-vamped.
- The old version is still floating around in the market and easy to download by mistake.
- Failed to catch a single threat.
Both William and I were anxious to learn what Adrienne thought of the results:
"Android malware is rare compared to desktop viruses, but it's beginning to appear. Users who download applications from unofficial markets should consider extra-security measures, like learning about what permissions mean and installing anti-virus software.
The results of your tests reflect the fact that Android malware detection is a new space, and not all anti-virus companies have caught up to the newest threats.
One challenge, from a technical perspective, is recognizing known malware that's been re-packaged with a legitimate application. We've seen mobile malware in the wild that uses this approach, so it's important to detect.
However, this is difficult because security applications need to avoid false positives; users would be unhappy if their security app flagged non-malicious applications because of a similarity heuristic. I'm aware of ongoing work being done by others in this area, so hopefully we'll see improvements over the next few months."
First, I want to mention my new-found appreciation for Android security-app developers. Between what Adrienne pointed out and all the variables surrounding Android phones and firmware, I now have an idea what they're up against.
That said, William and I feel an obligation to those who use security apps. That's why we have thrown a marker down where there was none.Special thanks
This piece would not have been possible without a great deal of help from William and Adrienne, to whom I extend my heartfelt thanks.