Google Play: Android's Bouncer can be pwned

Deciding whether to trust apps or not just got more complicated. Michael Kassner asked a pair of researchers why that is.

I can't believe my good fortune. I'm interviewing two Pwn2Own legends about their latest tag-team exploit; owning Android's anti-malware system -- Bouncer.

What is Bouncer?

When Google first introduced their app store -- Android Market -- in 2008, apps weren't tested for malicious content. Fellow TechRepublic writer William Francis and I verified that in this article about Android Permissions.

Google rectified their oversight this past February with the introduction of Bouncer:

"Today we're revealing a service codenamed Bouncer, which provides automated scanning of Android Market for potentially malicious software without disrupting the user experience of Android Market or requiring developers to go through an application approval process."

So that's it, no more malicious apps in their app store. Or is it....

Who are these legends?

Dr. Charlie Miller and Dr. Jon Oberheide, that's who.

I'll start with Charlie. Normally, it's enough to say someone worked for the National Security Agency, but he was just getting started. I mentioned Pwn2Own earlier; Charlie has won at Pwn2Own four years in a row.

My favorite "Miller" exploit surfaced last year. Charlie planted a "sleeper app" in Apple's App Store. The app exploited a vulnerability that allowed Charlie to send "other than Apple-approved" commands to devices that downloaded his app. Charlie alerted Apple about the vulnerability ahead of his going public with it at Pwn2Own, but neglected to mention the sleeper app. Apple didn't take kindly to his omission, revoking his developer's license.

Next is Jon. He's the serious one on the left. Currently, Jon is co-founder and CTO of Duo Security. The company provides two-factor authentication as a service. When Jon is not working at Duo Security or with Charlie, he's sneaking around trying to find exploits.

I remember last year's Pwn2Own -- Jon probably would have won. But rather than save his Android exploit for the contest, Jon reported it to Google, netting himself $1337 US. That's nice, until you consider Jon could have won $15,000 US if he had waited. I'll let Jon explain.

So what's with Bouncer?

Charlie and Jon are able to bypass Bouncer and slip malicious apps into Google Play. (Note: I don't know about you, but I'm confused as to whether it's Google Play or Google Play Store. For now, let's agree to use Play, even though on my phone, it's Play Store.)

Charlie and Jon presented Dissecting Android's Bouncer (Warning: Charlie is exposing more than code in the pdf.) at SummerCon 2012. After reading the transcript, I contacted my heroes to see if they would agree to an interview. And they were.

Kassner: Hello, Jon. Thanks for helping me understand what you and Charlie accomplished. Why did you decide to look at Bouncer in the first place? Oberheide: Literally, a couple days after the public announcement of Bouncer, I decided remote blackbox analysis of Bouncer would make interesting research. In a previous life, I worked on malware analysis, both writing and breaking sandboxes, so this was an area relevant to both my past and current interests.

Charlie and I are always bouncing mobile stuff back and forth. So we decided this would be a fun project to work on together and present at SummerCon.

Kassner: If I understand correctly, you are the first ones not associated with Google to figure out how Bouncer works. Can you give us an overview of what you found? Oberheide: We started with a completely blackbox perspective, knowing nothing about how the Bouncer system operated. We then created a list of questions we hoped to answer about how Bouncer works and what potential weaknesses are present.

If I were to sum up our discoveries, I'd claim that Bouncer is currently in a 1.0 state. Which is fair, given it was only recently publicly announced. We discovered several ways to fingerprint the Bouncer environment. The most interesting ones would allow a malicious app to hide its true intentions while running with Bouncer, and still perform malicious activity on a real user's device.

So, there's a lot Google needs to do to address Bouncer's weaknesses and advance its capabilities. That being said, I wouldn't be surprised if Bouncer is fairly effective at cleaning up the current generation of mobile malware, which tends to be unsophisticated.

However, as mobile devices become more enticing targets and attackers find it economically worthwhile to put effort into targeting mobile users; we'll see an increase of sophistication to sneak past Bouncer. If there's anything we've learned from traditional malware, it's that attackers will adapt quickly, and they're quite efficient at sharing (or stealing) techniques.

Kassner: I'm curious as to why Bouncer allows the app under test to access the Internet. Couldn't Bouncer fake it? That would prevent you from communicating with Bouncer. Right? Oberheide: Absolutely, there are a number of networking approaches one can take when designing a dynamic-analysis system: blocking, filtering, emulating (aka "faking" like you mention), and unrestricted network access are examples. Problem being, each approach has its pros and cons.

If the designers of Bouncer had decided to fake network access, it would have been difficult to do in such a way that is indistinguishable from real network access. But you're right; the available network access was useful for leaking information to our command and control servers.

Kassner: I read you were playing cat-and-mouse with a real person and not just the automated Bouncer process. How did you determine that? Did that person shut you down? Oberheide: During some of our more aggressive experiments, we noticed periodic "callbacks" -- the app phoning home to us from within Bouncer -- coming from a different Google IP range. And the callbacks exhibited request patterns and probes that indicated a human operator was remotely analyzing the app.

We hypothesized this was part of the overall Bouncer workflow. That is, if an app is flagged by the automated dynamic analysis of Bouncer, it would trigger a manual review by a person.

One time when I submitted an app that opened a remote shell into Bouncer, I didn't catch the warning signs I mentioned earlier and proceeded to send commands to the shell. When that didn't work, I had an idea why. So I tried to send the person on the other end a few friendly messages.

Kassner: Going through the presentation, I noticed this slide.

"We can submit apps without paying!"

I also read you've informed Google about your findings. So how you were able to submit apps without paying?

Oberheide: One of our main concerns during our Bouncer experimentation was to keep real users out of harm's way when we submitted our apps (our Hippocratic Oath: Do no harm).

Thankfully, we discovered Bouncer would analyze the submitted app after it was saved in the Play publisher interface, but before it was made available to the public. So we could perform our experiments with Bouncer without affecting users.

The icing on the cake was our ability to upload applications and have them analyzed by Bouncer, even if Google Play denied our payment. Obviously, you can't publish apps if you don't pay, but we didn't need to go that far, this was a boon for our research.

Google should address this issue. Attackers could probe Bouncer without having to use a legitimate credit card number.

Kassner: I've read Apple validates apps through static analysis (checking object code) and through your efforts, we know Google is using dynamic analysis (running the app). Which is better? Oberheide: I've never seen any evidence published that Apple performs any security review of applications, dynamic or static, so I can't comment authoritatively on that. It appears that Apple may use static analysis to check for the use of private APIs, but that's hardly indicative of whether an app is malicious or not.

Generally, there are pros and cons of both static and dynamic analysis, depending on the threat model and characteristics of the target platform. For example, static analysis would be a bit easier for Apple, since they perform code signing. Code signing gives them higher confidence that the code submitted along with the app is the only code that will be executed in the app's lifetime. It's very difficult for Google to perform static analysis since Android allows additional code to be pulled down at runtime changing the behavior of an application.

More importantly, static and dynamic analyses are not mutually exclusive. Any good system tasked with analyzing potentially malicious software will employ a combination of both techniques.

Kassner: In the presentation, you mention there's an individual's Google account associated with Bouncer:

Google account associated with the Bouncer device:

  • base64.b64decode('OyBtaWxlcy5rYXJsc29uQGdtYWlsLmNvbSwgY29tLmdvb2
  • dsZQ==')';,'

And, you found interesting tidbits about this person, such as the contact list of the account holder. Why store personal information in Bouncer?

Oberheide: In order to make Bouncer effective, it must be indistinguishable from a real user's mobile device. Otherwise, a malicious application will be able to determine it is running with Bouncer and not execute its malicious payload.

As you can imagine, creating a fake device that appears to be real is a difficult problem. A real mobile device has a variety of data sources (phone book, SMS messages, photos, music, third-party applications, etc.) and sensors (microphone, camera, GPS, accelerometer, etc), many of which are difficult to fake in a believable and scalable manner.

Kassner: As I read the presentation, I noticed that you were keeping track of all the ways a malicious app developer could deceive Bouncer.

Bouncer runs your app for 5 minutes

  • Don't do anything bad for 5 minutes! Duh.
  • Not long term. Could be run later, longer...

A while back, I wrote a piece about Bouncer and how people were finding ways to circumvent it. From what you know now, should we be concerned about the apps in Google Play? What safety precautions would you suggest we take?

Oberheide: Yep, I had the same thoughts (regarding circumvention) when it announced. The RootSmart malware Dr. Jiang discovered was based on the RootStrap approach I had proposed two years ago at SummerCon 2010.

It took malware authors almost two years to adopt that technique...but then again, there wasn't much "evolutionary pressure" for them to do so until Bouncer came around.

Users should always be cautious about downloading suspicious apps from an app store:

  • Regardless of the efficacy of Bouncer.
  • Regardless of what mobile platform they're using.
  • Regardless if "antivirus" is installed on their mobile device.

As we've seen in the desktop-computing world; no malware protection, whether gated at the app store or performed on-device, will be entirely effective. That said, it's still worthwhile for Google to block as much malware upfront as possible, and I'm sure they're hard at work revising Bouncer to address the weaknesses we discovered.

Final thoughts

Charlie and Jon are best of the best when it comes to finding vulnerabilities. But I fear others may not be far behind. Let's hope Google is hard at work removing this vulnerability.

Jon wanted to make sure I attributed the awesome high-def slides in this article to Busticati Productions. (I think we've been busticated.)