Security

Escaping the dragnet of surveillance: What the experts say about encryption

Just-released documents by the Guardian explain how intelligence agencies collude with technology companies to thwart Internet-based encryption protocols.
[09-12-2013 There is an update to this article. See below.]

To set the tone, here’s the Guardian describing what intelligence agencies are doing to overcome their biggest hindrance, “The use of ubiquitous encryption across the internet.”

[M]ethods include covert measures to ensure NSA control over setting of international encryption standards, the use of supercomputers to break encryption with "brute force", and — the most closely guarded secret of all — collaboration with technology companies and Internet service providers themselves.

Through these covert partnerships, the agencies have inserted secret vulnerabilities — known as backdoors or trapdoors — into commercial encryption software.

It’s like a Sherlock Holmes mystery, with each new release of intelligence-agency documents providing another clue as to how intensely citizens are being surveilled. We even have our modern-day digital detectives, who help interpret these clues.

poul_henning.png
Poul-Henning Kamp
With that, I’d like to introduce our first sleuth, Poul-Henning Kamp, a Unix guru who is synonymous with the FreeBSD project. Poul-Henning wrote an eye-opening article for ACM titled, "More Encryption is not the Solution." Poul-Henning shakes things up right away by offering the following prediction:

“The recent exposure of the dragnet-style surveillance of Internet traffic has provoked a number of responses that are variations of the general formula, ‘More encryption is the solution.’ This is not the case. In fact, more encryption will probably only make the privacy crisis worse than it already is.”

Poul-Henning then offers three “Inconvenient Facts about Privacy” as explanation why encryption does not ensure privacy:

  • Inconvenient fact number one: Politics trumps cryptography. Nation-states offer their citizens a choice, unlock encrypted files or go to jail.
  • Inconvenient fact number two: Not everybody has the right to privacy. For example, in most nation-states: prisoners are only allowed private communications with their attorneys; employees give up large chunks of privacy as part of their employment agreement; and finally, most citizens are now witnessing the loss of their privacy through judicial oversight.
  • Inconvenient fact number three: Encryption will be broken if need be. If a nation-state determines that someone should not have any privacy, it will do everything possible to make it so.

When I started this article, I intended to devote the entire piece to Poul-Henning’s ACM paper and how he builds a case for his “Inconvenient Facts.” That all changed two days ago, when the Guardian released new documents proving Poul-Henning correct.

My reporter curiosity had me wondering, so I asked Poul-Henning if he knew about these particular documents before they were made public: “No, I simply looked at the plausible NSA budget (that was also before the "black budget" was released) and thought about how I would use the money if I were in charge of NSA.”

As you will see in a bit, Poul-Henning was scary accurate.

bruce_schneier.jpg
Bruce Schneier
This brings us to our next digital detective: fellow Minnesotan, author, and world-renowned security expert, Bruce Schneier. It was Bruce’s article, "NSA surveillance: A guide to staying secure," that alerted me to the latest document release by the Guardian.

Bruce starts out by mentioning he’s been working with the people at the Guardian for several weeks now, sifting through hundreds of agency documents. This gave Bruce valuable insight into what intelligence agencies have managed to assemble:

“The primary way the NSA eavesdrops on Internet communications is in the network. That's where their capabilities best scale. They have invested in enormous programs to automatically collect and analyze network traffic.”

Each time I read the papers and the Guardian articles, I come to the same conclusion, intelligence agencies have the ability to compromise everything digital. Bruce offers his blunt assessment:

“These are hacker tools designed by hackers with an essentially unlimited budget. What I took away from reading the Snowden documents was that if the NSA wants in to your computer, it's in. Period.”

Encryption, NSA-style

The Guardian, Poul-Henning, and Bruce all mention that major encryption processes are compromised, but I didn’t understand how intelligence agencies could subvert something like HTTPS. Poul-Henning explains one way:

With expenditures of this scale, there are a whole host of things one could buy to weaken encryption. I would contact providers of popular cloud and ‘whatever-as-service’ providers, and make them an offer they couldn't refuse: on all HTTPS connections out of the country, the symmetric key cannot be random; it must come from a dictionary of 100 million random-looking keys that I provide. The key from the other side? Slip that in there somewhere, and I can find it (encrypted in a Set-Cookie header?).

If I understand, this means the process itself is not flawed. The key randomness is reduced, allowing those with powerful processing capabilities to easily crunch through the possible keys. Bruce verified Poul-Henning:

Basically, the NSA asks companies to subtly change their products in undetectable ways: making the random number generator less random, leaking the key somehow, adding a common exponent to a public-key exchange protocol, and so on.

Our options

If you remember, Bruce’s article was titled “A guide to staying secure.” So, Bruce must have some options for us:

1. Hide in the network: Whenever possible use services like Tor; doing so increases the surveillance effort markedly.

2. Encrypt your communications: It’s true, intelligence agencies target encrypted traffic, but any encryption is still better than sending traffic in the clear.

3. Assume your computer can be compromised: This is the tough one. Bruce suggests we create files and encrypt them on a computer that has never been attached to the Internet. Then using a flash drive, transfer the encrypted files to an Internet-facing computer for delivery. Decryption would be the exact opposite.

4. Be suspicious of commercial encryption software especially from large vendors: The secret agreements between intelligence agencies and technology companies extends to those developing security and encryption software. We should assume that every commercial application has an NSA-friendly back door.

5. Try to use public-domain encryption that has to be compatible with other implementations, which means:

  • Do not use proprietary software, back doors are easier to hide in proprietary software.
  • Use encryption applications employing symmetric cryptography instead of public-key cryptography.
  • Use encryption applications that are conventional discrete-log based, not elliptic-curve systems.

The advice I’m getting from Bruce and other experts is to make decoding our Internet traffic as difficult as possible. That way targeting us will not be worth the time and effort. Bruce concludes his article by saying:

Trust the math. Encryption is your friend. Use it well, and do your best to ensure that nothing can compromise it. That's how you can remain secure even in the face of the NSA.

The hard part will be figuring out what encryption process has not been compromised.

[Update: 12 Sep 2013]

Back in 2007, Bruce Schneier in this blog post raised a flag that NIST Special Publication 800-90, a document detailing a new random number generator being added to an NIST encryption standard was suspicious. Here is what Bruce said: 

It's in the standard only because it's been championed by the NSA, which first proposed it years ago in a related standardization project at the American National Standards Institute. The NSA has always been intimately involved in U.S. cryptography standards -- it is, after all, expert in making and breaking secret codes. So the agency's participation in the NIST  standard is not sinister in itself. It's only when you look under the hood at the NSA's contribution that questions arise.

The NIST has re-opened the public review period of this standard. It seems we need to listen to our security experts.

Final thoughts

The ethics and legality of what intelligence agencies are doing is debatable. What concerns me even more is if — more likely, when — the bad guys figure this stuff out. They’re not going to spend time debating ethics; they’ll use the built-in weaknesses for their purposes without a second thought.

Could it be that obscurity is the new security?


About Michael Kassner

Michael Kassner is currently a systems manager for an international company. Together with his son, he runs MKassner Net, a small IT publication consultancy.

23 comments
jkameleon
jkameleon

MHE

I have a regular job with a financial service company (for now). I don't know the details about its security, and I don't need to know. My job is to maintain the core application. Financial sector in my country is pretty much in the toilet (where it belongs IMHO), so I took an afternoon job with technology company, just in case. Throughout the 1980s and until the mid 1990s I used to make a living by developing embedded software. From the software point of view, microprocessors of that era are similar to today's low cost/low power microcontrollers. A nice chance to put my old skills to good use.

My afternoon employer's reaction to snooping was to give a secbox of its own design to every employee and contractor. We were told, that we can make internal phone calls and access company's systems only through it, period, end of story. Webmail, wikis and sofware repositories are no longer accessible through the standard https. That means one more gadget and more cables crammed in the laptop bag, and no more WiFi. Secbox only has ethernet connections. Very inconvenient. Thanks, NSA!

If you look past the usual Windows/Mac PCs and smartphones, uncompromised computer is quite achievable. There are tens of MCU manufacturers all around the world, there is huge variety of different models to choose from, many of which pack a considerable processing power. The cheapest developement board I came across so far was about 20€. Once programmed, MCU FLASH can be physically locked, which pretty much eliminates the malware problem. The only way of compromising embedded software is to break into developer's PC, and knowing exactly what he's doing. Developing MCU software is relatively easy nowadays, peripherals are much more software friedly than they used to be, and you have NetBSD or Linux ported on many of them. MCU based secure devices running open source software might therefore be a viable option number 6.

anil_g
anil_g

There's a lot of hysteria about state inspection of traffic. Kamp's "inconvenient fact" number one comment is such an example: "Politics trumps cryptography. Nation-states offer their citizens a choice, unlock encrypted files or go to jail."

Yeah, sure, but that alters the scale of inspection by such a massive amount that the strategic advantage is eliminated. Compare ability to automatically inspect all encrypted traffic as it transmits with ability to inspect about a dozen transmissions after several weeks legal process.

Schneier looks like he's still in the real world. He says encryption does help (contrast with Kamp) and genuinely explores how an agency may look at defeating encryption. No-one says this is happening, it's just his theory of how they could start attempting this difficult task.

Every (competent) organisation knows that security is a trade off between useability, security and cost. I'd like to question the hysterics on why they need to put so much effort into protecting their activities from inspection when we already have strong encryption methods available. Some comments read like collaboration between terrorists.


dpcrn
dpcrn

So does this make the claims by TrueCrypt and others invalid?  I.e., they say that the government has not been able to crack their encryption.  Granted, it's not a "commercial encryption product".  It's open source and they claim it has been examined by many people for back doors.

hlhowell
hlhowell

The encryption bypassing code can be inserted into the compiler.  It is a process that was described a long time ago.  Examining the source code wouldn't help you know if you were compromised or not.  Also moving files by a portable device is also suspect, because they are often loaded with back doors as well, and sometimes virus code.

I have no idea how to over come this, other than by writing your own assembly code to do the encryption on an embedded processor, where you can disassemble the code after creation to make sure there is no "back door".  Or you can write your own assembler, macro facility and loader in assembly code.  Don't publish it, then use it to create your own encryption code based on some standard but using a numerically robust key generation scheme that is not posted on line.  A good random number source is something like the fuzz in an AM radio tuned to a blank channel.  But remember that it, too is only psuedo random due to the bandwidth limitation.
JCitizen
JCitizen

So I guess this means that plugin for Thunderbird on encrypted email is probably a waste of time? Figures! I should think Open Sourch VPN could help, although you would have to assume one or the other end point was comprimised.

SkyNET32
SkyNET32

Perfect Forward Secrecy is another way that would thwart the collecting of old private keys.  Even if the NSA orders companies' random number generators to be less random, the key is per session, so after a client is finished, even if the NSA has snatched that session, they cant use it in the future.. But web servers need to implement it more.. I only know for sure that Google does..

pgit
pgit

Would any of your sources care to name a few trustworthy encryption schemes? And how does Linux (or BSD) stack up in this mess?


btw since the new changes here at the TR site I can't log in or post using firefox. (I'm using Linux) I have to load a page like this in chromium to be able to participate.  ...and I too safe in firefox?  :D

flhtc
flhtc

A much easier method is... Don't put/transfer anything on/over the Internet that you would not put on a sign in your front yard.

If you wanted to be really sneeky.  Don't use encryption.  Just come up with a rotating number array, say nested 3 deep (array[a][b][c]), then add a key array so your array subsets constantly change.  Place key words, or even just letters at those intervals in plain text emails.  It may take several emails to get the message across, but if they're concentrating on encrypted email, they may never know.  I did something like this for a "random" key generator similar to the MS license keys.  Just  don't send the key array over the Internet. <=^)

lehnerus2000
lehnerus2000

The reality is that cyber-criminals could help all of us out with this problem.

Instead of sending a billion unencrypted spam emails each day, they could send 50% encrytped with the maximum level of encryption (the level that is illegal to use).

This would force the security agencies to stockpile ridiculous amounts of data.

Michael Kassner
Michael Kassner

@anil_g 

You are forgetting one important fact. Poul-Henning did not have access to the documents that Bruce did. Poul-Henning's article came out weeks before the last release about encryption and secret agreements. 

Michael Kassner
Michael Kassner

@dpcrn 

The reality is that the encryption software has not been cracked. What's happening is organizations are limiting the number of encryption keys that are used, reducing the the randomness, which makes it easier for heavy-duty computers to run through the possible options rather quickly. That is why Bruce suggests using symmetric encryption rather than public key. 

That is my take, and please remember I am no cryptographer.

Michael Kassner
Michael Kassner

@JCitizen 

I think so, J. It's hard to believe they are that embedded, but when you are spending 300 million a year on agreements with companies and individuals, you get results. 

Michael Kassner
Michael Kassner

@SkyNET32 

That is correct. It seems ironic that the company people were worried about the most appears to be the only one trying to preserve our privacy.

flhtc
flhtc

@pgit  

Are you using NoScript and/or DoNotTrackMe?  I too use Linux and had to disable both in order to post.

Seems a little counter intuitive having to disable security and privacy addons to post in a security forum about privacy issues.  Hmm.

JCitizen
JCitizen

@pgit I here ya there pgit! I couldn't even get IE-9 to work until I complained to the feedback console at TR; they never communicate whether they are even trying to fix Chrome or Mozilla issues! I'm beginning to think CBS wants to marginalize TR for their inferior ZDNet and CNET products. It is starting to piss me off!  >:(

Michael Kassner
Michael Kassner

@pgit 

I have display issues in Chrome. 

As for Linux, I have read that all operating systems are suspect. That is implied in the actual documents. If I understand, they have flipped developers who are good enough to add weaknesses discreetly in open-source system code. I have not been able to second source that though. 

Michael Kassner
Michael Kassner

@flhtc 

Not sure if that works or not. With their capabilities, everything is suspect. As long as companies are working with the intelligence agencies, every digital device, every piece of software, and every network (voice or data) is suspect. 

Michael Kassner
Michael Kassner

@lehnerus2000 

I think the bad guys are happy the way it is now, and are working feverishly to figure out the installed weaknesses so they can use them as well. 

SkyNET32
SkyNET32

Yeah, Google may not be perfect but their disclosure is a bit better than other tech companies..imo

simonschilder
simonschilder

@flhtc @pgit I am using Ghostery on Chrome and I had to disable it fot TR in order to post something :(


flhtc
flhtc

@Michael Kassner @flhtc  

The point really isn't whether or not it works.  It was merely that it's usually harder to spot something hiding in plain site.  Kinda like rice.  You have a bowl full of white rice, then throw in a handful of brown.  They stand out like encrypted email. 

Editor's Picks

IT Buying Cycle

Learn more