Like beauty, security resides in the eye of the beholder — or security is subjective.

Take for instance the various forms of digital communications that fall under the heading of messaging: chat clients, texting applications, email programs, and video calling to name a few. Developers working in digital communications often claim their products are secure, but what if their idea of security is not the same as ours?

In late 2014, the Electronic Frontier Foundation (EFF) asked a similar question. “In the face of widespread internet surveillance, we need a secure and practical means of talking to each other from our phones and computers,” mentions this EFF blog post. “Many companies offer ‘secure-messaging’ products — but are these systems actually secure?”

Seeing an opportunity to add clarity, the EFF started a project called Secure Messaging Scorecard. “Many new tools claim to protect you, but don’t include critical features like end-to-end encryption or secure deletion,” explains Peter Eckersley, EFF technology-projects director for EFF. “This scorecard gives you the facts you need to choose the right technology to send your message.”

Testing criteria

At last count, the EFF tested 40 messaging applications using the following criteria.

Is your communication encrypted in transit? To get a green check, messages from the application (not metadata such as user names and addresses) must be encrypted along the entire external communication path.

Is your communication encrypted with a key the provider doesn’t have access to? Only the two parties communicating should be able to decrypt messages.

Can you independently verify your correspondent’s identity? The EFF feels each party should be able to verify the other even if the communications channel or service provider have been compromised. EFF mentions, “For the scorecard, we simply require that a mechanism is implemented and not evaluating the usability and security of that mechanism.”

Are past communications secure if your keys are stolen? The keys used to encrypt the messages must be routinely deleted, and methodology must be in place to ensure if one or both users delete their copy of a message, they are irretrievable by a third party.

Is the code open to independent review? The EFF prefers all source code be released under a free/open source license, but to pass only requires releasing the code that affects the communication and encryption performed by the client application.

Is the crypto design well-documented? This criterion requires explaining the cryptography used and in such a way that third-party reviewers can answer:

  • Which algorithms and parameters (such as key sizes or elliptic curve groups) are used in every step of the encryption and authentication process
  • How keys are generated, stored, and exchanged between users
  • The life-cycle of keys and the process for users to change or revoke their key

Has there been an independent security audit? A security review covering design and implementation of the application must be performed by an auditing party independent of the application’s main-development team and within 12 months of the EFF evaluation.

Notable findings

The people involved with the scorecard came up with some notable results:

  • Eight of the tools — ChatSecure, CryptoCat, Pidgin, Signal/RedPhone, Silent Phone, Silent Text, Telegram, and TextSecure — received all green checks.
  • Apple’s iMessage and FaceTime products stood out as the best of the mass-market options, although neither provides complete protection against sophisticated, targeted forms of surveillance.
  • Many options — including Google, Facebook, and Apple’s email products, Yahoo’s web and mobile chat, Secret, and WhatsApp — lack the end-to-end encryption necessary to protect against disclosure by the service provider.
  • Several major messaging platforms, including QQ, Mxit, and the desktop version of Yahoo Messenger, have no encryption at all.

People at the EFF are hoping the scorecard will empower developers. Nate Cardozo, EFF staff attorney, states, “We hope the Secure Messaging Scorecard will start a race-to-the-top, spurring innovation in stronger and more usable cryptography.”

Note: TechRepublic and ZDNet are CBS Interactive properties.