Big Data presents all manner of challenges to IT leaders, from questions about who “owns” Big Data initiatives to struggles with the new and emerging technologies that facilitate near real-time analysis of large data sets. One of the last areas of concern to most new technologies is security, and it’s easy to see why this topic often gets short shrift. Much like insurance, security provides little immediate value. It’s rarely a major concern until after an incident, when the laments about insufficient preparation become apparent.

The IT industry does little to help, with security generally ignored in the trenches due to more pressing concerns, and presented with an almost laughable overdose of doom and gloom by vendors who have a vested interest in stoking fear. Like insurance, however, it’s relatively easy to price security and do an internal risk assessment to determine what level of security is appropriate for your Big Data initiatives.

The dangers of Big Data

Just this week, a major newspaper noted that a police officer with access to license plate surveillance data pled guilty to bribing people based on their vehicle’s license being recorded at various “unsavory” locations. Data security breaches are nothing new, but with Big Data there are a few new pieces to the puzzle.

Truly “big” data has an obvious increase in the size and scope of data being manipulated, often amplifying the impact of a data breach. Since Big Data promises a near real-time ability to make decisions from critical data, we’re usually dealing with sensitive information ranging from key internal sales and financial metrics, to sensitive customer data, to medical- or security-related information. Furthermore, the real-time nature of Big Data may make security seem like an overly “expensive” proposition in terms of time, but this cost must be regarded against the cost of a data breach.

Another danger to Big Data in particular is the number of hands that will likely be touching the data. Business unit staff will certainly be gathering data and developing a plan for their analysis, while the usual suspects in IT will be performing everything from data loads to tweaking database performance. Due to the complexity of Big Data analytics, there may additionally be data scientists or statisticians, some of them from outside parties, who might lack experience in handling sensitive data. With so many hands in the pudding and the compressed schedules necessitated by Big Data, it’s easy to ignore data security.

The final security risk that is somewhat unique to Big Data comes in the form of the technical tools, many of which are new offerings from trusted vendors or open source software with a life cycle measured in weeks rather than years. In all cases, these are fairly new products subject to frequent refreshes and market pressure to “ship first, secure later.” Even when the product features robust security, pressures to get it up and running in your IT environment may leave these features off or incompletely configured.

The insurance model of Big Data security

Just as no one looks forward to purchasing personal insurance until they are shocked to discover inadequate or missing coverage, an ounce of prevention is worth a pound of cure. As an IT leader, the insurance analogy is instantly recognizable and cuts through the fear-driven pitches of the security vendors.

From a pragmatic perspective, unless you’re in a highly sensitive industry it’s unlikely a nefarious gang of state-sponsored hackers is plotting its takeover of your Big Data initiative, but it is quite possible that absent some reasonable precautions, data could fall into the wrong hands through careless oversight. Look at the value of the data that are feeding your Big Data initiatives, and spend some time with corporate risk management to determine the cost to your company if that data were to be released to the public or simply “lost” through sheer carelessness. Aside from a direct financial impact, there are relatively obvious costs in damage to your organization’s reputation, or penalties and legal troubles for sensitive personal information. These costs are quite high, making a week’s time spent on data security look like a very cost-effective investment.