A recent webcast from ZDNet and CBS Interactive focused on big data, and how the growth in the sheer volume of data collected has led to a data explosion throughout the world. The topic was so popular that the panelists didn’t have time to address every question asked via Twitter during the webcast. So, we’ve taken five of the top questions that remained, and asked the panelists to respond.

Find out what our experts think about the following big data topics:

  • Defining big data
  • First step when considering big data
  • Advantage of using big data with BI
  • Unstructured versus structured data
  • Role of social media

The experts are David Boyle, senior vice president of insight for EMI Music; Ken Wincko, senior director of marketing for Dun & Bradstreet, which sponsored the webcast; and Hilary Mason, chief scientist for bitly. Mason was unable to participate in the webcast as scheduled, but shared her thoughts here.

Watch TechRepublic’s and ZDNet’s Big Data Priorities 2013 on-demand webcast, sponsored by Dun & Bradstreet.

Defining big data

There were varied opinions from the experts on whether there is a standard definition of big data.

Boyle: Yes, but the definition isn’t very important to me when thinking about insight. If you’re trying to get insight from big data you typically need to work with only a small subset of it at any point in time. You might then need to run your analysis against the whole data set once you’re done, but a small, representative set of the data will work just fine for working out what the insight actually is.
Wincko: No. In fact, there are a multitude of definitions depending on whom you are speaking to.  Some people think of large data sets and data storage, while others think of business intelligence and analytics, among other variations.

Traditionally, big data describes data that’s too large for existing systems to process.  And to a certain extent that is true.  We can all agree that the volume, variety and velocity of information continues to expand at an exponential rate, and the speed at which it is being distributed is also accelerating.  These are the characteristics I would use to define big data.

Mason: The standard technical definition of big data is data too big or complex to analyze on one machine. This requires a specialized set of tools and infrastructure to translate that data into useful answers to questions.

First step when considering big data

Planning is key when thinking about launching a big data project. Consider what the goals are, and what key data sets are needed to reach those goals.

Boyle: The first step absolutely has to be to think about what you want to achieve. Specifically what useful business question do you have that you think the big data can actually solve for you?
Mason: First, step back and think about the kinds of questions you plan to ask. What aspect of the business can you optimize with better insight into the data? Which parts of your products can you improve with insight into your customer behavior? This is the best point to bring in a data scientist who can help with the process of translating business constraints into data analysis. Then, do an audit of your available data, and see what technical systems are required to build the kinds of systems you need.
Wincko: The important thing to remember is that it is impossible for companies to cost effectively capture and process all of the information being produced.  So, the critical questions businesses need to ask themselves when dealing with big data are:

  1. What are the key business drivers we are trying to influence?
  2. What are the key data sets that we need to aggregate, analyze and distribute to help us get to where we want to go?
  3. How can we use existing technologies, data sources and tools, analytics and processes to effectively and efficiently manage it all?

Advantage of using big data with BI

Companies already using Business Intelligence (BI) for decision making are already experienced in collecting data for analysis. Big data can give an advantage by adding real time analytics to the offline BI reports.

Wincko: Business Intelligence solutions are great for helping companies to gain meaningful knowledge from historical, current and some predictive views of business operations to make better decisions.

BI uses technologies, processes, and applications to analyze mostly internal, structured data and business processes.  Big data capabilities can help your company to make even smarter business decisions by providing additional context and the foresight you need to gain a view into previously unforeseen market opportunities.  Some examples include additional vertical-specific information from third-party data providers, social and mobile information and other digital sources.

Big data can help your company develop a complete and informed view of prospects, customers, competitors, suppliers and partners – your whole ecosystem – through the relevant data and analytics that results in actionable insights, so that you can anticipate market opportunities – and capture them.

From a financial perspective, if you are able to gain a complete financial picture of your customers, suppliers and partners, you will be able to know your best move as it relates to: better managing cash flow, reducing financial risk, making better investment decisions, conducting better diligence on M&A targets, building the best and most reliable supplier network, eliminating compliance issues and related fines, and finding the best new partners.

From a sales and marketing perspective, you can utilize big data to derive unique insights into current and future customer behavior and transform that insight into a form of intellectual property and sustainable competitive advantage.   You can utilize these insights to anticipate current and future customer needs, identify new markets, deliver innovative solutions, and provide superior customer service – to get there first – ahead of the competition.

Boyle: If you already have good decision-making using small data, then there seems like little opportunity to help no matter how big the data is. It depends how good your decision-making is and whether there is a business need for better decision making. And then it depends if you have a set of big data that you can use and that can help in the areas where you need it.
Mason: BI is one set of practices for using data to make better decisions. Modern data science allows us to expand those capabilities in several ways, including moving from offline BI reports to realtime analytics, from studying patterns in large cohorts to online personal optimization, and more.

Unstructured versus structured data

During the webcast, Boyle talked about dealing with structured versus unstructured data. He explained to us how EMI turns what could be unstructured data, such as from conversations, into structured data by using online, structured interviews to capture the data.

Meanwhile, Wincko discussed how Dun & Bradstreet handles it from a B2B perspective.

Wincko: Multi-sourcing of data, especially unstructured data such as social media and news is critical to identifying and anticipating needs – but you definitely need a way to effectively connect it all together and verify the accuracy of the information you are capturing.

In the digital age it’s much easier to create online and social personas – so it’s perpetual truth checking – in addition to finding the additional context you need to know.  And that requires a lot of work.  You need to be able to answer questions such as: Who are they?  Where are they? What do I know about them and their company?  Have we done business with them before?  How can we address their needs?

Here are some best practices companies should consider:

First, you need a referential database to check the validity of the data, prevent duplicates, and address gaps.  While no source of data is perfect all of the time, you want to be able to develop a trustworthy, actionable foundation of company and contact records to get that “single source of the truth”.  From there, you can then connect other types of data, such as social media profiles, to those company and contact records, to give you the complete and relevant insight that you need.

Second, you need a unique company identifier to be able to attach the relevant insight to those company records – for D&B, the DUNS Number is that company identifier – you can kind of think of it as a social security number for a business.  An identifier would be able to help you connect a company, to a person, to a social profile and so forth.

Finally, you need to be able to automate the aggregation and dissemination of data across systems and incorporate pre-built analytics with data where you can.  A very cost-effective and efficient way to do this is to utilize data-as-a-service, which utilizes API web services to connect and deliver disparate data and analytics into any system real-time.  Utilizing data-as-a-service can help companies to improve both the quality and speed of decision-making, while streamlining processes and reducing data management complexities.

Role of social media

Social media can be analyzed to understand behavior, although it’s important to remember that it only provides a specialized subset of opinions on any particular topic.

Mason: My work is in the realm of analyzing social media! We look at the links shared and clicked via bitly to understand aggregate human behavior. We’re able to learn things like the half-life of links shared on Twitter is 2.8 hours, sports stories spread the fastest of any topics, and there’s a big difference between what people share and what they click.
Boyle: It depends on the business questions you’re asking. If you want to know what a tiny and specialized subset of the world thinks about something, look at Twitter data. If you also want to know what the vast majority of people who don’t comment about your product on Twitter think, then it’s not very helpful at all.
Wincko: In the social age we are now in, companies are dealing with buyers who are more knowledgeable and sophisticated than ever before, which has fundamentally changed the buying process and shifted power to the buyers.  You combine that with the proliferation of touchpoints across social, mobile and other traditional channels with prospects and customers, and you have a situation where marketing, sales and service teams are struggling to effectively connect with and confidently engage with prospects and customers across these interactions.

From a sales and marketing perspective, you can use big data to better leverage social channels to rapidly detect changes in buyer activities and preferences, identify market shifts, monitor competitive activity, and more to identify the best new opportunities for your business.  Data and insights can help by finding the right potential customer at the right time through the right channel with the right communication.

Actionable insight can help you build a 360-degree view of contacts and companies which result in an effective buyer engagement hub connected across channels.  For example, you can combine your own customer data with third-party data, with buyer sentiment from social media activity, web activity and transactional data to better anticipate potential buyer needs.  You can then use this foresight to target new customers more effectively through customer modeling, drive higher cross-sell and up-sell with existing customers through knowledge of corporate linkages, and proactively engage with current customers to increase retention through better customer service.