Part of this challenge rests in how to develop queries that can effectively probe big data for ground-breaking answers. However, an equally looming challenge rests in how to create effective reports that present answers to business users in formats that they can immediately apply to the business or manipulate for even greater value.
Developing effective reports is an old battle that has confronted businesses since the dawn of information, let alone the beginning of the computer age. Most organizations today (as in the past) find themselves falling within the "80-20" rule, because they are usually obtaining all of their information from only 20 percent of the reports they have developed, leaving the rest of their reports to sit on the shelf.
Big data reporting is no different. Once the analytics have been run against raw data, there have to be effective reporting mechanisms that give users actionable information. Within most organizations, two types of reporting mechanisms are needed: the bottom line reports that inform C-level executives and high ranking managers about various aspects of company performance, and the more drill-down reporting that gives line managers and staff both information and tools that they need to further troubleshoot situations. In both cases, users of reports want ease of use and minimal learning curves.
This invariably raises the question about whether dashboards or spreadsheets are better for big data reporting.
This is what we knowAutomated dashboards that monitor a system's performance, or that even monitor elements like online sales as they occur during an Internet marketing campaign, can flash a "green," "yellow" or "red" light to business and IT users signifying whether the item being measured is going well, proceeding in a cautionary state, or is in a red "halt" state. For many upper level executives and managers, this is all they need to take action. If there is a problem, their next step is likely to call down to the next level of management to troubleshoot the situation.
If you are a manager or staff member at the middle or lower levels of the organization, it is most likely you will use spreadsheet reporting and tools to analyze (or even to re-query) the results you have received from big data analytics. Spreadsheets are flexible software. They allow you to select on another data field for a sort, or to program computations or macros. With these analytics "extenders," you can continue to exploit the value of the analytics as you drill deeper with your own questions.
Spreadsheet reporting continues to hold its own in companies, with most organizations possessing "power users" who know everything there is to know about spreadsheet data manipulation. This is also a major reason why big data solution providers have continued to employ the concept of a Microsoft Excel-style spreadsheet, and have even expanded on the spreadsheets by creating new spreadsheet tools and capabilities that are capable of working with big data.
Let's look at an example of customer data that has been compiled from records of past buying patterns.
Initially, an end user in marketing might be able to see who the top thousand most profitable customers are, based on volume. But the ability to further model big data analytics that are delivered to a spreadsheet might also enable that same user to sort on product mixes or customer demographic factors. Based on what this person uncovers, he will also have something of value to report back to upper management that can potentially improve corporate performance.
Big data solution providers understand this dynamic between the "need to know" at the top level dashboard level and the need to "drill down" and manipulate at the spreadsheet level. They also know that organizations will more aggressively adopt their solutions if the solutions come with tools that are easy to use and already familiar. Dashboards and spreadsheets meet these requirements - potentially removing some of the unknown factors companies face as they get their feet wet with big data.
Big data is transitioning from one of the most hyped and anticipated tech trends of recent years into one of the biggest challenges that IT is now trying to wrestle and harness. We examine the technologies and best practices for taking advantage of big data and provide a look at organizations that are putting it to good use.
Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President of Product Research and Software Development for Summit Information Systems, a computer software company; and Vice President of Strategic Planning and Technology at FSI International, a multinational manufacturing company in the semiconductor industry. Mary is a keynote speaker and has more than 1,000 articles, research studies, and technology publications in print.