Excel isn't the most robust tool for big data work. Fortunately, there are spreadsheet solutions that fit the bill, and that may relieve some users' anxieties about big data.
Ad hoc reporting through spreadsheets first caught fire in the mid-1980s with the introduction of Microsoft Excel. Ever since, Excel has been a mainstay in end user data collection, analysis, and reporting, largely due to its fast learning curve, ease of use, and the ability of business users to quickly get reports without having to go to IT with requests.
Unfortunately, Excel's historic row, column, and other limitations have not made it a tool that is significantly robust for it to work with big data. Over the past few years, this has made it easy for pundits and businesses to wonder out loud if the premier "killer app" report generator, which 10 percent of enterprise jobs posted demand skills in, was on its way out the door.
Microsoft is answering the challenge with its Data Explorer for Excel 2013 product, which appears to take direct aim at big data. The product comes equipped with a mouse click-enabled dashboard that allows users to perform mashups of structured data from databases and unstructured data from sources such as Hadoop, Facebook, or the web at large, and to display this data in standard spreadsheet format or in summary visual display like bar charts and maps. This spreadsheet approach to big data cultivation hasn't been lost on other vendors.
IBM's BigSheets combines Hadoop with a web-based frontend that allows end users to perform their own mashups of structured and unstructured data from multiple sources, manifesting them on a spreadsheet.
Datameer launched an analytics solution for both structured and unstructured data that was orchestrated around spreadsheet tools in 2010, and then joined efforts with 1010 Data to provide spreadsheet analytics featuring up to a trillion rows per spreadsheet through a cloud-based service.
Google's BigQuery (PDF link) lets businesses run SQL queries against billions of rows of data in the cloud and integrates this functionality with dashboards and spreadsheets.
All of this is welcome news for enterprises and SMBs that continue to struggle with big data. A 2012 CompTIA survey revealed that only 37 percent of companies surveyed felt that they were even familiar with big data. It was also in 2012 that a Robert Half survey of 262 company finance executives showed that 64 percent of U.S. organizations with revenues exceeding $100 million per year used spreadsheets or other manual solutions for their accounting functions, with the number soaring to 74 percent for companies with annual revenues under $100 million.
If your company is struggling to find meaning in big data, yet you know that you must have it, spreadsheets can be the easy door in to practical big data application. This is because your employees already have spreadsheet skills, and now spreadsheets are being expanded for big data uses. Those who were around when spreadsheets were first used in businesses in the 1980s remember end users' welcome sighs of relief when they were finally able to free themselves from nightly report runs because they suddenly had the ability to create their own information. A similar ease of entry into big data might also come from the spreadsheet.
Will the use of a new and expanded spreadsheet work in genome analysis or in the modeling of a stellar constellation? Not likely. But if we can invoke "the 80/20 rule" and get 80 percent of companies using even 20 percent of big data, we will be well on the way to relieving the uncertainty and anxiety that surround big data today in so many organizations.