If 2017 has proven anything, it is that big data and analytics projects are no longer thought of as emerging and experimental, with companies patiently waiting for results. With the honeymoon period over, the tasks for CDOs and CIOs in 2018 will be to work with big data and analytics as mature technologies that are an inherent part of their companies' IT.
Executives in charge of big data and analytics can make the transition by focusing on these five key areas in 2018.
SEE: Data classification policy (Tech Pro Research)
The amount of data companies are storing is growing fast, and big data is a major contributor. In the rush to install big data and analytics systems, many IT departments have not paid much attention to the maintenance side of these systems—such as where the data ultimately gets stored. In many cases, companies have opted to keep all their data, fearful of coming up short if they are subjected to eDiscovery or regulatory requirements—but the result has been soaring storage costs. Companies have also made little progress on defining which data is important, which data is sensitive, and which data is useless and can be discarded.
In 2018, CDOs and CIOs can tackle this problem by meeting with legal counsel, regulators and business units to identify which data should be kept, and which can be discarded. They can then design policies for data retention, and they can begin to assign data to appropriate storage devices based upon how often data is used. Storage is better utilized, because only your frequently accessed data needs to be on expensive solid state storage. Data is that is seldom or never used can be on cheap cold storage disk.
SEE: Hiring kit: MongoDB administrator (Tech Pro Research)
Hand-in-hand with the organization of storage is the overarching data architecture for big data and analytics. IT has two challenges in this area. First, data should be defined and distributed to smaller data marts that individual departments can use, which reduces access contention on a single repository. Second is maintaining a central, master repository that constantly refreshes the data distributed to data marts so that all departments are using the same up-to-date information.
Data scientists and analysts need to be redeployed so that they are no longer the ones developing analytics queries and reports, and are instead deployed to assist end users with user queries. This is the only way that organizations will be able to leverage their big data and analytics investments so that everyday users can use them. What the data scientists and analysts need to do is to move the data into a state where it can be accessed and manipulated by end tools that users are already familiar with—such as an Excel spreadsheet or a third-party reporting tool like Tableau.
The test of a big data project is integration into the company's day-to-day operation or strategic evaluations. No big data project should be undertaken without a final goal of company adoption because if the project is rejected by the company, it fails.
SEE: Turning big data into business insights (free PDF) (ZDNet/TechRepublic special report)
5. Analytics as everyday business
In the end, the goal is not only company adoption of big data and analytics in business processes, but acceptance of big data and analytics as a way of doing business. Companies are getting there. They are using geospatial data and linking it with demographics to come up with profiles of populations in applications like marketing, healthcare and metro area management. Transporters and food and beverage companies are using sensors, GPS and analytics successfully to track perishable foods from farm to table. More use cases will follow, because 2018 will be the year when CDOs and CIOs will be expected to show acceptance by adoption of big data and analytics technologies.
- 6 big data privacy practices every company should adopt in 2018 (TechRepublic)
- Predictive analytics: The smart person's guide (TechRepublic)
- Why data science is a secret weapon for tech consultants (TechRepublic)
- Big Data 2018: Cloud storage becomes the de facto data lake (ZDNet)
- Big data case study: How UPS is using analytics to improve performance (ZDNet)
- Five organizations that are using big data to power digital transformation (ZDNet)
Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President of Product Research and Software Development for Summit Information Systems, a computer software company; and Vice President of Strategic Planning and Technology at FSI International, a multinational manufacturing company in the semiconductor industry. Mary is a keynote speaker and has more than 1,000 articles, research studies, and technology publications in print.