Big Data is becoming a strategic technology across a number of industries right now. If you are starting to make some Big Data moves for your own organization, a key first step is to foster your Big Data expertise in house before involving outsourcing providers or consultants.
Fostering Big Data expertise in-house includes:
Recruiting your Big Data team internally
Since Big Data sits at the intersection of IT and analytics, you may already have a strong start of a Big Data team internally without even knowing it. The seeds of a Big Data team actually begin with your power users who are technology and business savvy with the institutional and job specific knowledge of your core operations. They also need to have strong analytical skills and be comfortable with analytical tools. There is no college degree as a prerequisite. The team members don't even need to be programmers.
Some initial skills to seed a Big Data team with includes:
- SPSS or SAS statistical modeling
- Structured and unstructured databases
- Hadoop, an open source Apache framework for distributed computing and storage
- Data warehousing
- Data mining
- Machine learning techniques
Finding such talent in house might bring up some non-traditional candidates for your Big Data initiative but they have domain expertise in your business area. A core Big Data team of internal staff is going to be the people who understand your data best.
Hunting the elusive data scientist (or maybe not)
One of the toughest job classifications to fill these days is Data Scientist. These people interpret the information in Big Data. They've become the latest rock star in the tech industry right now. Getting a Data Scientist for your Big Data efforts can be a difficult undertaking even if you can offer a big salary and outrageous benefits.
While there are Big Data tool vendors who claim their tools can replace the need for an in-house Data Scientist but like new tools in any emerging or even established market, you are bound to run into limitations as your Big Data Team gains more experience and understanding of the tools and Big Data visualization in general.
Filling the Data Scientist gap can be daunting, so it may come down to creative hiring to foster the strategic analysis expertise required to interpret data into actionable information for organizations. The well-documented Big Data expertise shortage makes fostering Big Data expertise in-house make even more business sense. Time and resources you would squander on the hunt for a Big Data Scientist can be better spent building the Big Data qualifications of your current employees. Eventually, data scientists will become more plentiful as data science programs produce more graduates.
Start treating Big Data as its own career track
Your new Big Data team is going to require its own career track for a number of reasons. Big Data can be a game changer for your organization meaning new if not potentially higher level roles for the team. Branching out into Big Data also means figuring out new levels of responsibility for team members.
Develop a training plan for the Big Data team
Ongoing employee technology training is critical to Big Data team operations and staff development. Big Data is new territory for many organizations so training is necessary to ensure your Big Data team is conversant in the latest best practices and technology developments.
While training budgets can be tight for many organizations due to the economy, Big Data University, a free online Big Data training site and a growing list of vendors offering training mean that you can string together an in-house training program that can meet your budget.
Choose the right Big Data tools
One of the more interesting facets of Big Data to me is the accessibility of Big Data to a wide user community. Many of the popular Big Data analytical don't even require specialized training much less a vendor certification to use effectively It is these very tools that put the power data mining, manipulation, and reporting into the hands of power users...not 3rd degree Big Data black belts.
Another benefit of the right tools is that it frees up developers and system administrators from having to run reports. When you equip Business Analysts and other power users with tools like Datameer, they can create their own custom reports based on your Big Data repository.
When your Big Data team starts to immerse themselves in the new tools, then it is time to document processes and procedures to ensure your Big Data efforts are replicable. This internal documentation can then help with your future internal Big Data training.
Set realistic expectations and create a plan
Following developments in Big Data and how they are impacting multiple industries makes it very clear that to foster Big Data Expertise in-house requires realistic expectations if not a charter and business plan for the team that charts out the following:
- Team member responsibilities
- Big Data roadmap for the company
- Big Data project plan
Level setting expectations for progress into Big Data is also a prudent move to ensure stakeholders and executives are comfortable with your company's progression towards Big Data.
The value proposition of In-house Big Data Expertise
Fostering Big Data expertise in house makes good business sense because it is about establishing a strong foundation for your Big Data efforts using a core team of people who already understand your business and its data. Outsourcing and consultants at the early stages are only going to become a distraction, and place valuable lessons learned and knowledge your organization needs in the hands of a third party.
Will Kelly is a freelance technical writer and analyst currently focusing on enterprise mobility, Bring Your Own Device (BYOD), and the consumerization of IT. He has also written about cloud computing, Big Data, virtualization, project management applications, Google Apps, Microsoft technologies, and online collaboration for TechRepublic and other sites. Will also works as a contract technical writer for clients in the Washington, DC area and nationwide. Follow Will on Twitter: @willkelly.