Integrated vs. specialized: Which shines the brighter light on ‘dark data?’
Tools in the big data market love to divide themselves up into functional categories, leading to best-of-breed solution approaches. But does this help customers? Or do they need a broader approach?
Ingest, data prep, analysis, visualization, and export -- they’re all part of the big data analytics lifecycle. The good news? The market sports multiple product categories that handle each of these. The bad news: there are far fewer products that handle many of these areas together. It’s getting more complex too, as other big data lifecycle categories, like data lake management/data cataloging and big data operations/DevOps are emerging.
This may be frustrating, but it’s also understandable: if there’s an area of functionality that the market is neglecting, funding a company to develop and provide that functionality makes sense. It helps the market, it helps the company and, hopefully, it helps the company’s investors.
Does it help the customer, though? That’s harder to say. If you’re doing your data prep in one tool and your analysis in another, that’s a bit regimented, and the context switch may be jarring. Moreover, if your analysis reveals to you more prep work to be done, you are sent back to the data prep tool for a “do-over,” rather than embarking on iterative effort.