General discussion
-
Topic
-
Big data- does it mean applying more science on data or usage of Hadoop
LockedAs a senior architect heading various enterprise projects at Oxyent, I am now chartered to head data analytics projects. For last few years I have been working in projects related to GPU, Hadoop using various languages like Java, Python etc. But suddenly world is caught by Big data buzz and introduction of new tool providers on Hadoop or utilising Hadoop. For an architect still the problem remains solving the business goal in a technically sound manner, utilising team’s capabilities and creating maintainable code. But the buzz has caused us to learn Hive, Pig, Impala, Druid etc. which have capabilities to process huge data sets in a clustered or real time manner.
Do you think the too much buzz around technology is taking away focus from still the basic art form of coding? I believe that it is expected now (more than ever) that good programmers will be good reasoners of data. They will think more as researchers and dissect the data from various dimensions. They will help in visualising it so it solves business goal (while relying on designers for aesthetics). Thus stretching the engineer’s boundaries.
Although engineers (with advanced mathematical or engineering graduation degree) may have been exposed to some of these in there research work. But good programmers may find it a nuisance (as they have always focussed on language syntax) and left data insights to business analysts.
It will be great to hear what other experts in the domain have to say on the same.
