Mary Shacklett covers the key planning and research considerations for networks and Big Data.
I recently blogged about the need for companies to include network planning in their Big Data strategies-but there is also an immediate need for IT to "get ready" for major breakthroughs in networking technology that are beginning to hit the marketplace and that could transform the world of Big Data.
This comes at a time when most IT departments still view their major Big Data challenge in networks as constricted bandwidth, which can't accommodate the new, real-time Big Data payloads that are coming into enterprises from the outside-and are then being sent out again.
But at the same time that corporate budgets are being upped for network expenditures, IT should also be researching new network technologies that will help their Big Data transport in the future. For instance:
In Kansas City last year, the Google fiber team connected homes and businesses to a new Internet pipeline that is 100 times faster than anything residents had ever seen before-and also very capable of transporting Big Data payloads.
Internet2 continues to work on a next generation 100G (gigabits per second) network that will more effectively integrate Big Data applications by finding ways to shorten latencies encountered with network security when Big Data passes security checkpoints.
Technology providers like Avere are revolutionizing NAS (network attached storage) to accommodate Big Data that is shared by organizations outside of enterprise walls, with access to Big Data from edge storage devices that have built-in data access management that works in real time-and that offloads these Big Data payloads from the central corporate network.
This is what is driving progressive IT organizations to move more network technology research into its end to end Big Data planning.
Key Big Data planning and research considerations for the network should include:Thinking differently about networks—-A majority of companies still limit their network expansion thinking to bandwidth growth, network node growth and network security-all performed within the network group's IT silo. IT needs to think more holistically about how its networks interact with applications and other IT resources-and to actively research new network technologies that can foster more holistic technology integration. Moving to an IT service orientation-With Big Data applications now moving wholly integrated storage, CPU, software and networking into service packages for end users, IT staff also needs to coalesce around these new business services and to work together as a seamless team. In part, this effort will involve everyone working with the same set of technical tools. The other work area entails making IT culture changes so that technical silos come down and every element of a business service comes together in work and planning efforts. Revisiting network security by placing an emphasis on security throughput as well as on effectiveness-Security must be as robust as ever, but it also can't continue to be a slowing point for Big Data transport. New technologies are on the way to improve throughput through security and should be researched and investigated—at the same time that IT ensures continuing conformance with governance and regulatory requirements. Focusing on end to end network (and IT infrastructure) management-Every vendor wants to sell you its data access optimization management-but how do you pull it all together so you have an overall way of managing your entire IT infrastructure, regardless of the heterogeneous mix of solutions you are employing? This is an ongoing struggle for IT for both transaction and Big Data applications.