Big Data

Big Data and the million-node network

As you consider how to cope with a million-node network, apply Big Data diligently and carefully, with a business problem the tools are expected to solve guiding your implementation.

As networking technology has advanced, it has matured to the point of near-magical capabilities. Having come of age in an era of DOS drivers and static IP addresses, I'm amazed that I pull down network speeds at 70 mph (as a passenger, of course) on a pocket-sized device, which was unfathomable on a hardwired LAN not that long ago. That same lowly copper cable can move mountains of data, and new standards promise even more speed.

While new speed records elicit little more than a yawn and are now essentially expected, one of the larger changes enterprises are facing on the networking frontier is an explosion of network nodes. Until relatively recently, network device counts were somewhat static and relatively easy to predict. Employees might be issued a network device or two, every 10-100 users required a certain number of printers and infrastructure, and the largest bumps might result from a physical expansion of the company or adding additional redundancy to the network.

With the proliferation of mobile devices, we're getting a taste of what's to come. Employees who might have owned a network node or two now have a smartphone and potentially a tablet, and the "one-two" punch of wearable computers and sensors and connected products are going to appear on your network.

The wearable network node

Wearable network devices are an emerging category of network node that might be anything from a relatively "dumb" sensor embedded in a piece of office furniture, to a watch-sized device that's essentially a full-scale computing device running a networked OS and maintaining a network connection throughout the day. In the case of the former, companies are increasingly embedding sensors in the workplace to determine how employees communicate, where they congregate, and what types of office layouts foster the most productivity.

Sensors can be embedded into everything from chairs that report their location and "occupancy rate," to employee badges that gather data about where an employee spends his or her time and how they move through the office environment-generally with employee consent and awareness, of course.

Smart watches and devices like Google Glass are essentially full-scale computers with a WiFi radio and frequent connectivity demands. These devices are even creating ad-hoc "micro networks," with devices like consumer fitness sensors and car radios connecting to a smartphone, which then acts as a network gateway to the larger internet.

Similarly, the products that your company manufactures may become increasingly connected. The average television now sports a WiFi radio or Ethernet jack, and these devices are likely reporting status back to the manufacturer, requesting firmware updates, and being employed in remote diagnostics.

Management and analysis

While conventional network tools can scale to a significant number of network nodes, it's not difficult to envision scenarios in which the number of nodes on your network requiring tracking and management could double or triple in a matter of months.

Furthermore, the main driver behind connecting devices and sensors to a network is to gather data for analysis. While many of your IT initiatives around Big Data may be targeted at business users, this is a prime use case for Big Data within IT.

However, before you fire up the latest Big Data tools, embed a connected sensor in every chair, and wait for every employee to demand a dozen network connections, consider what questions these network nodes will raise, and the data they generate will answer. Do you merely need to track assets for rudimentary management purposes, or is a deep understanding of network usage and allocation or office interactions critical to some business decision?

Like all data-related projects, the great risk is becoming what amounts to a data hoarder, building the capture and analysis infrastructure and loading it with data, only to gather massive quantities of data that produce little more than corporate trivia and no measurable or actionable outcomes. As prices for the technology to enable this level of tracking fall, it's tempting to "gather now, and consider analysis later," but this is a recipe for unwieldy troves of data that consume significant maintenance expense, yet produce little value.

As you consider how to cope with a million-node network, apply Big Data diligently and carefully, with a business problem the tools are expected to solve guiding your implementation. Done correctly, the massive networks can be effectively managed and provide valuable insights. Done poorly, and you'll need teams of people to gather and curate troves of data that do little more than collect digital dust.


Patrick Gray works for a global Fortune 500 consulting and IT services company and is the author of Breakthrough IT: Supercharging Organizational Value through Technology as well as the companion e-book The Breakthrough CIO's Companion. He has spent ...