Big data
– and all data – continue to up the ante for faster processing. The way the
industry has traditionally addressed this is by coming up with faster chips for
the CPU and other processing functions in hardware. From a data center
perspective, managers want to see all of the capacity utilized, and in big data
environments it is not uncommon to see upwards of 90 percent server utilization.
But in transaction processing environments, server utilization can fall to as
low as 20-30 percent, which is not an effective optimization of hardware.

Beyond the Stone Age

“We
have beaten up hardware relentlessly to improve performance and also energy efficiency
in data centers,” said Mike Hoskins, Chief
Technology Officer At Actian, a provider of big data analytics solutions, “There
has been so much invested, yet the results have been poor.”

Hoskins believes
that the path toward more effective server utilization in data centers rests in
software, and he uses the example of a two processor, 16 core server to
illustrate his point.

“One
can argue that software is in a kind of “stone age,” in that it just
hasn’t kept pace with hardware innovations,” said Hoskins. “In the
case of a two processer, 16-core server, single-threaded software, which most
software today is, only keeps one to two cores busy and the other 14 cores go
unused. We experienced a bit of a breakthrough when virtualization technology
came on the scene, because in an environment like VMware, you could take eight
cores of processing and spread them over four virtual machines that used two
cores each.” This compensated for the limitation of single-threaded software
because the software could be spread across four different server engines with
the core divisions afforded by the virtualization.

Unfortunately,
when we are talking about big data and analytics, “sleight of hand”
virtualization techniques that can improve software performance simply don’t
work. The reason is that big data with its massively parallel processing, isn’t
well suited for virtualization. Consequently, sites are potentially left with the
challenge of crunching through massive amounts of data in a small amount of
time – and possibly bumping up against limits in the software they are using. At
the same time, their budgets constrain them from investing in even more
powerful processing that goes beyond what they have available to work with – various
incarnations of x86-based server technology.

Hoskins
believes that sites can overcome the single-threaded core utilization limits of
most software if they can somehow lash processing cores together in a
harmonious memory-only data flow engine that can parallel-process incoming data
and also address the various steps of big data processing that have to be done,
such as data cleaning, aggregating, ingesting, and finally analytics.

Tiers of data

The idea
is to move tiers of data in memory closer to the CPU to improve overall performance.
Here is how it works.

There
are three tiers of in-memory data storage in cache: L1, L2, and L3. Any one of
them gives faster processing results than having to go out to external memory.
The L1 tier is the first tier that is checked for in-memory data, but it is
small in size, has smaller blocks of data, and also fewer blocks of data. As
access moves into the L2 and L3 tiers of this in-memory storage, the data blocks
are larger and there are more data blocks. The L3 level in particular has the
block size (and number of blocks) to handle big data that is being parallel
processed. There is also no need to go out to external memory if data is
resident in L3 data cache.

“In
this environment, a processing engine can know where data is executing and then
push the data that will be required into L3 cache,” said Hoskins. “This
is one way that we can get around software constraints and get the most out of
hardware.”

The
technology is promising and exciting to sites that want to stick with commodity
x86 servers (PDF) for their big data processing. As use cases begin to
appear, it is also likely to add fuel to current architectural debates as to
whether x86, Unix-based or other hybrid platforms are best suited for
enterprise HPC (high performance computing).