CXO optimize

Should high performance computing matter to IT?

2013 will be the beginning of HPC migration into the data center. CIOs will be ahead of the game if they take these four steps.

Supercomputing and data crunching used to be the sole domain of universities and research institutes--but with today's business analytics and new "affordable" platforms for HPC (high performance computing), more enterprises are taking a hard look at moving HPC into their data centers. First, though, there is planning to be done and decisions to be made.

The decisions begin with who is going to fund HPC and who is going to run it.

To date, it has been end user departments in businesses that have funded HPC investments because they are the ones making the business cases for bringing HPC into the enterprise. These HPC business cases have been very specific. They have ranged from modeling new drugs to simulating the collision impact of new car designs to performing advanced risk analysis on loan portfolios in order to minimize the risk of loss. For those departments with engineering staff onboard, it is engineers who actually develop the algorithms and write the applications for HPC. If they are not developing these apps directly, they are making decision to purchase them from outside providers.

This arrangement has been fine and even preferable for IT, which continues to have its hands full with the normal lineup of daily processing and more standard computing requests-but now, many enterprises are beginning to change the way they view HPC in the enterprises. With this change, some businesses are insisting that IT take a more active role.

Just what kind of role do businesses want IT to assume with HPC?

They aren't asking IT to start programming HPC applications, but they are increasingly recognizing the custodial value of housing HPC assets in the data center under IT supervision. The reason is simple. IT already has resource management disciplines and expertise in place.  The policies and the procedures needed to maintain computing assets on an ongoing basis are mature and proven. In contrast, end user departments focus on their business and not on the underlying computing assets that support it. In short, they have no concept on how well the underlying hardware and software are running.

Because of these limitations, IT can deliver HPC value to the enterprise in two critical areas:

  1. It can assure that HPC hardware and software are performing the way that they are supposed to, and that HPC resources are optimized. This resource optimization comes primarily in the form of verifying that HPC resource utilization is in the 90 to 95 percent range-a significant utilization boost over the 60 to 80 percent of resource utilization that enterprises have learned to expect from traditional transaction processing systems.
  2. IT has the expertise to optimally schedule the jobs that must run in the HPC environment, based on job priority and resource consumption. This is an important point. Many internal departments running HPC jobs will share the same HPC computing clusters in the data center-and even though they might concurrently run in parallel, choices as to which jobs are prioritized to get the first crack at HPC resources can become a political problems as rapidly as they become technical challenges. In this scenario, especially if IT itself is not an HPC end user, corporate management could see IT as a disinterested and neutral party that is focused on the computing alone-and ask IT to be the job scheduler.

I believe that 2013 will be the beginning of HPC migration into the data center for the above reasons-and that the time is now for IT to start planning for its HPC custodial role. CIOs will be ahead of the game if they take these four steps:

  1. Determine how HPC is going to fit with the rest of data center infrastructure---and who in IT will be assigned to administer the HPC resources.
  2. Arrange to meet with upper management and HPC user departments to establish general guidelines and priorities for HPC scheduling that are based upon corporate priorities.
  3. Build primary contact relationships with HPC vendors, even if other end user departments initiated these relationships and signed the purchase contracts.
  4. Incorporate HPC into the IT strategic plan. HPC at the onset is likely to be more of a "siloed" resource in the data center---but as business analytics becomes more sophisticated, it is likely that HPC will be pulled in and integrated with other data center resources.

About

Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President o...

6 comments
ion_tichy
ion_tichy

I worked in a large telecom manufacturer many years ago. I was on the manufacturing side but also working with our R&D sites located in North America and Europe. I worked in a group of 200 engineers that performed hardware/software/firmware sustaining engineering as well as providing extra hands to our R&D sites for overflow work. So, we needed the same tools as our R&D people : high-performance large databases for hardware and software PDM, high-speed compilers for software, hardware modellers, digital telephony switches, digital switch emulators, Unix-based servers and workstations, and the like. We had our own IT staff to manage our specific environment, separate from the central IT group. Our IT group did work with central IT at our site as well as the IT group at the R&D site but was dedicated to serving our specialized environment. Our embedded IT team also had the responsibility of synchronizing data and CAD/CAM/CASE/PDM toolsets with the external sites. Some of the embedded IT people had engineering experience using the tools and were actually doing 2 jobs, a balance of IT and engineering, which I have seen to be a rare commodity. The rest of the facility of 3000 people included different functions such as finance, accounting, product management, manufacturing, component/test/manufacturing engineering which were best served by the central IT organization because they all used PCs and generic applications. By way of this example, I am saying that if you have very specialized departments with their very own unique computing environments, that these departments may be best served by a dedicated embedded IT team with the skills that the central IT group may not have.

CharlieSpencer
CharlieSpencer

I'm not a cloud advocate, but this sounds like the type of support that all the gurus are advocating be outsourced, not moved under IT's wing. Is this another case of one set of 'experts' staking out a position in opposition to a different group?

HAL 9000
HAL 9000

The Departments who want this stuff are attempting to pass the costs to IT and save themselves more money and causing IT to look like more of a Business Expense with no valid return to the business. Never underestimate the Power of the Bean Counters to transfer legitimate costs from their departments to another to make themselves look better/more profitable and pass all of the costs to another making them look worse while decreasing their Budgets and still expecting the same returns or better. The entire thing is ridiculous and a perfect example of why Accountants should never be placed in a position of Power over anything. Col

Tony Hopkinson
Tony Hopkinson

You are talking like IT has some influence on proceedings! I don't get it, have you been away?

nick
nick

In my experience IT has always owned the environment. I have worked in several banks and finance organisations and the Mainframes and other dedicated computing devices have always sat in the data centre. The day to day operations and maintenance has also been an IT function. Often the business that needed the computing power had its dedicated team of analysts and developers running the apps and often the OS, Are you suggesting that in some way HPC's are handled differently in a large number of organisations? The only time that I have seen issues are when the business has gone behind IT back and created their own dedicated IT environment running on high end workstations or even PCs and then demanded IT support when things go wrong or key staff moved on taking their knowledge with them.

Tony Hopkinson
Tony Hopkinson

and then dialing in at 300 baud to get the answer... :D