Data Management

IBM's Devil's Triangle: An enterprise software soap opera

IBM faces lawsuits and public embarrassment in the Philippines over a failed government project involving the company's DB2 database product. Michael Krigsman explains why the situation offers a textbook example of the Devil's Triangle.

This is a guest post from Michael Krigsman of TechRepublic's sister site ZDNet. You can follow Michael on his ZDNet blog IT Project Failures, or subscribe to the RSS feed.

IBM faces lawsuits and public embarrassment in the Philippines over a failed government project involving the company's DB2 database product. The situation offers a textbook example of the Devil's Triangle, and demonstrates the tensions and conflicts that arise between technology vendors, customers, and system integrators.

Background

The Government Service Insurance System (GSIS), a Philippine agency responsible for managing the pensions of government employees, installed DB2 in 2006. By early 2008, the system began showing signs of weakness. Local newspaper, the Philippine Daily Inquirer, describes what happened:

[GSIS chief legal counsel Estrella Elamparo] explained that the software started showing problems in early 2008, particularly in handling voluminous chunks of data.

"IBM upgraded its database system purportedly to enable it to handle unlimited volumes of data," Elamparo said. "However, the reported upgrade only worsened the problem because instead of fixing the problem, the database began mishandling data and prevented the simultaneous use of data."

The government threatened lawsuits in response, according to the paper:

The Government Service Insurance System warned...it would launch "a series of legal actions" against IBM for "incalculable damages" that the pension fund sustained due to defective computer software.

GSIS went on to purchase a full-page spot in a major newspaper, the Manila Bulletin, and published an open letter explaining the situation. The Bulletin reports:

The Government Service Insurance System (GSIS) has put out an open letter to its active members and pensioners, detailing how the software installed by computer giant IBM in the GSIS system has, to borrow the Fund's own term, "turned into a nightmare."

GSIS has been having difficulties processing the claims and benefits of members, pensioners and other beneficiaries, as well as the glitches encountered in some instances in the posting of payments made to the GSIS.

Another Daily Inquirer article describes the technical environment:

Full swing implementation was in 2008 after GSIS tapped systems integrator Questronix Corporation. Under the contract, Questronix would implement a full database package using DB2 version 8 and an SAP business application.

However, the agency started seeing problems with DB2 version 8, particularly "bad pages" on the tables. The agency later discovered that the DB2 tables could only handle 256 gigabytes of data, which was lower than what GSIS needed.

The database software was then upgraded to DB2 version 9.1, which could handle 512 exabytes of data, or one billion gigabytes.

[Elamparo] explained their database table exceeded 2 terabytes (2,000 gigabytes) worth of transactions. "We expected DB2 to keep running since it was only 2 terabytes."

GSIS is now pursuing various legal measures against IBM, including filing criminal charges accusing IBM Philippines of bid rigging in connection with the project. From local newspaper BusinessMirror:

Elamparo [said] that IBM Philippines may be held liable for possibly dictating the bids of its dealers as a third party who is technically not a bidder.

"By giving various discount levels to its dealers involved in the bid-from 30 percent to 80 percent -IBM Philippines is in effect empowering one dealer over another, since those who had gotten a bigger discount would naturally be able to bid lower," Elamparo said.

She brought the National Bureau of Investigation's attention to Section 65 of RA 9184, which bars a bidder from employing schemes that restrain natural rivalry of bidders, refrain from bidding to provide another bidder undue advantage, and knowingly submit high bids to have the contract awarded to a prearranged lowest bidder.

IBM response

In a statement to local television station, ABS CBN news, IBM denied creating the problems, pushing blame back to GSIS and third-party system integrators:

"GSIS did not engage IBM in the selection, customization and implementation of this system. IBM was the OEM (original equipment manufacturer) provider to one of the technology vendors engaged by GSIS. GSIS does not have any maintenance or support contract with IBM," the statement read.

"Nonetheless, in view of our long standing relationship with GSIS and out of goodwill, IBM has been working with GSIS' solution providers to resolve GSIS' system issue."

IBM also denied violating procurement laws, according to another local station, GMA NEWS TV:

The Philippine unit of US-based IBM said "it does not agree" with the statements made by the pension fund which earlier said that the company asked one of its dealers to pull out from a bidding for an IBM Linux server.

"When IBM was notified of GSIS's concerns last year, we took immediate action to thoroughly investigate the matter and found there were no irregularities in the way we conducted our business. We shared these findings with GSIS," IBM Philippines said in a statement.

SAP distanced itself from the situation, saying it has no contract with GSIS:

SAP Philippines said that it had no contractual relationship with the agency in the implementation of its IT infrastructure software or consultancy services.

Instead, the software company stressed that GSIS licensed SAP software from Team Synergia, a local value-added reseller.

The project failures analysis

Despite the hyperbole, accusations, and denials, it appears we can ascertain these facts:

  • GSIS purchased a DB2 system from IBM as part of a broader system implementation.
  • IBM did not supply either implementation services or substantial software aside from DB2.
  • Something went horribly wrong with the DB2 deployment, interfering with GSIS's ability to process data resulting in substantial negative impact on the agency's members.
  • SAP supplied ERP software through a third-party reseller, but did not perform any contractual services directly to GSIS.

On the surface, IBM appears to be the bad guy, supplying a malfunctioning database and then refusing to help when things turned bad. However, large-scale system deployments are so complex that one cannot isolate the technology from the associated implementation services.

In this situation, we do not have enough information to divide responsibility between IBM, the customer, and third-party system integrators. This situation is a classic Devil's Triangle relationship, with tensions arising between vendor, customer, and integrator due to conflicts of interest that are present amidst shared goals and outcomes.

I asked IT expert and fellow ZDNet blogger, Paul Murphy, for his opinion on this confusing mess. Paul suggests GSIS owns substantial responsibility:

DB2 can handle the job - but getting AIX to handle large quantities of non local disk is non-trivial and because this particular transition involves migrating people and skills from OS/390 to AIX I'd bet the farm that the technical mistakes causing the failures were made by deeply resentful mainframers doing jobs they don't understand and don't want to do - and those people, of course, work for GSIS, not IBM.

Thus senior IBM people in the sales channel probably bear some responsibility for helping the client get into this mess, but blaming either DB2 or IBM's technical people for it is likely to be wrong.

My take

It's almost impossible to accurately analyze this situation from a distance. In my opinion, whether IBM is ultimately at fault is somewhat beside the point. The company sold the software knowing that the customer's environment is complex and therefore should be deeply committed to helping make things right.

In addition, GSIS should disclose more details about how it managed the project. I suspect Paul's assessment is accurate and that there's plenty of blame to go around. The situation will likely remain a stalemate until GSIS is more forthcoming and accepts its own responsibilities.

Sadly, the victims of this enterprise software soap opera are those who rely on GSIS for loans and other forms of financial support. This is hardly a shining moment for enterprise software, IBM, GSIS, or the system integrators.

4 comments
cliveamcgregor
cliveamcgregor

This is an interesting article. Having been personally involved in the migration of a core business legacy systems from an IBM mainframe environment to a DEC environment, i can appreciate the levels of complexities and issues involved. Fortunately in my case, the implementation went well. This issue probably all started when the respective sales teams did their presentations to the decision makers at GSIS. My experience has been that when the sales team go out to get the business, they try to convince the customer that thier products will get the job done. Thier presentations usually highlight the "superior" capabilities of thier software/hardware but "gloss" over the technical issues that will have to be resolved regarding the integration and implementation of the total solution (really, thier job is to sell, sell and sell , not to get involved with putting together the nuts and bolts of the system). It is ususlly after the customer commits to the proposed solution(s) and the (respective) technical teams steps in that the levels of complexities become known and this is where things begin to unravel. My take on the matter is as follows: - The techies (representing the respective stakeholders - IBM, SAP, GSIS and all related third parties) could probably have worked more closely to determine the techincal issues to be resolved in order to develop the most appropriate integration and implementation strategy (the assumption i am making here is that they did work together but more from an "arms length" perspective). - The management of the project may have been beyond the capabilities of the project managers. I am not saying tha the project managers are not competent, but an implementaiton of this magnitude would require persons who have "been there and done that" sucessfully (i.e. Have the experience needed to manage such a project). The project manager(s)would/should be cognizant of the risks that could arise and would/should be familiar with the measures that need to be in place to mitigate or lessen the effects of these risks). - A change management strategy (assuming that one was in place) could also ensure the buy in and co-operation of the end users at all levels of GSIS operations (It could be that a resistance to change and non-commitment from GSIS staff could have added to the woes being experienced). In any event, such an exercise would ensure that issues unseen by the technical teams could have been brought to the forefront by people from the grass root to the upper levels within GSIS and these would add value to any decision being made by the respective technical teams and project managers. Yup..there is plenty of blame to go around, but i agree with you..there were no winners here except the sale people who i am sure got thier big fat commissions.

santeewelding
santeewelding

But the State of California, USA, has just informed me that its Board of Equalization no longer accepts paper for sales and use tax returns. "The latest hardware and software security is [sic] in place," they tell me. They tell me I have to do this. "Efile," they say. Efucked.

Ron_007
Ron_007

I've seen a similar large scale snafu. A project was started for 2 very specific reasons. The day the contract was signed, the main reason was voided. End of project, implementation day came and went, nothing delivered. So the second reason was voided. But, actually the second reason was also voided the day the contract was signed because the C'level made a totally nonsensical decision that had huge technical impacts, largely voiding the second reason for doing the project. The project was implemented 2 years late, after 2 unsuccessful implementation attempts, and truly "heroic" actions and impressive "hoop jumping". So now years later the project is live, none of the original business case (actually it was a largely political driven) for doing it was met, and the company is paying huge performance (massive CPU cycle overhead) penalties for really dumb design decisions imposed by Sr management and the 3rd party project team. The sad part is that the company really had 2 good opportunities to cancel the project, the 2 failed implementations, but lacked the will to do so. Based on my experience I'm sure there is plenty of blame for bad decisions to go around to all parties involved, directly and indirectly.

matalinodaw
matalinodaw

agree with you..some sales people are not that technical..

Editor's Picks