Data Centers

Beware of data warehousing death knells

The client can have a detailed proposal prepared and the users can be thrilled with the prospect of data warehousing, but if you notice one of these three warning signs, your warehousing project is probably on the path to failure.


By Harris Kern

After working as an IT consultant for many years, I can’t help but recognize the warning signs of impending disaster on a technology project. Here are three situations I’ve encountered in the course of my data warehousing consulting practice that I’ve come to recognize as warehousing death knells.

1. "Don't talk to our people"
One of the worst things a consultant can encounter on a data warehousing project is an IT department or division that is resistant to the very idea of warehousing.

Many years ago, I had the pleasure of working with one of the key user groups in a financial institution in Southeast Asia. These were technology-savvy users who had long been frustrated by the state of management reporting in their organization, and they were now excited by the prospect of having integrated customer data at their fingertips through data warehousing technology. They were contacting product vendors, meeting with hardware companies, talking to consulting firms, and, in all aspects, leading the charge forward on this initiative.

With data warehousing so dependent on operational system data, it was inevitable that I would need to talk to the organization’s IT department to gain a better understanding of the existing technology infrastructure. And it was there that I hit the IT wall.

Not only did it take numerous attempts to set even a single meeting, each meeting we had was largely unproductive, with IT staff who were either cold or openly hostile when data warehousing was being discussed. Before long, I heard the death knell of this project: “You can borrow whatever documentation you want, but don’t talk to our people and don’t touch our systems.”

It became obvious fairly quickly that I had walked into an ongoing struggle between the IT department and the business users. The IT department had prioritized work on the operational systems, and, in fact, was constantly fire-fighting to keep the operational systems from crashing. They correctly realized that the last thing they needed—or wanted—was another project to dilute their focus. And once that became obvious to us, we knew it would be prudent to back off.

It has been four years now since my first encounter with that IT department; and it was only last year that that department started to once again explore data warehousing technologies. With the Y2K challenge successfully hurdled, perhaps it is now finally ready to discuss data warehousing and business intelligence applications.

2. The three-year turnkey warehouse
Three years ago, I was invited to attend a vendor’s meeting at yet another financial services institution. Interestingly, among the nine companies invited to the meeting, mine was the only consulting company. In this vendor’s meeting, user representatives from the prospective client distributed a rather thick requirements document. They also presented a brief overview of the data warehouse they wanted.

While it was obvious that the organization’s representatives had taken the time to prioritize their information requirements, they’d made it difficult for the invited vendors to meet these information needs. A thin section of their bulky Request For Proposal contained a high-level description of their requirements, which included Customer Profitability Analysis features, house holding, as well as the use of census and postal data from external sources.

I knew from prior contact with this client that it had significant data quality problems and was planning to modify its operational systems as a consequence of a massive business reengineering exercise.

Its request for proposal (RFP) document included a list of potential source systems, with its corresponding hardware platforms and operating systems, but not much else. As it turned out, the rest of the RFP document consisted of detailed contractual terms that the client required of each vendor who would respond to its bid.

Not only did this client require a turnkey bid for what obviously would be at least a three-year warehousing effort, but it also required each vendor to submit detailed project schedules. Its contractual terms had ridiculous penalty clauses that defined penalties for every day that any project milestone was missed. It also expected all vendors to fully specify the hardware platform that would be required, but the client was unwilling to provide enough information on its data volumes and expected growth rates to do so. The client also refused to provide any documentation on its operational systems.

I knew that a data warehouse of this magnitude could not be handled through a turnkey engagement. I was also unwilling to specify a hardware configuration without having enough information to base it upon—especially since the client’s penalty clauses required free upgrades and maintenance for at least 10 years. And while the client agreed to take full responsibility for the quality of its data, it also stated, somewhat naively, that its data quality levels were no lower than 90 percent.

In the face of such demands, I decided prudence was the better part of valor and withdrew from the bid. I found out later that of the nine vendors invited to the meeting, only one company had been brave—or foolish—enough to submit a proposal for what obviously would be a suicide mission. With only one vendor response, the whole exercise was officially declared a failed bid.

3. Focusing on the sizzle
Yet another organization, this time in the Telco vertical market, was in the process of evaluating warehouse vendors a year and a half ago. Its vendor evaluation process was very much user-driven, with the IT staff taking a backseat. The IT department was supportive of data warehousing, in general, but wanted no part in the decision-making process and was content to leave the selection to the user groups.

With the proliferation of data warehousing products, particularly GUI-based and Web-based front-end tools, it’s easy to impress nontechnical users with flashy demos. I’ve also sat in on some sales presentations where a product vendor waves away concerns about data quality and will casually promise a fully implemented data warehouse in 60 to 90 days.

By leaving the buying decision completely in the hands of the users, the IT department in this organization was not protecting the interests of the company. The business users did not realize that building and implementing a warehouse required more than just buying and populating the data structure of a front-end tool. They were focusing on the sizzle instead of the substance. The IT department, by trying to avoid culpability for any wrong decisions that could be made in the warehousing effort, was actually helping to set the stage for major problems in scalability, data integration, and data quality.

How do you measure up?
My experiences with the three clients above highlight just a handful of the typical obstacles that any organization will face when embarking on a data warehousing effort.

If your organization is embarking on a warehousing effort, check your own project. Listen for these and other warehousing death knells. If you hear them, you would do well to heed their warning.

For more information on the Harris Kern Enterprise Computing Institute, visit www.harriskern.com.

Editor's Picks

Free Newsletters, In your Inbox