A first business information (BI) data warehouse implementation often can be characterized as a utilitarian effort, subordinate to the company's core database systems. User perception and the mindset of non-IT management, in particular, may cast BI and its attendant data marts, analytics, and other innovations as simple conveniences. This is a flawed mindset, and deeply undermines the purpose of BI: Data warehousing is the embodiment of a commitment to optimized business intelligence and decision support.
These new facilities do not simply augment the manner in which your company already uses data. Instead, their implementation is a first step toward reinventing the way data is understood and perceived in your user community. Data is no longer stuffing for reports; it is a razor-edged tactical tool that will slash process time and increase returns. It is a springboard for changing minds to a more tactical orientation.
You're in the "data newspaper" business
A newspaper editor, like the administrator of a data warehouse, is in the business of providing timely, accurate information. But the similarities go deeper. Let's compare the newspaper business to database administration.
A newspaper competes successfully as an information service by knowing its readership; tailoring the content to appeal to that readership; presenting that content in an attractive, accessible manner; committing to accuracy in reporting and ensuring a consistent standard; developing solid, reliable sources; cultivating new subscribers and learning their tastes; and maintaining a well-organized history of all previously published stories as a service to the readership.
The ideal for the database realm is exactly the same. You need to know your user community and its specific needs, stock the warehouse with the information that is truly useful, organize it in such a way that it's easily accessible to the users, put procedures in place to ensure accuracy of data, know where to get good information, make the facility available to a growing user base, and maintain an easily accessed, well-organized data history for your users.
The two operations, as information services, are very similar. If you're looking for contrast, look at the differences between a data warehouse and a traditional database environment:
- In a traditional information processing system, available data define the users; in a data warehouse environment, the users define the available data.
- In a traditional database, information is stored in structures and formats convenient to the application programmers; in a data warehouse environment, the design of information format and storage is driven by user convenience.
- In a traditional system, current and historical data are separate systems; in a data warehouse, they are inextricably linked.
- In a traditional company, applications are the bridge between data and users; in a data warehouse environment, the user is the bridge between data and applications.
Each of these features of a data warehouse environment is a critical design driver, and you must have a solid grasp of these fundamental differences. You also must meet the bigger challenges of getting these concepts across to management and the user community and establishing solid support for these changes from the outset.
Your core design considerations
Let's take a closer look at key things to keep in mind as you plan a data warehouse.
As you move into distributed and extended applications, you create one big system by networking together increasingly specialized subsystems. You're no longer designing data storage around a common center. Instead, you're creating islands of data that are increasingly granular, and using new technology to tie them together efficiently. There doesn't need to be a center. And that makes it easier to add on later.
You're not designing this new data storage facility to please programmers. After all, programs will not be its primary visitors. You're designing it to accommodate the users themselves. And since they're the ones who will be meandering around inside, you need to make data available in structures they can understand. And what users can really relate to are shapes.
The concept of a cube is easy to grasp. Three dimensions can be clearly understood because they can be readily visualized by laypersons. Data "cubes" are easily assembled, flexible, and easily related to other cubes. (In addition, this structure is easily realized by designers and programmers.)
Regarding relational structures, there are many to choose from. You need a "shape," and it needs to be one that accommodates the new distributed/extended application paradigm. That is, the structure needs to help fulfill the users implied request: Take me from this starting point to all the other downline points I need to go in order to gather together my somewhat customized batch of information.
What you need is a wheel-with-spokes shape—a center (a starting point) with relationships going not in one or two directions, but many. Put more simply, you need a "star." As with cubes, it's easy to link these shapes because one spawns another, which spawns another, which spawns another.
One problem you will face is that information stored in your warehouse will come from a wide range of sources. Once you've gathered in the data, you will need to establish a common standard for labeling types of information, even if data within a single type has come from multiple sources. This change will be resisted by many parts of the enterprise. Once you've made this painful adjustment, however, the ease-of-use, the points of commonality for linkage, and the expansion of design will be so much simpler that it will be well worth your while to overcome any entrenched resistance.
The BI data warehouse is purposefully structured to permit rapid, flexible reconfiguration of information. This, combined with unprecedented ease-of-access, effectively puts the data directly into the user's hands. Users are no longer dependent upon transactional applications to get to the data they need. In a follow-up article we will discuss the result of this new flexibility—business objects.
Scott Robinson is a 20-year IT veteran with extensive experience in business intelligence and systems integration. An enterprise architect with a background in social psychology, he frequently consults and lectures on analytics, business intelligence and social informatics, primarily in the health care and HR industries.