google search

Custom Search

Saturday, October 25, 2008

Using Data Warehousing in Strategic Decision Making

Though you can read many definitions of data warehouses that say that these systems are designed for "strategic decision makers" (or some other similar term) there is little written about actually using data warehouses in strategic decision making processes. In this essay, I would like offer some insight into using data warehouses in such decision making exercises.

First, let me define strategic decision making. There probably are thousands of published definitions. For working purposes let me say that a strategic decision is one that involves spending a lot of money and/or firing/re-assigning/hiring a lot of people and/or that is going to cause a lot of pain/joy until the next strategic decision is made. (Of course "a lot of" is a relative term.)

I assert that most of the uses of data warehouses are not for strategic decision making. Probably the most important reason for this is that strategic decision making usually is not done that often. Rather, I believe that most data warehouses are used primarily for post decision monitoring of the effects of decisions. Nevertheless, some data warehouse do get used in strategic decision making and are used very profitably.

What follows are some personal observations on how you may actually use a data warehouse in a strategic decision making exercise.
Creating "special" databases, modeling (not in the IS sense of the word), and formal reporting are the most time consuming tasks when using data warehouses in strategic decision making.
Later I will go into more detail regarding these topics.
Systems for strategic decision making tend to be relatively short-lived.
The amount of time spent using these systems sometimes can be measured in days counted on one hand. Those couple of days using the system, though, can bring more payoff than some canned reporting system used for years.
Usually the work must be done quickly and is requested with little advanced notice.
This work usually has to be done in anything from a long afternoon to several weeks. This is "figure it out as you go along work" where IS often must take the part of the business analyst. There is usually no time for formal interviewing and extended data modeling exercises. The "requirements" are usually gleaned from "business" meetings which IS may have a little struggle to get into or are related secondhand from attendees of these meetings. These requirements are usually ambiguous. IS usually has to put on its business hat and figure out what is really needed by the business.
You will probably have to aggregate data differently, use different calculations for derived numbers, and combine data that never have before been combined.
The work you are doing allows the business to see a point of view that is not the common view of the business. (In other words, a part of many effective strategic decision making exercises is to see the business in a different perspective.) You are doing this work because when you built the data warehouse, you built it according to what then was the common view of the business.
You may need to create special databases.
Often you need to run repeated queries against a subset of the data warehouse. The subset may be one created by an extract query with quite complex constraints. Or, as I just mentioned, you may need to repeatedly access new aggregates and calculations or you may have to repeatedly concurrently access data that are not in the production data warehouse or that are in the production database but are not easily combined. For the sake of simplicity and efficiency, your best course is to create a special database. You may be thinking you created a data warehouse so you would not have to build special "extracts" but, perhaps to no surprise, often there just is no way of avoiding these extracts. (For more on somewhat similar ideas about these special databases, see Ralph Kimball's discussion of "behavioral studies".)
You may have to "feed" data into user maintained spreadsheet models.
Much of the use of data warehousing for strategic decision making ultimately involves "feeding" user maintained spreadsheets. These "feeds" are either links to data stored in a data warehouse or the actual loading of data into spreadsheets. The spreadsheets are used because the user needs to change complex calculations - maybe as part of a scenario analysis but usually because there is continual doubt about how certain calculations should be made - and the user is most knowledgeable about doing these changes in the spreadsheet environment. (To put this in a little more technical terms, many of these calculations are inter-record, cross dimensional calculations). Many OLAP tools allow a great deal of flexibility in making calculations but these capabilities tend to be too difficult for the user who is in a hurry in the strategic decision making exercise. Note also that oftentimes it is necessary to, in turn, feed spreadsheet data into the special databases you have created.
Sometimes data cleanliness is much less of a concern in strategic decision making.
Sometimes the analysis being done with highly summarized data and/or the need for speed lessens the need for extremely clean data. I do suggest, however, that whatever the data expectations are, you keep an audit trail that lets you trace how data were derived from feeder systems.
You may have to create some highly formatted reports.
The information from the data warehouse has to be communicated to people who do not have and/or want direct access to the data warehouse. In a strategic decision making exercise, despite the rush, your users may want to communicate the information in printed reports that look just "so". These reports are usually being created to persuade someone. Many of your users will want a polished look to the reports in order to convey credibility. Also, graphs are usually created for these exercises. By the way, there is usually some give and take as to whether these reports and graphs should be created manually (i.e., with a word processor, presentation tool, spreadsheet) or generated directly from the database.

Now some advice:
Probably the most important determinant of the benefit you will get from technology is your ability to figure out the most insightful questions that the technology enables you to ask.
Do not assume that your users have full appreciation of the power of the technology. Unless you have some users with good gut instincts about technology, IS has to take the part of the business analyst to spur the imagination of the users.
Try to get in "the loop" early.
Users will tend to either grossly underestimate or overestimate the power of the data warehouses in these strategic decision making exercises. This means that either IS can miss an opportunity or be faced with an impossible task that must be done quickly. Note that there are usually politics in getting in the loop early. However, having previously built up a relationship of trust with a "decision maker" helps greatly.
When you are initially designing the warehouse, do not try to design for every contingency that could occur in a strategic decision making exercise.
You are not going to be able to foresee everything that will be needed in these exercises. Do not put everything you can possibly think of in the data warehouse. Do, though, try to keep atomic data in some electronically retrievable format. Do your best to conform the main dimensions of data used in your business. (That means customer, product, financial account, and internal "entity", i.e., people and department, identification.) Do address the slowly changing dimension issue. And do not make yourself completely dependent on outside resources whose availability you cannot control. These exercises come up unexpectedly.
Do not let the knowledge of the systems stay in the minds of the outside technical consultants
This trite and obvious piece of advice needs to be repeated. The technical consultants are gone and not available when these opportunities come up. If the key knowledge of your systems are in the heads of consultants, you may be up the creek when these exercises come up.
Learn spreadsheets and how your data warehouse can interact with them.
We in the data warehouse world often forget that the spreadsheet is by far the most used decision support tool. Persons supporting data warehouses that really will be used for decision support should be encouraged to learn the scripting language of the spreadsheet (which for most people is Visual Basic for Applications) so they have the flexibility in coming up with solutions in these strategic decision making exercises.
Don't "production-ize" your work.
The technical work done in these exercises is usually not "industrial strength" and it is probably not worth the effort to make it so. You may learn, though, that you need to modify your production data warehouse database. Also, do keep your work around so you can cannibalize code for the next strategic decision making exercise.
Do not claim that data warehousing alone will necessarily improve strategic decision making

It needs to be oft-repeated that if a person is a mediocre decision maker, technology alone will not make that person a better decision maker - especially in the realm of strategic decision making where, despite our 100 TB databases, much more remains unknown than known.
Don't miss these opportunities.
It is hard to calculate the expected ROI of a data warehouse project. Most businesses have to go on faith that the effort somehow will be worth it. Well, success (or, sometimes, just participation) in a strategic decision making exercise, despite the messiness of the work, can strongly bolster the belief that the data warehouse was worth the effort. If you do not justify a data warehouse before building it, it is smart, perhaps imperative, to justify the data warehouse after the fact. And the best way you are going to do this is "anecdotally" with successful war stories like a strategic decision making exercise.

0 comments:

 

blogger templates | Make Money Online