Did sales of other products improve? Did customer attrition increase? Did calls to customer service decrease? And so on. Having the data available makes it possible to understand the effects of an action, whether the action was spurred by data mining results or by something else.
Of particular value in terms of measurement is the effect of various marketing actions on the longer-term customer relationship. Often, marketing campaigns are measured in terms of response. While response is clearly a dimension of interest, it is only one. The longer term behavior of customers is also of interest. Did an acquisition campaign bring in good customers or did the newly acquired customers leave before they even paid? Did an upsell campaign stick, or did customers return to their previous products? Measurement enables an organization to learn from its mistakes and to build on its successes.
Scalable Hardware and RDBMS Support
The final synergy between data mining and data warehousing is on the systems level. The same scalable hardware and software that makes it possible to store and query large databases provides a good system for analyzing data.
Chapter 17 talks about building the customer signature. Often, the best place to build the signature is in the central repository or, failing that, in a data mart with similar amounts of data.
There is also the question of running data mining algorithms in parallel, taking further advantage of the powerful machines. This is often not necessary, because actually building models represents a small part of the time devoted to data mining—preparing the data and understanding the results are much more important. Databases, such as Oracle and Microsoft SQL Server, are increasingly providing support for data mining algorithms, which enables such algorithms to run in parallel.
Lessons Learned
Data warehousing is not a system but a process that can greatly benefit data mining and data analysis efforts. From the perspective of data mining, the most important functionality is the ability to recreate accurate snapshots of history. Another very important facet is support for ad hoc reporting. In order to learn from data, you need to know what really happened.
470643 c15.qxd 3/8/04 11:20 AM Page 512
512 Chapter 15
A typical data warehousing system contains the following components:
■■
The source systems provide the input into the data warehouse.
■■
The extraction , transformation, and load tools clean the data and apply business rules so that new data is compatible with historical data.
■■
The central repository is a relational database specifically designed to be a decision-support system of record.
■■
The data marts provide the interface to different varieties of users with different needs.
■■
The metadata repository informs users and developers about what is inside the data warehouse.
One of the challenges in data warehousing is the massive amount of data that must be stored, particularly if the goal is to keep all customer interactions.
Fortunately, computers are sufficiently powerful that the question is more about budget than possibility. Relational databases can also take advantage of the most powerful hardware, parallel computers.
Online Analytic Processing (OLAP) is a powerful part of data warehousing.
OLAP tools are very good at handling summarized data, allowing users summarize information along one or several dimensions at one time. Because these systems are optimized for user reporting, they often have interactive response TEAMFLY
times of less than 5 seconds.
Any well-designed OLAP system has time as a dimension, making it very useful for seeing trends over time. Trying to accomplish the same thing on a normalized data warehouse requires very complicated queries that are prone to error. To be most useful, OLAP systems should allow users to drill down to detail data for all reports. This capability ensures that all data is making it into the cubes, as well as giving users the ability to spot important patterns that may not appear in the dimensions.
As we have pointed out throughout this chapter, OLAP complements data mining. It is not a substitute for it. It provides better understanding of data, and the dimensions developed for OLAP can make data mining results more actionable. However, OLAP does not automatically find patterns in data.
OLAP is a powerful way to distribute information to many end users for advanced reporting needs. It provides the ability to let many more users base their decisions on data, instead of on hunches, educated guesses, and personal experience. OLAP complements undirected data mining techniques such as clustering. OLAP can provide the insight needed to find the business value in the identified clusters. It also provides a good visualization tool to use with other methods, such as decision trees and memory-based reasoning.
Data warehousing and data mining are not the same thing; however, they do complement each other, and data mining applications are often part of the data warehouse solution.
Team-Fly®
470643 c16.qxd 3/8/04 11:29 AM Page 513
C H A P T E R
16
Building the Data Mining
Environment
In the Big Rock Candy Mountains,
There’s a land that’s fair and bright,
Where the handouts grow on bushes
And you sleep out every night.
Where the boxcars all are empty
And the sun shines every day
And the birds and the bees
And the cigarette trees
The lemonade springs
Where the bluebird sings
In the Big Rock Candy Mountains.
Twentieth century hoboes had a vision of utopia, so why not twenty-first century data miners? For us, the vision is one of a company that puts the customer at the center of its operations and measures its actions by their effect on long-term customer value. In this ideal organization, business decisions are based on reliable information distilled from vast quantities of customer data. Needless to say, data miners—the people with the skills to turn all that data into the information needed to run the company—are held in great esteem.
This chapter starts with a utopian vision of a truly customer-centric organization with the ideal data mining environment to produce the information on which all decisions are based. Having a description of what the ideal data mining environment would look like is helpful for establishing more realistic near term goals. The chapter then goes on to look at the various components of the data mining environment—the staff, the data mining infrastructure, and the data mining software itself. Although we may not be able to achieve all elements of the utopian vision, we can use the vision to help create an environment suitable for successful data mining work.
513
470643 c16.qxd 3/8/04 11:29 AM Page 514
514 Chapter 16
A Customer-Centric Organization
Despite the familiar cliché that the customer is king, in most companies customers are not treated much like royalty. One reason is that most businesses are not organized around customers; they are organized around products.
Supermarkets, for example, have long been able to track the inventory levels of tens of thousands of products in order to keep the shelves well stocked, and they are able to calculate the profit margin on any item. But, until recently, these same stores knew nothing about individual customers—not their names, nor how many trips per month they make, nor what time of day they tend to shop, nor whether they use coupons, nor if they have children, nor what percent of the household’s shopping is done in this store, nor how close they live—nothing. We don’t mean to pick on supermarkets. Banks have been organized around loans; telephone companies have been organized around switches; airlines have been organized around operations. None have known much (or cared much) about customers.
In all of these industries, technology now makes it possible to shift the focus to customers. Such a shift is not easy; in fact, it is nothing short of revolutionary. By combining point-of-sale scanner data with a loyalty card program, a grocery retailer can, with a lot of effort, learn who is buying what and when they buy it, which customers are price-sensitive and which ones like to try new products, which ones like to bake from scratch and which ones prefer prepared meals, and so on. A telephone company can figure out who is making business calls and who is primarily chatting with friends. An online music store can make individualized recommendations of new music.
The harder challenge is being able to make effective use of this new ability to see customers in data. A truly customer-centric organization would be happy to continue offering an unprofitable service if the customers who use the loss-generating service spend more in other areas and therefore increase the profitability of the company as a whole. A customer-centric company does not have to ask the same questions every time a customer calls in. A customer-centric company judges a marketing campaign on the value customers generate over their lifetimes rather than on the initial response rate.
Becoming truly customer-centric means changing the corporate culture and the way everyone from top managers to call-center operators are rewarded. As long as each product line has a manager whose compensation is tied to the amount and margin of product sold, the company will remain focused on products rather than customers. In other words, the company is paying its managers to focus on products, and the managers are doing their jobs. In the ideal customer-centric organization, everyone is rewarded for increasing customer value and understands that this requires learning from each customer