Names 555
Addresses 555
Free Text
556
Binary Data (Audio, Image, Etc.)
557
Data for Data Mining
557
Constructing the Customer Signature
558
Cataloging the Data
559
Identifying the Customer
560
First Attempt
562
Identifying the Time Frames
562
Taking a Recent Snapshot
562
Pivoting Columns
563
Calculating the Target
563
Making Progress
564
Practical Issues
564
Exploring Variables
565
Distributions Are Histograms
565
Changes over Time
566
Crosstabulations 567
Deriving Variables
568
Extracting Features from a Single Value
569
Combining Values within a Record
569
Looking Up Auxiliary Information
569
Pivoting Regular Time Series
572
Summarizing Transactional Records
574
Summarizing Fields across the Model Set
574
470643 ftoc.qxd 3/8/04 11:33 AM Page xviii
xviii Contents
Examples of Behavior-Based Variables
575
Frequency of Purchase
575
Declining Usage
577
Revolvers, Transactors, and Convenience Users:
Defining Customer Behavior
580
Data 581
Segmenting by Estimating Revenue
581
Segmentation by Potential
583
Customer Behavior by Comparison to Ideals
585
The Ideal Convenience User
587
The Dark Side of Data
590
Missing Values
590
Dirty Data
592
Inconsistent Values
593
Computational Issues
594
Source Systems
594
Extraction Tools
595
Special-Purpose Code
595
Data Mining Tools
595
Lessons Learned
596
Chapter 18 Putting Data Mining to Work
597
Getting Started
598
What to Expect from a Proof-of-Concept Project
599
Identifying a Proof-of-Concept Project
599
Implementing the Proof-of-Concept Project
601
Act on Your Findings
602
Measure the Results of the Actions
603
Choosing a Data Mining Technique
605
Formulate the Business Goal as a Data Mining Task
605
Determine the Relevant Characteristics of the Data
606
Data Type
606
Number of Input Fields
607
Free-Form Text
607
Consider Hybrid Approaches
608
How One Company Began Data Mining
608
A Controlled Experiment in Retention
609
The Data
611
The Findings
613
The Proof of the Pudding
614
Lessons Learned
614
Index
615
470643 c01.qxd 3/8/04 11:08 AM Page 1
C H A P T E R
1
Why and What Is Data Mining?
In the first edition of this book, the first sentence of the first chapter began with the words “Somerville, Massachusetts, home to one of the authors of this book,
. . .” and went on to tell of two small businesses in that town and how they had formed learning relationships with their customers. In the intervening years, the little girl whose relationship with her hair braider was described in the chapter has grown up and moved away and no longer wears her hair in cornrows. Her father has moved to nearby Cambridge. But one thing has not changed. The author is still a loyal customer of the Wine Cask, where some of the same people who first introduced him to cheap Algerian reds in 1978 and later to the wine-growing regions of France are now helping him to explore Italy and Germany.
After a quarter of a century, they still have a loyal customer. That loyalty is no accident. Dan and Steve at the Wine Cask learn the tastes of their customers and their price ranges. When asked for advice, their response will be based on their accumulated knowledge of that customer’s tastes and budgets as well as on their knowledge of their stock.
The people at The Wine Cask know a lot about wine. Although that knowledge is one reason to shop there rather than at a big discount liquor store, it is their intimate knowledge of each customer that keeps people coming back.
Another wine shop could open across the street and hire a staff of expert oenophiles, but it would take them months or years to achieve the same level of customer knowledge.
1
470643 c01.qxd 3/8/04 11:08 AM Page 2
2
Chapter 1
Well-run small businesses naturally form learning relationships with their customers. Over time, they learn more and more about their customers, and they use that knowledge to serve them better. The result is happy, loyal customers and profitable businesses. Larger companies, with hundreds of thousands or millions of customers, do not enjoy the luxury of actual personal relationships with each one. These larger firms must rely on other means to form learning relationships with their customers. In particular, they must learn to take full advantage of something they have in abundance—the data produced by nearly every customer interaction. This book is about analytic techniques that can be used to turn customer data into customer knowledge.
Analytic Customer Relationship Management
It is widely recognized that firms of all sizes need to learn to emulate what small, service-oriented businesses have always done well—creating one-to-one relationships with their customers. Customer relationship management is a broad topic that is the subject of many books and conferences. Everything from lead-tracking software to campaign management software to call center software is now marketed as a customer relationship management tool. The focus of this book is narrower—the role that data mining can play in improv
TEAMFLY
ing customer relationship management by improving the firm’s ability to form learning relationships with its customers.
In every industry, forward-looking companies are moving toward the goal of understanding each customer individually and using that understanding to make it easier for the customer to do business with them rather than with competitors. These same firms are learning to look at the value of each customer so that they know which ones are worth investing money and effort to hold on to and which ones should be allowed to depart. This change in focus from broad market segments to individual customers requires changes throughout the enterprise, and nowhere more than in marketing, sales, and customer support.
Building a business around the customer relationship is a revolutionary change for most companies. Banks have traditionally focused on maintaining the spread between the rate they pay to bring money in and the rate they charge to lend money out. Telephone companies have concentrated on connecting calls through the network. Insurance companies have focused on processing claims and managing investments. It takes more than data mining to turn a product-focused organization into a customer-centric one. A data mining result that suggests offering a particular customer a widget instead of a gizmo will be ignored if the manager’s bonus depends on the number of gizmos sold this quarter and not on the number of widgets (even if the latter are more profitable).
Team-Fly®
470643 c01.qxd 3/8/04 11:08 AM Page 3
Why and What Is Data Mining?
3
In the narrow sense, data mining is a collection of tools and techniques. It is one of several technologies required to support a customer-centric enterprise.
In a broader sense, data mining is an attitude that business actions should be based on learning, that informed decisions are better than uninformed decisions, and that measuring results is beneficial to the business. Data mining is also a process and a methodology for applying the tools and techniques. For data mining to be effective, the other requirements for analytic CRM must also be in place. In order to form a learning relationship with its customers, a firm must be able to:
■■
Notice what its customers are doing
■■
Remember what it and its customers have done over time
■■
Learn from what it has remembered
■■
Act on what it has learned to make customers more profitable Although the focus of this book is on the third bullet—learning from what has happened in the past—that learning cannot take place in a vacuum. There must be transaction processing systems to capture customer interactions, data warehouses to store historical customer behavior information, data mining to translate history into plans for future action, and a customer relationship strategy to put those plans into practice.
The Role of Transaction Processing Systems
A small business builds relationships with its customers by noticing their needs, remembering their preferences, and learning from past interactions how to serve them better in the future. How can a large enterprise accomplish something similar when most company employees may never interact personally with customers? Even where there is customer interaction, it is likely to be with a different sales clerk or anonymous call-center employee each time, so how can the enterprise notice, remember, and learn from these interactions? What can replace the creative intuition of the sole proprietor who recognizes customers by name, face, and voice, and remembers their habits and preferences?
In a word, nothing. But that does not mean that we cannot try. Through the clever application of information technology, even the largest enterprise can come surprisingly close. In large commercial enterprises, the first step—noticing what the customer does—has already largely been automated. Transaction processing systems are everywhere, collecting data on seemingly everything. The records generated by automatic teller machines, telephone switches, Web servers, point-of-sale scanners, and the like are the raw material for data mining.
These days, we all go through life generating a constant stream of transaction records. When you pick up the phone to order a canoe paddle from L.L.
470643 c01.qxd 3/8/04 11:08 AM Page 4
4
Chapter 1
Bean or a satin bra from Victoria’s Secret, a call detail record is generated at the local phone company showing, among other things, the time of your call, the number you dialed, and the long-distance company to which you have been connected. At the long-distance company, similar records are generated recording the duration of your call and the exact routing it takes through the switching system. This data will be combined with other records that store your billing plan, name, and address in order to generate a bill. At the catalog company, your call is logged again along with information about the particular catalog from which you ordered and any special promotions you are responding to. When the customer service representative that answered your call asks for your credit card number and expiration date, the information is immediately relayed to a credit card verification system to approve the transaction; this too creates a record. All too soon, the transaction reaches the bank that issued your credit card, where it appears on your next monthly statement.