470643 c01.qxd 3/8/04 11:08 AM Page 19
Why and What Is Data Mining?
19
And Just about Anything Else
These applications should give you a feel for what is possible using data mining, but they do not come close to covering the full range of applications. The data mining techniques described in this book have been used to find quasars, design army uniforms, detect second-press olive oil masquerading as “extra virgin,” teach machines to read aloud, and recognize handwritten letters. They will, no doubt, be used to do many of the things your business will require to grow and prosper for the rest of the century. In the next chapter, we turn to how businesses make effective use of data mining, using the virtuous cycle of data mining.
Lessons Learned
Data Mining is an important component of analytic customer relationship management. The goal of analytic customer relationship management is to recreate, to the extent possible, the intimate, learning relationship that a well-run small business enjoys with its customers. A company’s interactions with its customers generates large volumes of data. This data is initially captured in transaction processing systems such as automatic teller machines, telephone switch records, and supermarket scanner files. The data can then be collected, cleaned, and summarized for inclusion in a customer data warehouse. A well-designed customer data warehouse contains a historical record of customer interactions that becomes the memory of the corporation. Data mining tools can be applied to this historical record to learn things about customers that will allow the company to serve them better in the future. The chapter presented several examples of commercial applications of data mining such as better targeted couponing, making recommendations, cross selling, customer retention, and credit risk reduction.
Data mining itself is the process of finding useful patterns and rules in large volumes of data. This chapter introduced and defined six common data mining tasks: classification, estimation, prediction, affinity grouping, clustering, and profiling. The remainder of the book examines a variety of data mining algorithms and techniques that can be applied to these six tasks. To be successful, these techniques must become integral parts of a larger business process. That integration is the subject of the next chapter, The Virtuous Cycle of Data Mining.
470643 c01.qxd 3/8/04 11:08 AM Page 20
470643 c02.qxd 3/8/04 11:09 AM Page 21
C H A P T E R
2
The Virtuous Cycle
of Data Mining
In the first part of the nineteenth century, textile mills were the industrial success stories. These mills sprang up in the growing towns and cities along rivers in England and New England to harness hydropower. Water, running over water wheels, drove spinning, knitting, and weaving machines. For a century, the symbol of the industrial revolution was water driving textile machines.
The business world has changed. Old mill towns are now quaint historical curiosities. Long mill buildings alongside rivers are warehouses, shopping malls, artist studios and computer companies. Even manufacturing companies often provide more value in services than in goods. We were struck by an ad campaign by a leading international cement manufacturer, Cemex, that presented concrete as a service. Instead of focusing on the quality of cement, its price, or availability, the ad pictured a bridge over a river and sold the idea that
“cement” is a service that connects people by building bridges between them.
Concrete as a service? A very modern idea.
Access to electrical or mechanical power is no longer the criterion for success. For mass-market products, data about customer interactions is the new waterpower; knowledge drives the turbines of the service economy and, since the line between service and manufacturing is getting blurry, much of the manufacturing economy as well. Information from data focuses marketing efforts by segmenting customers, improves product designs by addressing real customer needs, and improves allocation of resources by understanding and predicting customer preferences.
21
470643 c02.qxd 3/8/04 11:09 AM Page 22
22
Chapter 2
Data is at the heart of most companies’ core business processes. It is generated by transactions in operational systems regardless of industry—retail, telecommunications, manufacturing, utilities, transportation, insurance, credit cards, and banking, for example. Adding to the deluge of internal data are external sources of demographic, lifestyle, and credit information on retail customers, and credit, financial, and marketing information on business customers. The promise of data mining is to find the interesting patterns lurking in all these billions and trillions of bytes. Merely finding patterns is not enough. You must respond to the patterns and act on them, ultimately turning data into information, information into action, and action into value. This is the virtuous cycle of data mining in a nutshell .
To achieve this promise, data mining needs to become an essential business process, incorporated into other processes including marketing, sales, customer support, product design, and inventory control. The virtuous cycle places data mining in the larger context of business, shifting the focus away from the discovery mechanism to the actions based on the discoveries.
Throughout this chapter and this book, we will be talking about actionable results from data mining (and this usage of “actionable” should not be confused with its definition in the legal domain, where it means that some action has grounds for legal action).
Marketing literature makes data mining seem so easy. Just apply the automated algorithms created by the best minds in academia, such as neural net
TEAMFLY
works, decision trees, and genetic algorithms, and you are on your way to untold successes. Although algorithms are important, the data mining solution is more than just a set of powerful techniques and data structures. The techniques have to be applied in the right areas, on the right data. The virtuous cycle of data mining is an iterative learning process that builds on results over time. Success in using data will transform an organization from reactive to proactive. This is the virtuous cycle of data mining, used by the authors for extracting maximum benefit from the techniques described later in the book.
This chapter opens with a brief case history describing an actual example of the application of data mining techniques to a real business problem. The case study is used to introduce the virtuous cycle of data mining. Data mining is presented as an ongoing activity within the business with the results of one data mining project becoming inputs to the next. Each project goes through four major stages, which together form one trip around the virtuous cycle.
Once these stages have been introduced, they are illustrated with additional case studies.
A Case Study in Business Data Mining
Once upon a time, there was a bank that had a business problem. One particular line of business, home equity lines of credit, was failing to attract good customers. There are several ways that a bank can attack this problem.
Team-Fly®
470643 c02.qxd 3/8/04 11:09 AM Page 23
The Virtuous Cycle of Data Mining
23
The bank could, for instance, lower interest rates on home equity loans. This would bring in more customers and increase market share at the expense of lowered margins. Existing customers might switch to the lower rates, further depressing margins. Even worse, assuming that the initial rates were reasonably competitive, lowering the rates might bring in the worst customers—the disloyal. Competitors can easily lure them away with slightly better terms.
The sidebar “Making Money or Losing Money” talks about the problems of retaining loyal customers.
In this example, Bank of America was anxious to expand its portfolio of home equity loans after several direct mail campaigns yielded disappointing results. The National Consumer Assets Group (NCAG) decided to use data mining to attack the problem, providing a good introduction to the virtuous cycle of data mining. (We would like to thank Larry Scroggins for allowing us to use material from a Bank of America Case Study he wrote. We also benefited from conversations with Bob Flynn, Lounette Dyer, and Jerry Modes, who at the time worked for Hyperparallel.)
Identifying the Business Challenge
BofA needed to do a better job of marketing home equity loans to customers.
Using common sense and business consultants, they came up with these insights:
■■
People with college-age children want to borrow against their home equity to pay tuition bills.
■■
People with high but variable incomes want to use home equity to smooth out the peaks and valleys in their income.
MAKING MONEY OR LOSING MONEY?
Home equity loans generate revenue for banks from interest payments on the loans, but sometimes companies grapple with services that lose money. As an example, Fidelity Investments once put its bill-paying service on the chopping block because this service consistently lost money. Some last-minute analysis saved it, though, by showing that Fidelity’s most loyal and most profitable customers used the bill paying service; although the bill paying service lost money, Fidelity made much more money on these customers’ other accounts.
After all, customers that trust their financial institution to pay their bills have a very high level of trust in that institution.