The goal of the data mining effort was to identify groups of subscribers with an unusually high likelihood to cancel their subscriptions in the next 60 days.
The data mining tool employed used a rule induction algorithm similar to decision trees to create segments of high-risk customers described by simple rules. The plan was to include these high-risk customers in telemarketing campaigns aimed at retaining them. The retention offers were to be tailored to different customer segments discovered through data mining. The experimental design allowed for the comparison of three groups:
■■
Group A consists of customers judged by the model to be high risk for whom no intervention was performed.
■■
Group B consists of customers judged by the model to be high risk for whom some intervention was performed.
■■
Group C is representative of the general population of customers.
The study design is illustrated in Figure 18.2. Our hope, of course, was that group A would suffer high attrition compared to groups B and C, proving that both the model and the intervention were effective.
Here the project ran into a little trouble. The first difficulty was that although the project included a budget for outbound telemarketing calls to the people identified as likely to cancel, there was neither budget nor authorization to actually offer anything to the people being called. Another problem was a technical problem in the call center. It was not possible to transfer a dissatisfied customer directly over to the customer service group at the phone company to resolve particular problems outside the scope of the retention effort (such as mistakes on bills). Yet another problem was that although the customer database included a home phone number for each customer, only about 75 percent of them turned out to be correct.
470643 c18.qxd 3/8/04 11:31 AM Page 611
Putting Data Mining to Work 611
Study
Population
At Risk
Test Group
Control Group
Entire Subscriber Base
Figure 18.2 Study design for the analytic customer relationship marketing test.
In the end, the outbound telemarketing company simply called people from the test and control groups and asked them a series of questions designed to elicit their level of satisfaction and volunteered to refer any problems reported to customer service. Despite this rather lame intervention, 60-day retention was significantly better for the test group than for the control group. Apparently, just showing that the company cared enough to call was enough to decrease churn.
The Data
In the course of several interviews with the client, we identified two sources of data for use in the pilot. The first source was a customer profile database that had already been set up by a database marketing company. This database contained summary information for each subscriber including the billing plan, type of phone, local minutes of use by month, roaming minutes of use by month, number of calls to and from each identified cellular market in the United States, and dozens of other similar fields.
The second source was call detail data collected from the wireless switches.
Each time a mobile phone is switched on, it begins a two-way conversation with nearby cell sites. The cell sites relay data from the telephone such as the
470643 c18.qxd 3/8/04 11:31 AM Page 612
612 Chapter 18
serial number and phone type to a central switching office. Computers at the switching office figure out which cell site the phone should be talking to at the moment and send a message back to the phone telling it which cell it is using and what frequency to tune to.
When the subscriber enters a phone number and presses the send button, the number is relayed to the central switch, which in turn sets up the call over regular land lines or relays it to the cell closest to another wireless subscriber.
Every switch generates a call detail record that includes the subscriber ID, the originating number, the number called, the originating cell, the call duration, the call termination reason, and so on. These call detail records were used to generate a behavioral profile of each customer, including such things as the number of distinct numbers called and the proportion of calls by time of day and day of week.
The pilot project used 6 months of data for around 50,000 subscribers some of whom canceled their accounts and some of whom did not. Our original intention was to merge the two data sources so that a given subscriber’s data from the marketing database (billing plan, tenure, type of phone, total minutes of use, home town, and so on) would be linked to the detail records for each of his or her calls. That way, a single model could be built based on independent variables from both sources. For technical reasons, this proved difficult, so due to time and budgetary constraints we ended up building two separate models, TEAMFLY
one based on the marketing data and one based on call detail data.
The marketing data was already summarized at the customer level and stored in an easily accessible database system. Getting the call detail data into a usable form was more challenging. Each switch had its own collection of reel-to-reel tapes like the ones used to represent computers in 1960s movies. These tapes were continuously recycled so that a 90-day moving window was always current with the tapes from 90 days earlier being used to record the current day’s calls. Since eight tapes were written every day, we found ourselves looking at over 700 tape reels, each of which had to be loaded individually by hand into a borrowed 9-track tape drive. Once loaded, the call detail data, which was written in an arcane format unique to the switching equipment, needed extensive preprocessing in order to be made ready for analysis. The 70 million call detail records were reduced to 10 million by filtering out records that did not relate to calls to or from the churn model population of around.
Even before predictive modeling began, simple profiling of the call detail data suggested many possible avenues for increasing profitability. Once call detail was available in a queryable form, it became possible to answer questions such as:
■■
Are subscribers who make many short calls more or less loyal than those who make fewer, longer calls?
■■
Do dropped calls lead to calls to customer service?
Team-Fly®
470643 c18.qxd 3/8/04 11:31 AM Page 613
Putting Data Mining to Work 613
■■
What is the size of a subscriber’s “calling circle” for both mobile-tomobile and mobile-to-fixed-line calling?
■■
How does a subscriber’s usage vary from hour to hour, month to
month, and weekday to weekend?
■■
Does the subscriber call any radio station call-in lines?
■■
How often does a subscriber call voice mail?
■■
How often does a subscriber call customer service?
The answers to these and many other questions suggested a number of marketing initiatives to stimulate cellular phone use at particular times and in particular ways. Furthermore, as we had hoped, variables built around measures constructed from the call detail, such as size of calling circle, proved to be highly predictive of churn.
The Findings
Data mining isolated several customer segments at high risk for churn. Some of these were more actionable than others. For example, it turned out that subscribers who, judging by where their calls entered the network, commuted to New York were much more likely to churn than subscribers who commuted to Philadelphia. This was a coverage issue. Customers who lived in the Comcast coverage area and commuted to New York, found themselves roaming (making use of another company’s network) for most of every work day. The billing plans in effect at that time made roaming very expensive. Commuters to Philadelphia remained within the Comcast coverage area for their entire commute and work day and so incurred no roaming charges. This problem was not very actionable because neither changing the coverage area nor changing the rules governing rate plans was within the power of the sponsors of the study, although the information could be used by other parts of the business.
A potentially more actionable finding was that customers whose calling patterns did not match their rate plan were at high risk for churn. There are two ways that a customer’s calling behavior may be inappropriate for his or her rate plan. One segment of customers pays for more minutes than they actually use. Arguably, a wireless company might be able to increase the lifetime value of these customers by moving them to a lower rate plan. They would be worth less each month, but might last longer. The only way to find out for sure would be with a marketing test. After all, customers might accept the offer to pay less each month, but still churn at the same rate. Or, the rate of churn might be lowered, but not enough to make up for the loss in near-term revenue.
The other type of mismatch between calling behavior and rate plan occurs when subscribers sign up for a low-cost rate plan that does not include many minutes of use and find themselves frequently using more minutes than the