Even with a single product, the message can be important. A classic example is the trade-off between price and convenience. Some people are very price sensitive, and willing to shop in warehouses, make their phone calls late at night, always change planes, and arrange their trips to include a Saturday night. Others will pay a premium for the most convenient service. A message
470643 c04.qxd 3/8/04 11:10 AM Page 90
90
Chapter 4
based on price will not only fail to motivate the convenience seekers, it runs the risk of steering them toward less profitable products when they would be happy to pay more.
This chapter describes how simple, single-campaign response models can be combined to create a best next offer model that matches campaigns to customers. Collaborative filtering, an approach to grouping customers into like-minded segments that may respond to similar offers, is discussed in Chapter 8.
Data Mining to Choose the Right Place to Advertise
One way of targeting prospects is to look for people who resemble current customers. For instance, through surveys, one nationwide publication determined that its readers have the following characteristics:
■■
59 percent of readers are college educated.
■■
46 percent have professional or executive occupations.
■■
21 percent have household income in excess of $75,000/year.
■■
7 percent have household income in excess of $100,000/year.
Understanding this profile helps the publication in two ways: First, by targeting prospects who match the profile, it can increase the rate of response to its own promotional efforts. Second, this well-educated, high-income readership can be used to sell advertising space in the publication to companies wishing to reach such an audience. Since the theme of this section is targeting prospects, let’s look at how the publication used the profile to sharpen the focus of its prospecting efforts. The basic idea is simple. When the publication wishes to advertise on radio, it should look for stations whose listeners match the profile. When it wishes to place “take one” cards on store counters, it should do so in neighborhoods that match the profile. When it wishes to do outbound telemarketing, it should call people who match the profile. The data mining challenge was to come up with a good definition of what it means to match the profile.
Who Fits the Profile?
One way of determining whether a customer fits a profile is to measure the similarity—which we also call distance—between the customer and the profile. Several data mining techniques use this idea of measuring similarity as a distance. Memory-based reasoning, discussed in Chapter 8, is a technique for classifying records based on the classifications of known records that
470643 c04.qxd 3/8/04 11:10 AM Page 91
Data Mining Applications
91
are “in the same neighborhood.” Automatic cluster detection, the subject of Chapter 11, is another data mining technique that depends on the ability to calculate a distance between two records in order to find clusters of similar records close to each other.
For this profiling example, the purpose is simply to define a distance metric to determine how well prospects fit the profile. The data consists of survey results that represent a snapshot of subscribers at a particular time. What sort of measure makes sense with this data? In particular, what should be done about the fact that the profile is expressed in terms of percentages (58 percent are college educated; 7 percent make over $100,000), whereas an individual either is or is not college educated and either does or does not make more than $100,000?
Consider two survey participants. Amy is college educated, earns $80,000/year, and is a professional. Bob is a high-school graduate earning $50,000/year. Which one is a better match to the readership profile? The answer depends on how the comparison is made. Table 4.1 shows one way to develop a score using only the profile and a simple distance metric.
This table calculates a score based on the proportion of the audience that agrees with each characteristic. For instance, because 58 percent of the readership is college educated, Amy gets a score of 0.58 for this characteristic. Bob, who did not graduate from college, gets a score of 0.42 because the other 42 percent of the readership presumably did not graduate from college. This is continued for each characteristic, and the scores are added together.
Amy ends with a score of 2.18 and Bob with the higher score of 2.68. His higher score reflects the fact that he is more similar to the profile of current readers than is Amy.
Table 4.1 Calculating Fitness Scores for Individuals by Comparing Them along Each Demographic Measure
READER YES
NO
AMY
BOB
SHIP
SCORE SCORE AMY BOB SCORE SCORE
College 58%
0.58
0.42
YES
NO
0.58
0.42
educated
Prof or exec
46%
0.46
0.54
YES
NO
0.46
0.54
Income >$75K
21%
0.21
0.79
YES
NO
0.21
0.79
Income >$100K
7%
0.07
0.93
NO
NO
0.93
0.93
Total
2.18
2.68
470643 c04.qxd 3/8/04 11:10 AM Page 92
92
Chapter 4
The problem with this approach is that while Bob looks more like the profile than Amy does, Amy looks more like the audience the publication has targeted—namely, college-educated, higher-income individuals. The success of this targeting is evident from a comparison of the readership profile with the demographic characteristics of the U.S. population as a whole. This suggests a less naive approach to measuring an individual’s fit with the publication’s audience by taking into account the characteristics of the general population in addition to the characteristics of the readership. The approach measures the extent to which a prospect differs from the general population in the same ways that the readership does.
Compared to the population, the readership is better educated, more professional, and better paid. In Table 4.2, the “Index” columns compare the readership’s characteristics to the entire population by dividing the percent of the readership that has a particular attribute by the percent of the population that has it. Now, we see that the readership is almost three times more likely to be college educated than the population as a whole. Similarly, they are only about half as likely not to be college educated. By using the indexes as scores for each characteristic, Amy gets a score of 8.42 (2.86 + 2.40 + 2.21 + 0.95) versus Bob with a score of only 3.02 (0.53 + 0.67 + 0.87 + 0.95). The scores based on indexes correspond much better with the publication’s target audience. The new scores make more sense because they now incorporate the additional information TEAMFLY
about how the target audience differs from the U.S. population as a whole.
Table 4.2 Calculating Scores by Taking the Proportions in the Population into Account YES
NO
READER
US READER
US
SHIP
POP
INDEX
SHIP
POP
INDEX
College 58%
20.3%
2.86
42%
79.7%
0.53
educated
Prof or exec
46%
19.2%
2.40
54%
80.8%
0.67
Income >$75K
21%
9.5%
2.21
79%
90.5%
0.87
Income >$100K
7%
2.4%
2.92
93%
97.6%
0.95
Team-Fly®
470643 c04.qxd 3/8/04 11:10 AM Page 93
Data Mining Applications
93
T I P When comparing customer profiles, it is important to keep in mind the profile of the population as a whole. For this reason, using indexes is often better than using raw values.
Chapter 11 describes a related notion of similarity based on the difference between two angles. In that approach, each measured attribute is considered a separate dimension. Taking the average value of each attribute as the origin, the profile of current readers is a vector that represents how far he or she differs from the larger population and in what direction. The data representing a prospect is also a vector. If the angle between the two vectors is small, the prospect differs from the population in the same direction.
Measuring Fitness for Groups of Readers
The idea behind index-based scores can be extended to larger groups of people. This is important because the particular characteristics used for measuring the population may not be available for each customer or prospect. Fortunately, and not by accident, the preceding characteristics are all demographic characteristics that are available through the U.S. Census and can be measured by geographical divisions such as census tract (see the sidebar, “Data by Census Tract”).
The process here is to rate each census tract according to its fitness for the publication. The idea is to estimate the proportion of each census tract that fits the publication’s readership profile. For instance, if a census tract has an adult population that is 58 percent college educated, then everyone in it gets a fitness score of 1 for this characteristic. If 100 percent are college educated, then the score is still 1—a perfect fit is the best we can do. If, however, only 5.8 percent graduated from college, then the fitness score for this characteristic is 0.1.
The overall fitness score is the average of the individual scores for each characteristic.
Figure 4.1 provides an example for three census tracts in Manhattan. Each tract has a different proportion of the four characteristics being considered.