Team-Fly®
470643 c16.qxd 3/8/04 11:29 AM Page 523
Building the Data Mining Environment 523
Data mining may also seem unimportant to rapidly growing companies in a new market. In this situation, customer acquisition drives the business, and advertising, rather than direct marketing, is the principal way of attracting new customers. Applications for data mining in advertising are limited, and, at this stage in their development, companies are not yet focused on customer relationship management and customer retention. For the limited direct marketing they do, outsourced modeling is often sufficient.
Wireless communications, cable television, and Internet service providers all went through periods of exponential growth that have only recently come to an end as these markets matured (and before them, wired telephones, life insurance, catalogs, and credit cards went through similar cycles). During the initial growth phases, understanding customers may not be a worthwhile investment—an additional cell tower, switch, or whatever may provide better return. Eventually, though, the business and the customer base grow to a point where understanding the customers takes on increased importance. In our experience, it is better for companies to start early along the path of customer insight, rather than waiting until the need becomes critical.
Outsourcing Ongoing Data Mining
Even when a company has recognized the need for data mining, there is still the possibility of outsourcing. This is particularly true when the company is built around customer acquisition. In the United States, credit bureaus and household data suppliers are happy to provide modeling as a value added service with the data they sell. There are also direct marketing companies that handle everything from mailing lists to fulfillment—the actual delivery of products to customers. These companies often offer outsourced data mining.
Outsourcing arrangements have financial advantages for companies. The problem is that customer insight is being outsourced as well. A company that relies on outsourcing customers analytics runs the risk that customer understanding will be lost between the company and the vendor.
For instance,one company used direct mail for a significant proportion of its customer acquisition and outsourced the direct mail response modeling work to the mailing list vendors. Over the course of about 2 years, there were several direct mail managers in the company and the emphasis on this channel decreased. What no one had realized was that direct mail was driving acquisition that was being credited to other channels. Direct mail pieces could be filled in and returned by mail, in which case the new acquisition was credited to direct mail. However, the pieces also contained the company’s URL and a free phone number. Many prospects who received the direct mail found it more convenient to respond by phone or on the Web, often forgetting to provide the special code identifying them as direct mail prospects. Over time, the response attributed to direct mail decreased, and consequently the budget for
470643 c16.qxd 3/8/04 11:29 AM Page 524
524 Chapter 16
direct mail decreased as well. Only later, when decreased direct mail led to decreased responses in other channels, did the company realize that ignoring this echo effect had caused them to make a less-than-optimal business decision.
Insourcing Data Mining
The modeling process creates more then models and scores; it also produces insights. These insights often come during the process of data exploration and data preparation that is an important part of the data mining process. For that reason, we feel that any company with ongoing data mining needs should develop an in-house data mining group to keep the learning in the company.
Building an Interdisciplinary Data Mining Group
Once the decision has been made to bring customer understanding in-house, the question is where. In some companies, the data mining group has no permanent home. It consists of a group of people seconded from their usual jobs to come together to perform data mining. By its nature, such an arrangement seems temporary and often it is the result of some urgent requirement such as the need to understand a sudden upsurge in customer defaults. While it lasts, such a group can be very effective, but it is unlikely to last very long because the members will be recalled to their regular duties as soon as a new task requires their attention.
Building a Data Mining Group in IT
A possible home is in the systems group, since this group is often responsible for housing customer data and for running customer-facing operational systems. Because the data mining group is technical and needs access to data and powerful software and servers, the IT group seems like a natural location. In fact, analysis can be seen as an extension of providing databases and access tools and maintaining such systems.
Being part of IT has the advantage that the data mining group has access to hardware and data as needed, since the IT group has these technical resources and access to data. In addition, the IT group is a service organization with clients in many business units. In fact, the business units that are the “customers” for data mining are probably already used to relying on IT for data and reporting.
On the other hand, IT is sometimes a bit removed from the business problems that motivate customer analytics. Since very slight misunderstandings of the business problems can lead to useless results, it is very important that people from the business units be very closely involved with any IT-based data mining projects.
470643 c16.qxd 3/8/04 11:29 AM Page 525
Building the Data Mining Environment 525
Building a Data Mining Group in the Business Units
The alternative to putting the data mining group where the data and computers are is to put it close to the problems being addressed. That generally means the marketing group, the customer relationship management group (where such a thing exists), or the finance group. Sometimes there are several small data mining groups, one in each of several business units. A group in finance building credit risk models and collections models, one in marketing building response models, and one in CRM building cross-sell models and voluntary churn models.
The advantages and disadvantages of this approach are the inverse of those for putting data mining in IT. The business units have a great understanding of their own business problems, but may still have to rely on IT for data and computing resources. Although either approach can be successful, on balance we prefer to see data mining centered in the business units.
What to Look for in Data Mining Staff
The best data mining groups are often eclectic mixes of people. Because data mining has not existed very long as a separately named activity, there are few people who can claim to be trained data miners. There are data miners who used to be physicists, data miners who used to be geologists, data miners who used to be computer scientists, data miners who used to be marketing managers, data miners who used to be linguists, and data miners who are still statisticians.
This makes lunchtime conversation in a data mining group fairly interesting, but it doesn’t offer much guidance for hiring managers. The things that make good data miners better than mediocre ones are hard to teach and impossible to automate: good intuition, a feel for how to coax information out of data, and a natural curiosity.
No one indivdiual is likely to have all the skills required for completing a data mining project. Among them, the team members should cover the following:
■■
Database skills (SQL, if the data is stored in relational databases)
■■
Data transformation and programming skills (SAS, SPSS, S-Plus, PERL, other programming languages, ETL tools)
■■
Statistics
■■
Machine learning skills
■■
Industry knowledge in the relevant industry
■■
Data visualization skills
■■
Interviewing and requirements-gathering skills
■■
Presentation, writing, and communication skills
470643 c16.qxd 3/8/04 11:29 AM Page 526
526 Chapter 16
A new data mining group should include someone who has done commercial data mining before—preferably in the same industry. If necessary, this expertise can be provided by outside consultants.
Data Mining Infrastructure
In companies where data mining is merely an exploratory activity, useful data mining can be accomplished with little infrastructure. A desktop workstation with some data mining software and access to the corporate databases is likely to be sufficient. However, when data mining is central to the business, the data mining infrastructure must be considerably more robust. In these companies, updating customer profiles with new model scores either on a regular schedule such as once a month or, in some cases with each new transaction, is part of the regular production process of the data warehouse. The data mining infrastructure must provide a bridge between the exploratory world where models are developed and the production world where models are scored and marketing campaigns run.
A production-ready data mining environment must be able to support the following:
■■
The ability to access data from many sources and bring the data
together as customer signatures in a data mining model set.
■■