Behavior Correlation for Lead Generation

We built a data warehouse specifically for generating high quality leads for the sales force of a major telecom company. Large telecom companies gather a great deal of information about the behavior of their current and former customers, such as the exact times they make purchases, how many minutes they use the phone, what the entire set of products they have purchased, how long they have been a customer, whether they have churned between telecom companies, what products the persons neighbors have purchased, and what products they have purchased from partner companies (such as airlines). Some of this information comes online, some comes from manual systems, and some comes from third party sources.

We built a system that correlated disparate behaviors to help determine what further products a particular customer was likely to be interested in purchasing. The system had over 50 behaviors tracked. We calculated the correlation coefficient and linear regression on a large number of the behavior pairs and found and many unexpected and valuable correlations. The system had to be very efficient as we had 4 terabytes of data that tracked most of the households in North America.

Web Analytics

Our customer has a high traffic search site on the Internet with over 2 million accesses per day. We implemented the Omniture web analytics tool on our customers web site to gather information about online activity including information regarding: unique visitors, product impressions, product clicks, common paths used during sessions, page effectiveness, file downloads, ROI for products purchased, sales campaign effectiveness, and referral statistics.

All this information had to be pulled back from Omniture and loaded into a data warehouse for custom reporting and long term analysis. We designed the warehouse (on Oracle), implemented the ETL (through Ascential DataStage), and created the reports (through Cognos, Plumtree and custom Java). The reports were delivered as portlets, emailed, and as part of applications.

Click Fraud

Driving traffic to a web site through affiliate click-through programs is very common and full of problems. Affiliates get paid for every user they direct to a site, but it can be very difficult to tell real users from automated traffic. Our customer wanted a system that could help differentiate human behavior from automated behavior so as to reduce payments to affiliates for traffic that was not driving sales.

We put in a data warehouse that tracked speed of accesses on a per affiliate, per IP address, per minute basis. We looked for situations where the accesses were unlikely to be performed by humans such as where access counts increased or decreased too rapidly, where the time between requests was too consistent, where large volumes of traffic came from anonymous proxy servers, where large volumes of traffic came from overseas (this customer was unlikely to have much useful overseas traffic), and where the pattern of behavior on each session was too consistent. Based on this analysis we were able to save the customer millions of dollars in affiliate payments and pinpointed rogue affiliates in the program.

Business Intelligence

We built a Decision Support System (DSS) for the marketing department of a company that publishes telephone directories on the Internet and in print. We completely designed the system along with the customer right from user requirements through to ETL, star schema design, and reporting. The customer wanted all the typical DSS reports including market segmentation. The customer also wanted many custom reports unique to their industry such as statistics on first time user experiences, category/heading usage, failed search statistics, and syndication performance statistics. Some of the technologies involved included Sybase, Oracle Warehouse Builder and the OMG’s Common Warehouse Metamodel (CWM) specification. We used the CWM to specify the warehouse metadata across all the layers of the system (ETL, star schema, and cube reporting).