Evaluating Online Ad Campaigns in a Pipeline: Causal Models at Scale
Venue
Proceedings of ACM SIGKDD 2010, pp. 7-15
Publication Year
2010
Authors
David Chan, Rong Ge, Ori Gershony, Tim Hesterberg, Diane Lambert
BibTeX
Abstract
Display ads proliferate on the web, but are they effective? Or are they irrelevant
in light of all the other advertising that people see? We describe a way to answer
these questions, quickly and accurately, without randomized experiments, surveys,
focus groups or expert data analysts. Doubly robust estimation protects against the
selection bias that is inherent in observational data, and a nonparametric test
that is based on irrelevant outcomes provides further defense. Simulations based on
realistic scenarios show that the resulting estimates are more robust to selection
bias than traditional alternatives, such as regression modeling or propensity
scoring. Moreover, computations are fast enough that all processing, from data
retrieval through estimation, testing, validation and report generation, proceeds
in an automated pipeline, without anyone needing to see the raw data.
