Periodic Measurement of Advertising Effectiveness Using Multiple-Test-Period Geo Experiments
Abstract
In a previous paper [6] we described the application of geo experiments to the
measurement of advertising effectiveness. One reason this method of measurement is
attractive is that it provides the rigor of a randomized experiment. However,
related decisions, such as where and how to spend advertising budget, are not
static. To address this issue, we extend this methodology to provide periodic
(ongoing) measurement of ad effectiveness. In this approach, the test and control
assignments of each geographic region rotate across multiple test periods, and
these rotations provide the opportunity to generate a sequence of measurements of
campaign effectiveness. The data across test periods can also be pooled to create a
single aggregate measurement of campaign effectiveness. These sequential and pooled
measurements have smaller confidence intervals than measurements from a series of
geo experiments with a single test period. Alternatively, the same confidence
interval can be achieved with a reduced magnitude and/or duration of ad spend
change, thereby lowering the cost of measurement. The net result is a better method
for periodic and isolated measurement of ad effectiveness.
