How the Order of Response Options in a Running Tally Can Affect Online Survey Estimates
Venue
Papers Presented at the 64th Annual Conference of the American Association for Public Opinion Research (AAPOR), AMSTAT (2011), pp. 5582-5585
Publication Year
2011
Authors
Tom Wells, Mario Callegaro, Charles DiSogra
BibTeX
Abstract
In the design of online surveys, running tallies or constant sums are used to help
respondents sum up the allocation of amounts so that the total sums to 100%. We
hypothesized that for time allocation, the order of the presentation of the time
categories could make a difference in the distribution of reported time spent. We
expected primacy effects, with the first-presented time category having a higher
allocation of time than the later-presented options. An experiment was conducted
with a general population adult sample from KnowledgePanel®. In the experiment,
respondents were asked to provide running tallies of the percentage of television
they typically watch during the morning, afternoon, and evening (separately for
weekdays and weekends). The order of the categories was rotated. Primacy effects
were detected, however differences by position were small and not statistically
significant. Because time spent watching TV is a regular activity, viewing patterns
are more likely to be encoded or ingrained in memory, and more likely to be
reported reliably, with responses less susceptible to order effects.
