Cardinal Contests
Venue
Proceedings of the 24th International Conference on the World Wide Web (WWW) (2015), pp. 377-387
Publication Year
2015
Authors
Arpita Ghosh, Patrick Hummel
BibTeX
Abstract
Contests are widely used as a means for effort elicitation in settings ranging from
government R&D contests to online crowdsourcing contests on platforms such as
Kaggle, Innocentive, or TopCoder. Such rank-order mechanisms—— where agents'
rewards depend only on the relative ranking of their submissions' qualities——are
natural mechanisms for incentivizing effort when it is easier to obtain ordinal,
rather than cardinal, information about agents' outputs, or where absolute measures
of quality are unverifiable. An increasing number of online contests, however, rank
entries according to some numerical evaluation of their absolute quality——for
instance, the performance of an algorithm on a test dataset, or the performance of
an intervention in a randomized trial. Can the contest designer incentivize higher
effort by making the rewards in an ordinal rank-order mechanism contingent on such
cardinal information? We model and analyze cardinal contests, where a principal
running a rank-order tournament has access to an absolute measure of the qualities
of agents' submissions in addition to their relative rankings, and ask how
modifying the rank-order tournament to incorporate cardinal information can improve
incentives for effort. Our main result is that a simple threshold mechanism——a
mechanism that awards the prize for a rank if and only if the absolute quality of
the agent at that rank exceeds a certain threshold——is optimal amongst all mixed
cardinal-ordinal mechanisms where the fraction of the j-th prize awarded to the
j-th-ranked agent is any arbitrary non-decreasing function of her submission's
quality. Further, the optimal threshold mechanism uses exactly the same threshold
for each rank. We study what contest parameters determine the extent of the benefit
from incorporating such cardinal information into an ordinal rank-order contest,
and investigate the extent of improvement in equilibrium effort via numerical
simulations.
