Cardinal Contests
Abstract
Contests are widely used as a means for effort elicitation
in settings ranging from government R&D contests to online
crowdsourcing contests on platforms such as Kaggle, Innocentive, or TopCoder. Such rank-order mechanisms——
where agents' rewards depend only on the relative ranking
of their submissions' qualities——are natural mechanisms for
incentivizing effort when it is easier to obtain ordinal, rather than cardinal, information about agents' outputs, or where absolute measures of quality are unverifiable. An increasing number of online contests, however, rank entries according to some numerical evaluation of their absolute quality——for instance, the performance of an algorithm on a test dataset, or the performance of an intervention in a randomized trial. Can the contest designer incentivize higher effort by making the rewards in an ordinal rank-order mechanism contingent on such cardinal information?
We model and analyze cardinal contests, where a principal
running a rank-order tournament has access to an absolute
measure of the qualities of agents' submissions in addition
to their relative rankings, and ask how modifying the
rank-order tournament to incorporate cardinal information
can improve incentives for effort. Our main result is that a simple threshold mechanism——a mechanism that awards the
prize for a rank if and only if the absolute quality of the
agent at that rank exceeds a certain threshold——is optimal
amongst all mixed cardinal-ordinal mechanisms where the
fraction of the j-th prize awarded to the j-th-ranked agent
is any arbitrary non-decreasing function of her submission's quality. Further, the optimal threshold mechanism uses exactly the same threshold for each rank. We study what contest parameters determine the extent of the benefit from incorporating such cardinal information into an ordinal rank-order contest, and investigate the extent of improvement in equilibrium effort via numerical simulations.