In this paper, we make the following contributions: (1) a comparison of the quality
and efficacy of annotations for medical relation extraction provided by both crowd
and medical experts, showing that crowd annotations are equivalent to those of
experts, with appropriate processing; (2) an openly available dataset of 900
English sentences for medical relation extraction, centering primarily on the cause
relation, that have been processed with disagreement analysis and by experts.