Perception and Understanding of Social Annotations in Web Search
Venue
In Proc. of WWW2013, International World Wide Web Conferences Steering Committee, pp. 403-412
Publication Year
2013
Authors
BibTeX
Abstract
As web search increasingly becomes reliant on social signals, it is imperative for
us to understand the effect of these signals on users' behavior. There are multiple
ways in which social signals can be used in search: (a) to surface and rank
important social content; (b) to signal to users which results are more trustworthy
and important by placing annotations on search results. We focus on the latter
problem of understanding how social annotations affect user behavior. In previous
work, through eyetracking research we learned that users do not generally seem to
fixate on social annotations when they are placed at the bottom of the search
result block, with 11% probability of fixation [22]. A second eyetracking study
showed that placing the annotation on top of the snippet block might mitigate this
issue [22], but this study was conducted using mock-ups and with expert searchers.
In this paper, we describe a study conducted with a new eyetracking mix-method
using a live traffic search engine with the suggested design changes on real users
using the same experimental procedures. The study comprised of 11 subjects with an
average of 18 tasks per subject using an eyetrace-assisted retrospective
think-aloud protocol. Using a funnel analysis, we found that users are indeed more
likely to notice the annotations with a 60% probability of fixation (if the
annotation was in view). Moreover, we found no learning effects across search
sessions but found significant differences in query types, with subjects having a
lower chance of fixating on annotations for queries in the news category. In the
interview portion of the study, users reported interesting "wow" moments as well as
usefulness in recalling or re-finding content previously shared by oneself or
friends. The results not only shed light on how social annotations should be
designed in search engines, but also how users make use of social annotations to
make decisions about which pages are useful and potentially trustworthy.
