Octopus: Evaluating Touchscreen Keyboard Correction and Recognition Algorithms via “Remulation”
Venue
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), ACM, New York, NY, USA, pp. 543-552
Publication Year
2013
Authors
Xiaojun Bi, Shiri Azenkot, Kurt Partridge, Shumin Zhai
BibTeX
Abstract
The time and labor demanded by a typical laboratory-based keyboard evaluation are
limiting resources for algorithmic adjustment and optimization. We propose
Remulation, a complementary method for evaluating touchscreen keyboard correction
and recognition algorithms. It replicates prior user study data through real-time,
on-device simulation. To demonstrate remulation, we have developed Octopus, an
evaluation tool that enables keyboard developers to efficiently measure and inspect
the impact of algorithmic changes without conducting resource-intensive user
studies. It can also be used to evaluate third-party keyboards in a “black box”
fashion, without access to their algorithms or source code. Octopus can evaluate
both touch keyboards and word-gesture keyboards. Two empirical examples show that
Remulation can efficiently and effectively measure many aspects of touch screen
keyboards at both macro and micro levels. Additionally, we contribute two new
metrics to measure keyboard accuracy at the word level: the Ratio of Error
Reduction (RER) and the Word Score.
