We introduce confidence-weighted linear classifiers, which add parameter confidence
information to linear classifiers. Online learners in this setting update both
classifier parameters and the estimate of their confidence. The particular online
algorithms we study here maintain a Gaussian distribution over parameter vectors
and update the mean and covariance of the distribution with each instance.
Empirical evaluation on a range of NLP tasks show that our algorithm improves over
other state of the art online and batch methods, learns faster in the online
setting, and lends itself to better classifier combination after parallel training.