Posterior Sparsity in Dependency Grammar Induction
Venue
Journal of Machine Learning Research, vol. 12 (2011), pp. 455-490
Publication Year
2011
Authors
Jennifer Gillenwater, Kuzman Ganchev, Joao Graca, Fernando Pereira, Ben Taskar
BibTeX
Abstract
A strong inductive bias is essential in unsupervised grammar induction. In this
paper, we explore a particular sparsity bias in dependency grammars that encourages
a small number of unique dependency types. We use part-of-speech (POS) tags to
group dependencies by parent-child types and investigate sparsity-inducing
penalties on the posterior distributions of parent-child POS tag pairs in the
posterior regularization (PR) framework of Graça et al. (2007). In experiments with
12 different languages, we achieve significant gains in directed attachment
accuracy over the standard expectation maximization (EM) baseline, with an average
accuracy improvement of 6.5%, outperforming EM by at least 1% for 9 out of 12
languages. Furthermore, the new method outperforms models based on standard
Bayesian sparsity-inducing parameter priors with an average improvement of 5% and
positive gains of at least 1% for 9 out of 12 languages. On English text in
particular, we show that our approach improves performance over other
state-of-the-art techniques.
