Virtual Adversarial Training for Semi-Supervised Text Classification
Venue
arXiv preprint (2016)
Publication Year
2016
Authors
Takeru Miayto, Andrew M. Dai, Ian Goodfellow
BibTeX
Abstract
Adversarial training provides a means of regularizing supervised learning
algorithms while virtual adversarial training is able to extend supervised learning
algorithms to the semi-supervised setting. However, both methods require making
small perturbations to numerous entries of the input vector, which is inappropriate
for sparse high-dimensional inputs such as one-hot word representations. We extend
adversarial and virtual adversarial training to the text domain by applying
perturbations to the word embeddings in a recurrent neural network rather than to
the original input itself. The proposed method achieves state of the art results on
multiple benchmark semi-supervised and purely supervised tasks. We provide
visualizations and analysis showing that the learned word embeddings have improved
in quality and that while training, the model is less prone to overfitting.
