MuProp: Unbiased Backpropagation for Stochastic Neural Networks
Venue
International Conference on Learning Representations (2016)
Publication Year
2016
Authors
Shixiang Gu, Sergey Levine, Ilya Sutskever, Andriy Mnih
BibTeX
Abstract
Deep neural networks are powerful parametric models that can be trained efficiently
using the backpropagation algorithm. Stochastic neural networks combine the power
of large parametric functions with that of graphical models, which makes it
possible to learn very complex distributions. However, as backpropagation is not
directly applicable to stochastic networks that include discrete sampling
operations within their computational graph, training such networks remains
difficult. We present MuProp, an unbiased gradient estimator for stochastic
networks, designed to make this task easier. MuProp improves on the
likelihood-ratio estimator by reducing its variance using a control variate based
on the first-order Taylor expansion of a mean-field network. Crucially, unlike
prior attempts at using backpropagation for training stochastic networks, the
resulting estimator is unbiased and well behaved. Our experiments on structured
output prediction and discrete latent variable modeling demonstrate that MuProp
yields consistently good performance across a range of difficult tasks.
