Deep Networks With Large Output Spaces
Venue
International Conference on Learning Representations (2015)
Publication Year
2015
Authors
Sudheendra Vijayanarasimhan, Jonathon Shlens, Rajat Monga, Jay Yagnik
BibTeX
Abstract
Deep neural networks have been extremely successful at various image, speech, video
recognition tasks because of their ability to model deep structures within the
data. However, they are still prohibitively expensive to train and apply for
problems containing millions of classes in the output layer. Based on the
observation that the key computation common to most neural network layers is a
vector/matrix product, we propose a fast locality-sensitive hashing technique to
approximate the actual dot product enabling us to scale up the training and
inference to millions of output classes. We evaluate our technique on three diverse
large-scale recognition tasks and show that our approach can train large-scale
models at a faster rate (in terms of steps/total time) compared to baseline
methods.
