Deep boosting
Venue
Proceedings of the Thirty-First International Conference on Machine Learning (ICML 2014)
Publication Year
2014
Authors
Corinna Cortes, Mehryar Mohri, Umar Syed
BibTeX
Abstract
We present a new ensemble learning algorithm, DeepBoost, which can use as base
classifiers a hypothesis set containing deep decision trees, or members of other
rich or complex families, and succeed in achieving high accuracy without
overfitting the data. The key to the success of the algorithm is a
capacity-conscious criterion for the selection of the hypotheses. We give new data-
dependent learning bounds for convex ensembles expressed in terms of the Rademacher
complexities of the sub-families composing the base classifier set, and the mixture
weight assigned to each sub-family. Our algorithm directly benefits from these
guarantees since it seeks to minimize the corresponding learning bound. We give a
full description of our algorithm, including the details of its derivation, and
report the results of several experiments showing that its performance compares
favorably to that of AdaBoost and Logistic Regression and their L1-regularized
variants.
