Learning Deep Models of Optimization Landscapes
Abstract
In all but the most trivial optimization problems,
the structure of the solutions exhibit complex interdependencies
between the input parameters. Decades of research with
stochastic search techniques has shown the benefit of explicitly
modeling the interactions between sets of parameters and the
overall quality of the solutions discovered. We demonstrate a
novel method, based on learning deep networks, to model the
global landscapes of optimization problems. To represent the
search space concisely and accurately, the deep networks must
encode information about the underlying parameter interactions
and their contributions to the quality of the solution. Once
the networks are trained, the networks are probed to reveal
parameter combinations with high expected performance with
respect to the optimization task. These estimates are used to
initialize fast, randomized, local-search algorithms, which in
turn expose more information about the search space that is
subsequently used to refine the models. We demonstrate the
technique on multiple problems that have arisen in a variety of
real-world domains, including: packing, graphics, job scheduling,
layout and compression. Strengths, limitations, and extensions of
the approach are extensively discussed and demonstrated.