We present a simple and robust optimization algorithm related to genetic
algorithms, and with analogies to the popular CMA-ES search algorithm, that serves
as a cheap alternative to Bayesian Optimization. The algorithm is robust against
both monotonic transforms of the objective function value and affine
transformations of the feasible region. It is fast and easy to implement, and has
performance comparable to CMA-ES on a suite of benchmarks while spending less CPU
in the optimization algorithm, and can exhibit better overall performance than
Bayesian Optimization when the objective function is cheap.