MRPSO: MapReduce Particle Swarm Optimization
Abstract
Abstract— In optimization problems involving large amounts
of data, such as web content, commercial transaction information,
or bioinformatics data, individual function evaluations may take
minutes or even hours. Particle Swarm Optimization (PSO) must
be parallelized for such functions. However, large-scale parallel
programs must communicate efficiently, balance work across all
processors, and address problems such as failed nodes.
We present MapReduce Particle Swarm Optimization
(MRPSO), a PSO implementation based on the MapReduce
parallel programming model. We describe MapReduce and show
how PSO can be naturally expressed in this model, without
explicitly addressing any of the details of parallelization. We
present a benchmark function for evaluating MRPSO and note
that MRPSO is not appropriate for optimizing easily evaluated
functions. We demonstrate that MRPSO scales to 256 processors
on moderately difficult problems and tolerates node failures.