Comparison of 31 algorithms from BBOB-2009 on bbob suite

ECDF of runtimes for 31 algorithms on the bbob suite

Empirical runtime distributions (runtime in number of function evaluations divided by dimension) on all functions with target values in {100, … , 1e-8} in dimension 10. The cross indicates the maximum number of function evaluations. A decline in steepness right after the cross (e.g. for IPOP-SEP-CMA-ES) indicates that the maximum number of function evaluations should have been chosen larger. A steep increase right after the cross (e.g. for simple GA) indicates that a restart should have been invoked earlier.

The plot stems from the BBOB-2010 paper "Comparing Results of 31 Algorithms from the Black-Box Optimization Benchmarking BBOB-2009". For a more detailed performance assessment of all so-far benchmarked algorithms on bbob, see the ppdata archive.