cocoex.Suite

Suite of benchmark problems.

Input arguments to Suite are name: str, instance: str, options: str, and passed to the respective C code (see coco.h).

>>> import cocoex as ex
>>> suite = ex.Suite("bbob", "", "")
>>> f = suite.next_problem()
>>> assert f.number_of_objectives == 1
>>> assert f.evaluations == 0
>>> print("f([1,2]) = %.11f" % f([1,2]))
f([1,2]) = 90.00369408000
>>> assert f.evaluations == 1

Sweeping through all problems is as simple as:

>>> import cocoex as ex
>>> suite = ex.Suite("bbob-biobj", "", "")
>>> observer = ex.Observer("bbob-biobj", "result_folder:doctest")
>>> for fun in suite:
...     if fun.index == 0:
...         print("Number of objectives %d, %d, %d" %
...                 (fun.number_of_objectives,
...                  suite.number_of_objectives[0],
...                  suite.number_of_objectives[-1]))
...     fun.observe_with(observer)
...     assert fun.evaluations == 0
...     assert fun.number_of_objectives == suite.number_of_objectives[0]
...     # run run run using fun  # doctest: +ELLIPSIS
Number of objectives 2, 2, 2...

In the example, an observer was added to produce output data for the COCO post-processing.

The following example runs the entire bbob2009 benchmark suite on random search:

>>> import numpy as np
>>> from cocoex import Suite, Observer
...
>>> MAX_FE = 22  # max f-evaluations
>>> def random_search(f, lb, ub, m):  # don't use m >> 1e5 with this implementation
...     candidates = lb + (ub - lb) * np.random.rand(m, len(lb))
...     return candidates[np.argmin([f(x) for x in candidates])]
...
>>> solver = random_search
>>> suite = Suite("bbob", "year:2009", "")
>>> observer = Observer("bbob",
...              "result_folder: %s_on_%s" % (solver.__name__, "bbob2009"))
>>> for fun in suite:
...     assert fun.evaluations == 0
...     if fun.dimension >= 10:
...         break
...     print('Current problem index = %d' % fun.index)
...     fun.observe_with(observer)
...     assert fun.evaluations == 0
...     solver(fun, fun.lower_bounds, fun.upper_bounds, MAX_FE)
...     # data should be now in the "exdata/random_search_on_bbob2009" folder
...     assert fun.evaluations == MAX_FE  # depends on the solver
...     # doctest: +ELLIPSIS
Current problem index = 0...
>>> #
>>> # Exactly the same using another looping technique:
>>> for id in suite.ids():
...     fun = suite.get_problem(id, observer)
...     _ = solver(fun, fun.lower_bounds, fun.upper_bounds, MAX_FE)
...     print("Evaluations on %s: %d" % (fun.name, fun.evaluations))
...     fun.free()  # this is absolutely necessary here
...     # doctest: +ELLIPSIS
Evaluations on ...

We can select a single function, say BBOB f9 in 20D, of a given suite like:

>>> import cocoex as ex
>>> suite = ex.Suite("bbob", "", "dimensions:20 instance_indices:1")
>>> len(suite)
24
>>> f9 = suite.get_problem(8)
>>> x = f9.initial_solution  # a copy of a feasible point
>>> all(x == 0)
True

See module attribute cocoex.known_suite_names for known suite names:

>>> import cocoex as ex
>>> for suite_name in ex.known_suite_names:
...     suite = ex.Suite(suite_name, "", "")
...     print(suite.dimensions)
...     for f in suite:
...         assert f.dimension in suite.dimensions
...         assert f.evaluations == 0
...         # doctest: +ELLIPSIS
[2, 3, 5, 10, 20, 40]...

See file example_experiment.py for a full example use case.

Details: depending on the benchmark suite and observer, only one problem can be open at a time. Using get_problem without free or mixing the use of next_problem and get_problem may not be possible. For example, in this case the "bbob" observer is known to lead to a crash of the Python interpreter.

See also Observer and example_experiment.py.