class DataWithFewSuccesses:
Constructor: DataWithFewSuccesses(folder_name, minsuccesses, budget_multiplier, success_threshold, exclude)
This class is superseded by cocoex.ExperimentRepeater for its main usecase,
to continue experimenting on functions with low success rate until the budget is exhausted.
The all property is an OrderedDict with all (dimension, funcId)-
tuples that have less than self.minsuccesses successes. The tuples
are the dict keys, the values are the respective numbers for
[evaluations, successes, trials], where evaluations are the average
overall (sum of) evaluations per instance. The minimal number of desired
successes can be changed at any time by re-assigning the minsuccesses
attribute in which case the return value of result may change. When
the success_threshold attribute is change, compute_successes has to
be called to update the results. The with_budget_left property gives
those entries of all which have less than budget_multiplier x
dimension evaluations. The current property gives with_budget_left
or all if the former is empty.
The first argument can be a folder or filename or any other string
excepted by cocopp.load matching a single dataset or it can be a
DataSetList.
Usage concept example:
>> run_more = DataWithFewSuccesses('folder_or_file_name_for_cocopp.load').current
>> for p in cocoex.Suite('bbob', '', ''):
.. if (p.id_function, p.dimension) not in run_more:
.. continue
.. p.observe_with(...)
.. # run solver on problem p
.. [...]
Details
The success number is calculated with the raw_value parameter of
DataSet.detSuccesses thereby bypassing successes from instance
balancing copies. However it falls back with a warning when this does
not work.
In the experimental setup from 2009, the used budget [evaluations per instance] is "overestimated" by a factor of three because each instance is run three times to begin with.
TODO: When in this case a DataSetList is passed instead
of a folder name, we could check whether it was instantiated with
genericsettings.balance_instances == False which is not the default
setting. Either len(self.evals[0]) != len(self._evals[0]) or
self.instance_multipliers is not None and
np.any(self.instance_multipliers > 1) indicate that a balancing
action was taken.
See also cocopp.load.
| Method | __init__ |
folder_name can also be a filename or a data entry or a DataSetList |
| Method | __len__ |
Undocumented |
| Method | compute |
assign .evaluations as a list of overall evaluations spent per instance (on average) |
| Method | compute |
Assign successes attribute as a list of number of successful trials |
| Method | print |
return a str with the number of data sets with too few successes |
| Instance Variable | budget |
Undocumented |
| Instance Variable | dsl |
Undocumented |
| Instance Variable | evaluations |
list of average evaluations per instance for each function+dimension |
| Instance Variable | exclude |
Undocumented |
| Instance Variable | input |
Undocumented |
| Instance Variable | minsuccesses |
Undocumented |
| Instance Variable | success |
Undocumented |
| Instance Variable | successes |
list of successful trials, depends on success_threshold. Can be recomputed by calling compute_successes. |
| Instance Variable | trials |
number of trials in each data set, for the record only |
| Property | all |
depends on attributes minsuccesses and successes only |
| Property | current |
OrderedDict of (function, dimension) with budget left or otherwise all |
| Property | with |
depends on attributes minsuccesses, successes, budget_multiplier and evaluations |
folder_name can also be a filename or a data entry or a DataSetList
assign .evaluations as a list of overall evaluations spent per instance (on average)
for each function+dimension.
Assign successes attribute as a list of number of successful trials
in the data sets of self.dsl and return self. When given, reassign
also the success_threshold attribute.
list of successful trials, depends on success_threshold.
Can be recomputed by calling compute_successes.