class DataWithFewSuccesses:
Constructor: DataWithFewSuccesses(folder_name, minsuccesses, budget_multiplier, success_threshold, exclude)
The all property is an OrderedDict with all (dimension, funcId)-
tuples that have less than self.minsuccesses successes. The tuples
are the dict keys, the values are the respective numbers for
[evaluations, successes, trials], where evaluations are the average
overall (sum of) evaluations per instance. The minimal number of desired
successes can be changed at any time by re-assigning the minsuccesses
attribute in which case the return value of result may change. When
the success_threshold attribute is change, compute_successes has to
be called to update the results. The with_budget_left property gives
those entries of all which have less than budget_multiplier x
dimension evaluations. The current property gives with_budget_left
or all if the former is empty.
The first argument can be a folder or filename or any other string
excepted by cocopp.load matching a single dataset or it can be a
DataSetList.
Usage concept example:
>> run_more = DataWithFewSuccesses('folder_or_file_name_for_cocopp.load').current
>> for p in cocoex.Suite('bbob', '', ''):
.. if (p.id_function, p.dimension) not in run_more:
.. continue
.. p.observe_with(...)
.. # run solver on problem p
.. [...]
Details
The success number is calculated with the raw_value parameter of
DataSet.detSuccesses thereby bypassing successes from instance
balancing copies. However it falls back with a warning when this does
not work.
In the experimental setup from 2009, the used budget [evaluations per instance] is "overestimated" by a factor of three because each instance is run three times to begin with.
TODO: When in this case a DataSetList is passed instead
of a folder name, we could check whether it was instantiated with
genericsettings.balance_instances == False which is not the default
setting. Either len(self.evals[0]) != len(self._evals[0]) or
self.instance_multipliers is not None and
np.any(self.instance_multipliers > 1) indicate that a balancing
action was taken.
See also cocopp.load.
| Method | __init__ |
folder_name can also be a filename or a data entry or a DataSetList |
| Method | __len__ |
Undocumented |
| Method | compute |
assign .evaluations as a list of overall evaluations spent per instance (on average) |
| Method | compute |
Assign successes attribute as a list of number of successful trials |
| Method | print |
return a str with the number of data sets with too few successes |
| Instance Variable | budget |
Undocumented |
| Instance Variable | dsl |
Undocumented |
| Instance Variable | evaluations |
list of average evaluations per instance for each function+dimension |
| Instance Variable | exclude |
Undocumented |
| Instance Variable | input |
Undocumented |
| Instance Variable | minsuccesses |
Undocumented |
| Instance Variable | success |
Undocumented |
| Instance Variable | successes |
list of successful trials, depends on success_threshold. Can be recomputed by calling compute_successes. |
| Instance Variable | trials |
number of trials in each data set, for the record only |
| Property | all |
depends on attributes minsuccesses and successes only |
| Property | current |
OrderedDict of (function, dimension) with budget left or otherwise all |
| Property | with |
depends on attributes minsuccesses, successes, budget_multiplier and evaluations |
folder_name can also be a filename or a data entry or a DataSetList
assign .evaluations as a list of overall evaluations spent per instance (on average)
for each function+dimension.
Assign successes attribute as a list of number of successful trials
in the data sets of self.dsl and return self. When given, reassign
also the success_threshold attribute.
list of successful trials, depends on success_threshold.
Can be recomputed by calling compute_successes.