cocopp.pprldistr

module documentation

For generating empirical cumulative distribution function figures.

The outputs show empirical cumulative distribution functions (ECDFs) of the running times of trials. These ECDFs show on the y-axis the fraction of cases for which the running time (left subplots) or the df-value (right subplots) was smaller than the value given on the x-axis. On the left, ECDFs of the running times from trials are shown for different target values. Light brown lines in the background show ECDFs for target value 1e-8 of all algorithms benchmarked during BBOB-2009. On the right, ECDFs of df-values from all trials are shown for different numbers of function evaluations.

Example

CAVEAT: the naming conventions in this module mix up ERT (an estimate of the expected running length) and run lengths.

Function	`beautify`	Format the figure of the run length distribution.
Function	`beautifyECDF`	Generic formatting of ECDF figures.
Function	`beautifyFVD`	Formats the figure of the run length distribution.
Function	`beautifyRLD`	Format and save the figure of the run length distribution.
Function	`caption_single`	Undocumented
Function	`caption_two`	Undocumented
Function	`comp`	Generate figures of ECDF that compare 2 algorithms.
Function	`erld_data`	return `[sorted_runlengths_divided_by_dimension, nb_of_all_runs, functions_ids_found, functions_ids_solved]`
Function	`load_previous_data`	Undocumented
Function	`load_previous_RLBdata`	Undocumented
Function	`main`	Generate figures of empirical cumulative distribution functions.
Function	`plot`	Plot ECDF of evaluations and final function values in a single figure for demonstration purposes.
Function	`plot_previous_algorithms`	Display BBOB 2009 data, by default from `pprldistr.previous_data_filename = 'pprldistr2009_1e-8.pickle.gz'`
Function	`plotECDF`	Plot an empirical cumulative distribution function.
Function	`plotFVDistr`	Creates ECDF of final function values plot from a DataSetList.
Function	`plotRLB_previous_algorithms`	Display BBOB 2009 data, by default from `pprldistr.previous_data_filename = 'pprldistr2009_1e-8.pickle.gz'`
Function	`plotRLDistr`	Creates run length distributions from a sequence dataSetList.
Variable	`fmax`	Undocumented
Variable	`nbperdecade`	Undocumented
Variable	`previous_data_dict`	Undocumented
Variable	`previous_data_filename`	Undocumented
Variable	`previous_RLBdata_dict`	Undocumented
Variable	`previous_RLBdata_filename`	Undocumented
Variable	`refcolor`	Undocumented
Variable	`rldStyles`	Undocumented
Variable	`rldUnsuccStyles`	Undocumented
Variable	`runlen_xlimits_max`	Undocumented
Variable	`runlen_xlimits_min`	Undocumented
Function	`_plotRLDistr_old`	Creates run length distributions from a sequence dataSetList.

def beautify(): ¶

Format the figure of the run length distribution.

Used in conjunction with plot method (obsolete/outdated, see functions beautifyFVD and beautifyRLD).

def beautifyECDF(): ¶

Generic formatting of ECDF figures.

def beautifyFVD(isStoringXMax=False, ylabel=True): ¶

Formats the figure of the run length distribution.

This function is to be used with plotFVDistr

Parameters
isStoringXMax	Undocumented
ylabel	Undocumented
bool isStoringMaxF	if set to True, the first call `beautifyFVD` sets the global `fmax` and all subsequent call will have the same maximum xlim
bool ylabel	if True, y-axis will be labelled.

def beautifyRLD(xlimit_max=None): ¶

Format and save the figure of the run length distribution.

After calling this function, changing the boundaries of the figure will not update the ticks and tick labels.

def caption_single(): ¶

Undocumented

def caption_two(): ¶

Undocumented

def comp(dsList0, dsList1, targets, isStoringXMax=False, outputdir='', info='default'): ¶

Generate figures of ECDF that compare 2 algorithms.

Parameters
dsList0	Undocumented
dsList1	Undocumented
targets	Undocumented
isStoringXMax	Undocumented
outputdir	Undocumented
info	Undocumented
DataSetList dsList0	list of DataSet instances for ALG0
DataSetList dsList1	list of DataSet instances for ALG1
seq targets	target function values to be displayed
bool isStoringXMax	if set to True, the first call `beautifyFVD` sets the globals `fmax` and `maxEvals` and all subsequent calls will use these values as rightmost xlim in the generated figures.
string outputdir	output directory (must exist)
string info	string suffix for output file names.

def erld_data(dsList, target, max_fun_evals=np.inf): ¶

return [sorted_runlengths_divided_by_dimension, nb_of_all_runs, functions_ids_found, functions_ids_solved]

max_fun_evals is only used to compute function_ids_solved, that is elements in sorted_runlengths... can be larger.

copy-paste from plotRLDistr and not used.

def load_previous_data(filename=previous_data_filename, force=False): ¶

Undocumented

def load_previous_RLBdata(filename=previous_RLBdata_filename): ¶

Undocumented

def main(dsList, isStoringXMax=False, outputdir='', info='default'): ¶

Generate figures of empirical cumulative distribution functions.

This method has a feature which allows to keep the same boundaries for the x-axis, if isStoringXMax==True. This makes sense when dealing with different functions or subsets of functions for one given dimension.

CAVE: this is bug-prone, as some data depend on the maximum evaluations and the appearence therefore depends on the calling order.

Parameters
dsList	Undocumented
isStoringXMax	Undocumented
outputdir	Undocumented
info	Undocumented
DataSetList dsList	list of DataSet instances to process.
bool isStoringXMax	if set to True, the first call `beautifyFVD` sets the globals `fmax` and `maxEvals` and all subsequent calls will use these values as rightmost xlim in the generated figures.
string outputdir	output directory (must exist)
string info	string suffix for output file names.

def plot(dsList, targets=None, **plotArgs): ¶

Plot ECDF of evaluations and final function values in a single figure for demonstration purposes.

def plot_previous_algorithms(dim, funcs): ¶

Display BBOB 2009 data, by default from pprldistr.previous_data_filename = 'pprldistr2009_1e-8.pickle.gz'

def plotECDF(x, n=None, **plotArgs): ¶

Plot an empirical cumulative distribution function.

Parameters
x	Undocumented
n	Undocumented
seq x	data
int n	number of samples, if not provided len(x) is used
**plotArgs	optional keyword arguments provided to plot.
Returns
handles of the plot elements.

def plotFVDistr(dsList, budget, min_f=None, **plotArgs): ¶

Creates ECDF of final function values plot from a DataSetList.

CAVEAT: this routine is not instance-balanced

Parameters
dsList	data sets
budget	Undocumented
min_f	used for the left limit of the plot
float budget	maximum evaluations / dimension that "count"
**plotArgs	additional arguments passed to plot
Returns
handle

def plotRLB_previous_algorithms(dim, funcs): ¶

Display BBOB 2009 data, by default from pprldistr.previous_data_filename = 'pprldistr2009_1e-8.pickle.gz'

def plotRLDistr(dsList, target, label='', max_fun_evals=np.inf, **plotArgs): ¶

Creates run length distributions from a sequence dataSetList.

Labels of the line (for the legend) will be appended with the number of functions at least solved once.

Example:

plotRLDistr(dsl, lambda f: 1e-6)

Details: target is a function taking a (function_number, dimension) pair as input and returning a float. It can be defined as lambda fun_dim: targets(fun_dim)[j] returning the j-th element of targets(fun_dim), where targets is an instance of class pproc.TargetValues (see the pproc.TargetValues.__call__ method).

TODO: data generation and plotting should be in separate methods TODO: different number of runs/data biases the results, shouldn't

the number of data made the same, in case?

Parameters
dsList	Undocumented
target	a method that delivers single target values like `target((fun, dim))`
label	Undocumented
max_fun_evals	only used to determine success on a single function
DataSetList dsList	Input data sets
str label	target value label to be displayed in the legend
**plotArgs	additional arguments passed to the plot command
Returns
handles of the resulting plot.

fmax = ¶

Undocumented

nbperdecade: int = ¶

Undocumented

previous_data_dict = ¶

Undocumented

previous_data_filename = ¶

Undocumented

previous_RLBdata_dict = ¶

Undocumented

previous_RLBdata_filename = ¶

Undocumented

refcolor: str = ¶

Undocumented

rldStyles: tuple = ¶

Undocumented

rldUnsuccStyles: tuple = ¶

Undocumented

runlen_xlimits_max = ¶

Undocumented

runlen_xlimits_min: int = ¶

Undocumented

def _plotRLDistr_old(dsList, target, **plotArgs): ¶

Creates run length distributions from a sequence dataSetList.

Labels of the line (for the legend) will be set automatically with the following format: %+d: %d/%d % (log10()

Parameters
dsList	Undocumented
target	Undocumented
DataSetList dsList	Input data sets
dict or float target	target precision
**plotArgs	additional arguments passed to the plot command
Returns
handles of the resulting plot.