cocopp.archiving.OfficialArchives(object) class documentation

Part of cocopp.archiving (View In Hierarchy)

overdesigned class to connect URLs, names, and classes of "official" archives.

The all archive name gets special treatment. The 'all'-archive is currently needed in cocopp.main but could be dropped otherwise.

The class collects the "official" online data archives as attributes. Each suite is also a different class inheriting from COCODataArchive. The class may become the same for some or all suites in future.

cocopp.archives is an instance of OfficialArchives and contains as archive attributes (at this point in time):

all: COCODataArchive, the list of all archived data from all test suites.

bbob: COCOBBOBDataArchive, the list of archived data run on the bbob test suite.

bbob_noisy: COCOBBOBNoisyDataArchive, ditto on the bbob_noisy test suite.

bbob_biobj: COCOBBOBBiobjDataArchive, ditto...

The names property is a list of all "official" archive names available.

A Quick Guide

1) To list all available data:

>>> import cocopp
>>> cocopp.archives.all  # doctest:+ELLIPSIS,+SKIP,
['bbob/2009/AL...

2) To see all available data from a given test suite:

>>> cocopp.archives.bbob  # doctest:+ELLIPSIS,+SKIP,
['2009/ALPS...

or

>>> cocopp.archives.bbob_biobj  # doctest:+ELLIPSIS,+SKIP,
['2016/DEMO_Tusar...

or

>>> cocopp.archives.bbob_noisy  # doctest:+ELLIPSIS,+SKIP,
['2009/ALPS...

3) We can extract a subset from any test suite (such as cocopp.archives.all, cocopp.archives.bbob, ...) of the archive by sub-string matching:

>>> cocopp.archives.bbob.find('bfgs')  # doctest:+NORMALIZE_WHITESPACE,+ELLIPSIS,+SKIP
['2009/BFGS_ros_noiseless.tgz',
 '2012/DE-BFGS_voglis_noiseless.tgz',
 '2012/PSO-BFGS_voglis_noiseless.tgz',
 '2014-others/BFGS-scipy-Baudis.tgz',
 '2014-others/L-BFGS-B-scipy-Baudis.tgz'...

or by regex pattern matching:

>>> cocopp.archives.bbob.find('bfgs') == cocopp.archives.bbob.find('.*bfgs')
True

but

>>> len(cocopp.archives.bbob.find('bfgs')) > len(cocopp.archives.bbob.find('bfgs.*'))
True

The find method will not download data and is only for inspecting the archives. If we want to actually process the data we need to use get, get_all, get_extended or get_found, or use the same search string in cocopp.main appending a * if more than one algorithm should be processed:

4) When postprocessing data via `cocopp.main`, we can use the archive like

>>> cocopp.main(cocopp.archives.bbob.get_all('bfgs').as_string)  # doctest:+SKIP

or the shortcut

>>> cocopp.main('bfgs*')  # doctest:+SKIP

using the special meaning of the trailing * in this case (see below). The get and get_extended methods are called on each argument given in a string to main. We can do things like

>>> cocopp.main('bbob/2009/BIPOP DE-BFGS')  # doctest:+SKIP

When a string has multiple matches, the postprocessing bails out. For such cases, we can use the trailing symbols * (AKA take all matches) and ! (AKA take the first match) which uses the get_extended method under the hood:

>>> cocopp.main('BIPOP! 2012/DE*')  # doctest:+SKIP

will expand to the following:

Post-processing (2+)
  Using:
    /.../.cocopp/data-archive/bbob/2009/BIPOP-CMA-ES_hansen_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DE-AUTO_voglis_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DE-BFGS_voglis_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DE-ROLL_voglis_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DE-SIMPLEX_voglis_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DE_posik_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DEAE_posik_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DEb_posik_noiseless.tgz
    /.../.cocopp/data-archive/bbob/2012/DEctpb_posik_noiseless.tgz

Post-processing (2+)
  loading data...
[...]

5) If we want to also pass other arguments to the postprocessing (e.g. the output folder) in case 3) above, string concatenation does the trick:

>>> cocopp.main('-o myoutputfolder ' + cocopp.archives.bbob.get_all('bfgs').as_string)  # doctest:+SKIP

For case 4), this works directly:

>>> cocopp.main('-o myoutputfolder BIPOP! 2012/DE*')  # doctest:+SKIP
Method __init__ all URLs and classes (optional) in one place.
Method add_archive Allow to use a new official archive.
Method set_as_attributes_in Assign all archives as attribute of target except for 'test'.
Method names a list of valid key names
Method urls a name->URL dictionary
Method url return url of "official" archive named name.
Method class_ class of archive named name when returned by get
Method update_all Undocumented
Method _get get j-th entry of name, where j==0 is the URL and j==1 is the class
def __init__(self):

all URLs and classes (optional) in one place.

The archive names are identical with the last part of the URL. The only exception is made for 'all', which is removed to get the URL.

def add_archive(self, name):

Allow to use a new official archive.

The archive must exist as a subfolder of http://coco.gforge.inria.fr/data-archive

def set_as_attributes_in(self, target=None, except_for=('test',), update=False):

Assign all archives as attribute of target except for 'test'.

target is by default self.

Details: This method can only be called when the class names and the module attribute official_archives: OfficialArchive (used in get) are available. Depending on the implementation of get, it may download the definition files on its first-ever call by/on any given user/machine.

@property
def names(self):
a list of valid key names
@property
def urls(self):
a name->URL dictionary
def _get(self, name, j):
get j-th entry of name, where j==0 is the URL and j==1 is the class
def url(self, name):

return url of "official" archive named name.

The same value as self.urls.get(name).

def class_(self, name):
class of archive named name when returned by get
def update_all(self):
Undocumented
API Documentation for cocopp, generated by pydoctor at 2020-01-21 16:27:37.