cocopp.archiving.OfficialArchives

class documentation

class OfficialArchives(object):

Constructor: OfficialArchives(url)

View In Hierarchy

overdesigned class to connect URLs, names, and classes of "official" archives.

The all archive name gets special treatment. The 'all'-archive is currently needed in cocopp.main but could be dropped otherwise.

The class collects the "official" online data archives as attributes. Each suite is also a different class inheriting from COCODataArchive. The class may become the same for some or all suites in future.

cocopp.archives is an instance of OfficialArchives and contains as archive attributes (at this point in time):

all: COCODataArchive, the list of all archived data from all test suites.

bbob: COCOBBOBDataArchive, the list of archived data run on the bbob test suite.

bbob_noisy: COCOBBOBNoisyDataArchive, ditto on the bbob_noisy test suite.

bbob_biobj: COCOBBOBBiobjDataArchive, ditto...

The names property is a list of all "official" archive names available.

A Quick Guide

1) To list all available data:

>>> import cocopp
>>> cocopp.archives.all  # doctest:+ELLIPSIS,+SKIP,
['bbob/2009/AL...

2) To see all available data from a given test suite:

>>> cocopp.archives.bbob  # doctest:+ELLIPSIS,+SKIP,
['2009/ALPS...

or

>>> cocopp.archives.bbob_biobj  # doctest:+ELLIPSIS,+SKIP,
['2016/DEMO_Tusar...

or

>>> cocopp.archives.bbob_noisy  # doctest:+ELLIPSIS,+SKIP,
['2009/ALPS...

3) We can extract a subset from any test suite (such as cocopp.archives.all, cocopp.archives.bbob, ...) of the archive by sub-string matching:

>>> cocopp.archives.bbob.find('bfgs')  # doctest:+NORMALIZE_WHITESPACE,+ELLIPSIS,+SKIP
['2009/BFGS_ros_noiseless.tgz',
 '2012/DE-BFGS_voglis_noiseless.tgz',
 '2012/PSO-BFGS_voglis_noiseless.tgz',
 '2014-others/BFGS-scipy-Baudis.tgz',
 '2014-others/L-BFGS-B-scipy-Baudis.tgz'...

or by regex pattern matching:

>>> cocopp.archives.bbob.find('bfgs') == cocopp.archives.bbob.find('.*bfgs')
True

but

>>> len(cocopp.archives.bbob.find('bfgs')) > len(cocopp.archives.bbob.find('bfgs.*'))
True

The find method will not download data and is only for inspecting the archives. If we want to actually process the data we need to use get, get_all, get_extended or get_found, or use the same search string in cocopp.main appending a * if more than one algorithm should be processed:

4) When postprocessing data via `cocopp.main`, we can use the archive like

>>> cocopp.main(cocopp.archives.bbob.get_all('bfgs').as_string)  # doctest:+SKIP

or the shortcut

>>> cocopp.main('bfgs*')  # doctest:+SKIP

using the special meaning of the trailing * in this case (see below). The get and get_extended methods are called on each argument given in a string to main. We can do things like

>>> cocopp.main('bbob/2009/BIPOP DE-BFGS')  # doctest:+SKIP

When a string has multiple matches, the postprocessing bails out. For such cases, we can use the trailing symbols * (AKA take all matches) and ! (AKA take the first match) which uses the get_extended method under the hood:

>>> cocopp.main('BIPOP! 2012/DE*')  # doctest:+SKIP

will expand to the following:

Post-processing (2+)
  Using:
    /.../.cocopp/data-.../bbob/2009/BIPOP-CMA-ES_hansen_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DE-AUTO_voglis_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DE-BFGS_voglis_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DE-ROLL_voglis_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DE-SIMPLEX_voglis_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DE_posik_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DEAE_posik_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DEb_posik_noiseless.tgz
    /.../.cocopp/data-.../bbob/2012/DEctpb_posik_noiseless.tgz

Post-processing (2+)
  loading data...
[...]

5) If we want to also pass other arguments to the postprocessing (e.g. the output folder) in case 3) above, string concatenation does the trick:

>>> cocopp.main('-o myoutputfolder ' + cocopp.archives.bbob.get_all('bfgs').as_string)  # doctest:+SKIP

For case 4), this works directly:

>>> cocopp.main('-o myoutputfolder BIPOP! 2012/DE*')  # doctest:+SKIP
Method __init__ all URLs and classes (optional) in one place.
Method add_archive Allow to use a new official archive.
Method class_ class of archive named name when returned by get
Method link_as_attributes_in Assign all archives as attribute of target except for 'test'.
Method set_as_attributes_in Assign all archives as attribute of target except for 'test'.
Method update_all update archive definition files from their remote location.
Method url return url of "official" archive named name.
Instance Variable all Undocumented
Property names a list of valid key names
Property urls a name->URL dictionary
Method _get get j-th entry of name, where j==0 is the URL and j==1 is the class
Method _make_folder_skeleton workaround to avoid bailing when www is not reachable during
Instance Variable _base Undocumented
Instance Variable _list Undocumented
def __init__(self, url=None):

all URLs and classes (optional) in one place.

The archive names are identical with the last part of the URL. The only exception is made for 'all', which is removed to get the URL.

def add_archive(self, name):

Allow to use a new official archive.

The archive must exist as a subfolder of coco_url.

def class_(self, name):

class of archive named name when returned by get

def link_as_attributes_in(self, target, except_for=('test'), update=False):

Assign all archives as attribute of target except for 'test'.

target is by default self.

Details: This method can only be called when the class names and the module attribute official_archives: OfficialArchive (used in get) are available. It only creates links to the existing archives.

def set_as_attributes_in(self, target=None, except_for=('test'), update=False):

Assign all archives as attribute of target except for 'test'.

target is by default self.

Details: This method can only be called when the class names and the module attribute official_archives: OfficialArchive (used in get) are available. It creates new instances of the archives. Depending on the implementation of get, it may download the definition files on its first-ever call by/on any given user/machine.

def update_all(self):

update archive definition files from their remote location.

The update is necessary to account for newly added data. This method requires www connectivity with only a few KB of transmitted data.

def url(self, name):

return url of "official" archive named name.

The same value as self.urls.get(name).

all =

Undocumented

@property
names =

a list of valid key names

@property
urls =

a name->URL dictionary

def _get(self, name, j):

get j-th entry of name, where j==0 is the URL and j==1 is the class

def _make_folder_skeleton(self):

workaround to avoid bailing when www is not reachable during

the very first import.

_base =

Undocumented

_list =

Undocumented