Link Search Menu Expand Document

Algorithm data sets for the bbob test suite


In the first table below, you will find all official algorithm data sets on the bbob test suite, together with their year of publication, the authors, and related PDFs for each data set. Links to the source code to run the corresponding experiments/algorithms are provided whenever available.

A second table mentions data sets that have been collected on the bbob suite, but which are not complete in the sense that they miss at least one of the requested dimensions 2, 3, 5, 10, 20.

To sort the tables, simply click on the table header of the corresponding column.

Number Algorithm Name Year Author(s) link to data related PDFs, source code, and remarks
000 ALPS 2009 Hornby data pdf
001 AMALGAM 2009 Bosman et al. data pdf noiseless - pdf noisy
002 BAYEDA 2009 Gallagher data pdf noiseless - pdf noisy
003 BFGS 2009 Ros data pdf noiseless - pdf noisy
004 BIPOP-CMA-ES 2009 Hansen data pdf noiseless - pfd noisy
005 Cauchy-EDA 2009 Pošík data pdf
006 CMA-ESPLUSSEL 2009 Auger and Hansen data pdf noiseless - pdf noisy
007 DASA 2009 Korošec and Šilc data pdf noiseless - pdf noisy
008 DE-PSO 2009 García-Nieto et al. data pdf noiseless - pdf noisy
009 DIRECT 2009 Pošík data pdf - algorithm is deterministic and thus, only run on each instance once
010 EDA-PSO 2009 El-Abd and Kamel data pdf
011 FULLNEWUOA 2009 Ros data original 2009 noiseless paper - original 2009 noisy paper - 2010 comparison noiseless - 2010 comparison noisy
012 G3PCX 2009 Pošík data pdf
013 GA 2009 Nicolau data pdf
014 GLOBAL 2009 Pál et al. data BBOB-2009 paper not available online - corresponding ECJ paper
015 iAMALGAM 2009 Bosman et al. data pdf noiseless - pdf noisy
016 IPOP-SEP-CMA-ES 2009 Ros data pdf noiseless - pdf noisy
017 LSfminbnd 2009 Pošík data pdf
018 LSstep 2009 Pošík data pdf
019 MA-LS-CHAIN 2009 Molina et al. data pdf noiseless - pdf noisy
020 MCS 2009 Huyer and Neumaier data pdf noiseless - pdf noisy
021 NELDERDOERR 2009 Doerr et al. data pdf
022 NELDER 2009 Hansen data pdf
023 NEWUOA 2009 Ros data original 2009 noiseless paper - original 2009 noisy paper - 2010 comparison with BIPOP-CMA-ES noiseless - 2010 comparison noiseless - 2010 comparison noisy
024 ONEFIFTH 2009 Auger data pdf noiseless - pdf noisy
025 POEMS 2009 Kubalík data pdf
026 PSO 2009 El-Abd and Kamel data pdf
027 PSO_Bounds 2009 El-Abd and Kamel data pdf
028 RANDOMSEARCH 2009 Auger and Ros data pdf noiseless - pdf noisy
029 Rosenbrock 2009 Pošík data pdf
030 VNS 2009 García-Martínez and Lozano data pdf noiseless - pdf noisy
031 1komma2 2010 Auger et al. data pdf cmp1komma2mirAnd1komma2mirser - doi - pdf cmp1komma2mirAnd1komma2mirserNoisy - pdf cmp1komma2ser - pdf cmp1komma2serNoisy
032 1komma2mir 2010 Auger et al. data pdf cmp1komma2And1komma2mirser - doi - pdf cmp1komma2And1komma2mirserNoisy
033 1komma2mirser 2010 Auger et al. data pdf cmp1komma2And1komma2mir - doi - pdf cmp1komma2And1komma2mirNoisy
034 1komma2ser 2010 Auger et al. data pdf cmp1komma2 - cmp1komma2Noisy
035 1komma4 2010 Auger et al. data pdf cmp1komma4serNoisy - doi - the other three papers are not linked to at the moment
036 1komma4mir 2010 Auger et al. data pdf cmp1komma4And1komma4mirser - cmp1komma4And1komma4mirserNoisy
037 1komma4mirser 2010 Auger et al. data pdf noiseless - pdf noisy - pdf cmp1komma4And1komma4mir - pdf cmp1komma4And1komma4mirNoisy
038 1komma4ser 2010 Auger et al. data pdf cmp1komma4 - pdf cmp1komma4Noisy - doi
039 1plus1 2010 Auger et al. data pdf cmp1plus2mirser
040 1plus2mirser 2010 Auger et al. data pdf cmp1plus1
041 ABC 2010 El-Abd data doi
042 AVGNEWUOA 2010 Ros data mentioned in 2009 paper, data set submitted in 2009, but data presented only in 2010 - pdf cmpNEWUOAFULLNEWUOnoiseless - algname still NEWUOA
043 CMAEGS 2010 Finck and Beyer data pdf
044 DE-F-AUC 2010 Fialho et al. data pdf cmpPM-AdapSS-DE
045 DEuniform 2010 Fialho et al. data pdf cmpPM-AdapSS-DE
046 IPOP-ACTCMA-ES 2010 Hansen and Ros data pdf cmpIPOP-CMA-ESNoiseless - pdf cmpIPOP-CMA-ESNoisy
047 IPOP-CMA-ES 2010 Hansen and Ros data pdf cmpIPOP-ACTCMA-ESNoiseless - pdf cmpIPOP-ACTCMA-ESNoisy - cmpBIPOP-CMA-ESNoiseless - pdf cmpBIPOP-CMA-ESNoisy
048 MOS 2010 LaTorre et al. data pdf noiseless - pdf noisy
049 NBC-CMA 2010 Preuss data doi
050 PM-AdapSS-DE 2010 Fialho et al. data pdf cmpUniformDE - pdf cmpDE-F-AUC
051 RCGA 2010 Tran and Jin data doi noiseless - doi noisy
052 SPSA 2010 Finck and Beyer data pdf noiseless - pdf noisy
053 oPOEMS 2010 Kubalík data doi cmppPOEMS
054 pPOEMS 2010 Kubalík data doi cmpoPOEMS
055 ACOR 2012 Liao et al data pdf
056 BIPOPaCMA 2012 Loshchilov et al. data pdf
057 BIPOPsaACM 2012 Loshchilov et al. data pdf - source code
058 CMA 2012 Brockhoff et al. data pdf
059 CMAa 2012 Brockhoff et al. data pdf cmpCMAmaAndIPOP-ACTCMA-ES - pdf cmpCMAandCMAm
060 CMAm 2012 Brockhoff et al. data pdf
061 CMAma 2012 Brockhoff et al. data pdf cmpCMAaAndIPOP-ACTCMA-ES - pdf cmpCMAmah
062 CMAmah 2012 Brockhoff et al. data pdf cmpCMAma - pdf cmpCMAmh
063 CMAmh 2012 Brockhoff et al. data pdf
064 CMAES 2012 Pošík and Klemš data pdf - source code
065 DBRCGA 2012 Chuang and Chen data pdf
066 DE 2012 Pošík and Klemš data pdf - source code
067 DEAE 2012 Pošík and Klemš data pdf - source code
068 DEb 2012 Pošík and Klemš data pdf - source code
069 DEctpb 2012 Pošík and Klemš data pdf - source code
070 DE-AUTO 2012 Voglis et al. data pdf
071 DE-BFGS 2012 Voglis et al. data pdf - pdf cmpPSO-BFGS
072 DE-ROLL 2012 Voglis et al. data pdf
073 DE-SIMPLEX 2012 Voglis et al. data pdf
074 IPOPsaACM 2012 Loshchilov et al. data pdf cmpBIPOPsaACM - pdf cmpIPOP-ACTCMA-ESNoisy - source code
075 JADE 2012 Pošík and Klemš data pdf - source code
076 JADEb 2012 Pošík and Klemš data pdf - source code
077 JADEctpb 2012 Pošík and Klemš data pdf - source code
078 MVDE 2012 Veloso de Melo data pdf
079 NBIPOPaCMA 2012 Loshchilov et al. data pdf
080 NIPOPaCMA 2012 Loshchilov et al. data pdf
081 PSO-BFGS 2012 Voglis et al. data pdf
082 SNES 2012 Schaul data pdf
083 xNES 2012 Schaul data pdf - pdf cmpxNESas
084 xNESas 2012 Schaul data pdf - pdf cmpxNES - pdf cmpBIPOP-CMA-ES
085 BIPOP-saACM-k 2013 Loshchilov et al. data pdf - source code - BBOB2010 instances
086 BIPOP-aCMA-STEP 2013 Loshchilov et al. data pdf - source code - BBOB2010 instances
087 CGA-grid16 2013 Holtschulte and Moses data pdf
088 CGA-grid100 2013 Holtschulte and Moses data pdf
089 CGA-ring16 2013 Holtschulte and Moses data pdf
090 CGA-ring100 2013 Holtschulte and Moses data pdf
091 CMAES_Hutter 2013 Hutter et al. data pdf
092 DE_Pal 2013 Pàl data pdf - BBOB2010 instances
093 fmincon 2013 Pàl data pdf - BBOB2010 instances
094 fminunc 2013 Pàl data pdf - BBOB2010 instances
095 GA-100 2013 Holtschulte and Moses data pdf
096 HCMA 2013 Loshchilov et al. data pdf - source code - BBOB2010 instances
097 HILL 2013 Holtschulte and Moses data pdf
098 HMLSL 2013 Pàl data pdf - BBOB2010 instances
099 IP 2013 Liao and Stützle data pdf cmpLiaoTesting - pdf cmpLiaoBounding - BBOB2010 instances
100 IP-10DDr 2013 Liao and Stützle data pdf - BBOB2010 instances
101 IP-500 2013 Liao and Stützle data pdf - BBOB2010 instances
102 IPOP400D 2013 Auger et al. data pdf
103 lmm-CMA-ES 2013 Auger et al. data pdf
104 MEMPSODE 2013 Voglis data pdf
105 MLSL 2013 Pàl data pdf - BBOB2010 instances
106 OQNLP 2013 Pàl data pdf - BBOB2010 instances
107 P-DCN 2013 Tran et al. data pdf - source code
108 P-zero 2013 Tran et al. data pdf - source code
109 PRCGA 2013 Sawyerr et al. data pdf
110 simplex 2013 Pàl data pdf - BBOB2010 instances
111 SMAC-BBOB 2013 Hutter et al. data pdf
112 tany 2013 Liao and Stützle data pdf - BBOB2010 instances
113 texp 2013 Liao and Stützle data pdf - BBOB2010 instances
114 U-DCN 2013 Tran et al. data pdf - source code
115 U-zero 2013 Tran et al. data pdf - source code
116 MATSuMoToDefault-50D 2015 Brockhoff data CEC paper
117 R-DE-10e2 2015 Tanabe and Fukunaga data CEC paper
118 R-DE-10e5 2015 Tanabe and Fukunaga data CEC paper
119 R-SHADE-10e2 2015 Tanabe and Fukunaga data CEC paper
120 R-SHADE-10e5 2015 Tanabe and Fukunaga data CEC paper
121 RL-SHADE-10e2 2015 Tanabe and Fukunaga data CEC paper
122 RL-SHADE-10e5 2015 Tanabe and Fukunaga data CEC paper
123 SOO 2015 Derbel and Preux data CEC paper
124 BrentSTEPif 2015 Pošík and Baudiš data GECCO paper
125 BrentSTEPifeg 2015 Pošík and Baudiš data GECCO paper
126 BrentSTEPqi 2015 Pošík and Baudiš data GECCO paper
127 BrentSTEPrr 2015 Pošík and Baudiš data GECCO paper
128 CMA-CSA 2015 Atamna data GECCO paper - source code
129 CMA-MSR 2015 Atamna data GECCO paper - source code
130 CMA-TPA 2015 Atamna data GECCO paper - source code
131/132 GP1-CMAES 2015 Bajer et al. data - 2013 instances doi - slides - Thanks to the authors, two data sets are available with the 2015 and the 2013 instances
133/134 GP5-CMAES 2015 Bajer et al. data - 2013 instances doi - slides - Thanks to the authors, two data sets are available with the 2015 and the 2013 instances
135/136 IPOPCMAv3p61 2015 Bajer et al. data - 2013 instances doi - slides - Thanks to the authors, two data sets are available with the 2015 and the 2013 instances
137 LHD-10xDefault-MATSuMoTo 2015 Brockhoff et al. data GECCO paper
138 LHD-2xDefault-MATSuMoTo 2015 Brockhoff et al. data GECCO paper
139 RAND-2xDefault-MATSuMoTo 2015 Brockhoff et al. data GECCO paper
140/141 RF1-CMAES 2015 Bajer et al. data - 2013 instances doi - slides - Thanks to the authors, two data sets are available with the 2015 and the 2013 instances
142/153 RF5-CMAES 2015 Bajer et al. data - 2013 instances doi - slides - Thanks to the authors, two data sets are available with the 2015 and the 2013 instances
154 STEPif 2015 Pošík and Baudiš data GECCO paper
155 STEPifeg 2015 Pošík and Baudiš data GECCO paper
156 STEPrr 2015 Pošík and Baudiš data GECCO paper
157 PSAaLmC-CMA-ES 2016 Nishida and Akimoto data GECCO paper
158 PSAaLmD-CMA-ES 2016 Nishida and Akimoto data GECCO paper
159 PSAaSmC-CMA-ES 2016 Nishida and Akimoto data GECCO paper
160 PSAaSmD-CMA-ES 2016 Nishida and Akimoto data GECCO paper
161 CMAES-APOP 2017 Nguyen and Hansen data doi
161 DTS-CMA-ES 2017 Pitra et al. data doi - slides
163 EvoSpace-PSO-GA 2017 García-Valdez and Merelo data doi
164 KL-BIPOP-CMA-ES 2017 Yamaguchi and Akimoto data GECCO paper - slides
165 KL-IPOP-CMA-ES 2017 Yamaguchi and Akimoto data GECCO paper - slides
166 KL-Restart-CMA-ES 2017 Yamaguchi and Akimoto data GECCO paper - slides
167 Ord-H-DTS-CMA-ES 2017 Pitra et al. data doi - slides
168 Ord-N-DTS-CMA-ES 2017 Pitra et al. data doi - slides
169 Ord-Q-DTS-CMA-ES 2017 Pitra et al. data doi - slides
170 SSEABC 2017 Aydin and Yavuz data doi - slides
171 RANDOMSEARCH-4 2017 Brockhoff data COCO baseline data without BBOB paper
172 RANDOMSEARCH-4p5 2017 Brockhoff data COCO baseline data without BBOB paper
173 RANDOMSEARCH-5 2017 Brockhoff data COCO baseline data without BBOB paper
174 BBDE 2018 Vodopija et al. data GECCO 2018 paper doi
175 BBDE-best 2018 Vodopija et al. data GECCO 2018 paper doi
176 BBDE-N 2018 Vodopija et al. data GECCO 2018 paper doi
177 BBDE-ttb 2018 Vodopija et al. data GECCO 2018 paper doi
178 BFGS-M-17 2018 Blelly et al. data GECCO 2018 paper
179 BFGS-P-09 2018 Blelly et al. data GECCO 2018 paper
180 BFGS-P-Instances 2018 Blelly et al. data GECCO 2018 paper
181 BFGS-P-range 2018 Blelly et al. data GECCO 2018 paper
182 BFGS-P-StPt 2018 Blelly et al. data GECCO 2018 paper
183 CMAES-APOP-Var1 2018 Nguyen data GECCO 2018 paper
184 CMAES-APOP-Var2 2018 Nguyen data GECCO 2018 paper
185 CMAES-APOP-Var3 2018 Nguyen data GECCO 2018 paper
186 DE-best 2018 Vodopija et al. data GECCO 2018 paper doi
187 DE-rand 2018 Vodopija et al. data GECCO 2018 paper doi
188 DE-ttb 2018 Vodopija et al. data GECCO 2018 paper doi
189 PSA-CMA-ES 2018 Nishida and Akimoto data GECCO 2018 paper
190 PSA-CMA-ESwRS 2018 Nishida and Akimoto data GECCO 2018 paper
191 uBBDE 2018 Vodopija et al. data paper
192 uBBDE-best 2018 Vodopija et al. data paper
193 uBBDE-N 2018 Vodopija et al. data paper
194 uBBDE-ttb 2018 Vodopija et al. data paper
195 uDE-best 2018 Vodopija et al. data paper
196 uDE-rand 2018 Vodopija et al. data paper
197 uDE-ttb 2018 Vodopija et al. data paper
198 DTS-CMA-ES_005-2pop_v26_1model 2018 Bajer et al. data link to paper - details on the differences to #161
199 Adaptive-Two-Mode 2019 Bodner data doi
200 BFGS-scipy 2019 Varelas and Dahito data GECCO 2019 paper
201 CG-scipy 2019 Varelas and Dahito data GECCO 2019 paper
202 COBYLA-scipy 2019 Varelas and Dahito data GECCO 2019 paper
203 DE-scipy 2019 Varelas and Dahito data GECCO 2019 paper
204 L-BFGS-B-scipy 2019 Varelas and Dahito data GECCO 2019 paper
205 Nelder-Mead-scipy 2019 Varelas and Dahito data GECCO 2019 paper
206 Powell-scipy 2019 Varelas and Dahito data GECCO 2019 paper
207 RS-0p5-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
208 RS-0p5 2019 Brockhoff and Hansen data GECCO 2019 paper
209 RS-1-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
210 RS-1 2019 Brockhoff and Hansen data GECCO 2019 paper
211 RS-2-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
212 RS-2 2019 Brockhoff and Hansen data GECCO 2019 paper
213 RS-3-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
214 RS-3 2019 Brockhoff and Hansen data GECCO 2019 paper
215 RS-4-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
216 RS-5-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
217 RS-6-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
218 RS-6 2019 Brockhoff and Hansen data GECCO 2019 paper
219 RS-10-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
220 RS-10 2019 Brockhoff and Hansen data GECCO 2019 paper
221 RS-20-initIn0 2019 Brockhoff and Hansen data GECCO 2019 paper
222 RS-20 2019 Brockhoff and Hansen data GECCO 2019 paper
223 SLSQP-scipy 2019 Varelas and Dahito data GECCO 2019 paper
224 TNC-scipy 2019 Varelas and Dahito data GECCO 2019 paper
225 adapt-Nelder-Mead-scipy 2019 Varelas and Dahito data GECCO 2019 paper
226 HE-ES 2020 Glasmachers and Krause data continuous submission: accompanying paper - source code
227 CMA-ES-2019 2020 Hansen data continuous submission: accompanying paper - source code
228 lq-CMA-ES 2020 Hansen data continuous submission: accompanying paper - source code
229 SLSQP-11-scipy 2020 Hansen data continuous submission: accompanying paper - source code
230 SLSQP+lq-CMA-ES 2020 Hansen data continuous submission: accompanying paper - source code
231 SHADE-LM 2021 Okulewicz and Zaborski data GECCO 2021 paper
232 SHADE-LM-POP4-to-10 2021 Okulewicz and Zaborski data GECCO 2021 paper

Inofficial/incomplete data sets

Number Algorithm Name Year Author(s) link to data related PDFs, source code, and remarks
inofficial-001 BFGS-scipy 2014 Baudiš data arXiv paper
inofficial-002 CG-Fletcher-Reeves-scipy 2014 Baudiš data arXiv paper
inofficial-003 CMA-ES-python 2014 Baudiš data arXiv paper
inofficial-004 EG50-cocopf 2014 Baudiš data arXiv paper
inofficial-005 L-BFGS-B-scipy 2014 Baudiš data arXiv paper
inofficial-006 Nelder-Mead-scipy 2014 Baudiš data arXiv paper
inofficial-007 Powell-scipy 2014 Baudiš data arXiv paper
inofficial-008 SLSQP-scipy 2014 Baudiš data arXiv paper
inofficial-009 UNIF-cocopf 2014 Baudiš data arXiv paper
inofficial-010 GNN-CMA-ES 2019 Faury et al. data GECCO 2019 paper
inofficial-011 IPOP-CMA-ES-2019 2019 Faury et al. data GECCO 2019 paper