Overview page

Table showing the ERT in number of evaluations divided by the best ERT measured during BBOB-2009

Last dimension | Dimension = 2 | Next dimension

Δ fopt
f1, 2-D
1e1
2.0
1e0
6.0
1e-1
6.0
1e-2
6.0
1e-3
6.0
1e-5
6.0
1e-7
6.0
#succ
15/15
CMA-ES-Akimoto Gharafi 1.5 (1)2.6 (2)6.7 (5)13 (4)19 (4)30 (7)43 (6)15/15
CMA-ES-pycma Gharafi 1.6 (1)2.4 (2)7.1 (2)11 (4)15 (4)24 (5)33 (6)15/15
CMAES-APOP-KMA Nguyen 2.0 (2)2.8 (2)5.8 (4)10 (3)14 (4)22 (5)30 (5)15/15
CMAES-APOP-KP Nguyen 1.6 (2)2.5 (2)6.2 (4)12 (6)17 (7)28 (5)38 (5)15/15
CMAES-APOP-MA Nguyen 1.2 (1)2.7 (3)6.3 (4)8.2 (6)14 (4)23 (4)30 (6)15/15
DD-CMA-ES-Akimoto Gharafi 1.8 (2)2.9 (2)6.5 (5)13 (4)17 (5)27 (5)35 (5)15/15
DD-CMA-ES-pycma Gharafi 1.8 (2)2.3 (2)4.7 (3)10 (7)13 (6)20 (7)28 (6)15/15
EXPLO2 Huntsman 2.0 (2)4.1 (4)16 (22)39 (48)166 (198)670/15
oMads-2N Dahito 1.4 (1)1.0 (0.8)2.8 (2)4.0 (1)4.0 (1)9.2 (4)11 (2)15/15
oMads-Neg Dahito 1.6 (2)1.1 (0.8)2.9 (2)4.2 (2)4.5 (2)8.8 (2)10 (3)15/15

Δ fopt
f2, 2-D
1e1
16
1e0
19
1e-1
25
1e-2
25
1e-3
26
1e-5
28
1e-7
29
#succ
15/15
CMA-ES-Akimoto Gharafi 12 (7)16 (6)14 (3)15 (3)16 (3)17 (3)19 (2)15/15
CMA-ES-pycma Gharafi 8.3 (5)11 (3)9.4 (3)11 (3)12 (2)13 (2)14 (1)15/15
CMAES-APOP-KMA Nguyen 9.0 (8)13 (7)13 (5)16 (2)16 (2)16 (2)17 (2)15/15
CMAES-APOP-KP Nguyen 16 (11)19 (7)17 (3)18 (3)18 (3)19 (3)21 (3)15/15
CMAES-APOP-MA Nguyen 9.1 (8)15 (7)13 (3)15 (2)15 (2)16 (2)17 (1)15/15
DD-CMA-ES-Akimoto Gharafi 7.3 (3)9.0 (3)8.0 (3)9.1 (2)10 (2)11 (2)13 (2)15/15
DD-CMA-ES-pycma Gharafi 7.0 (5)7.9 (4)7.8 (2)9.0 (3)9.5 (2)10 (3)12 (3)15/15
EXPLO2 Huntsman 30 (34)670/15
oMads-2N Dahito 3.4 (2)3.7 (1)3.4 (1)4.4 (1)4.9 (1)6.8 (1)8.9 (2)15/15
oMads-Neg Dahito 3.1 (1)4.4 (2)3.8 (1)4.9 (2)5.3 (3)6.4 (2)8.2 (3)15/15

Δ fopt
f3, 2-D
1e1
15
1e0
271
1e-1
445
1e-2
446
1e-3
450
1e-5
454
1e-7
464
#succ
15/15
CMA-ES-Akimoto Gharafi 2.6 (2)2.1 (1)3.1 (2)3.8 (4)4.0 (4)4.2 (4)4.3 (4)15/15
CMA-ES-pycma Gharafi 3.0 (3)1.8 (1)4.7 (7)5.9 (7)7.0 (8)7.3 (9)7.5 (9)15/15
CMAES-APOP-KMA Nguyen 2.1 (2)4.1 (5)6.0 (6)7.0 (6)7.6 (7)8.3 (6)8.8 (6)15/15
CMAES-APOP-KP Nguyen 5.7 (3)2.8 (2)4.9 (4)6.1 (7)7.5 (7)8.5 (8)9.0 (9)15/15
CMAES-APOP-MA Nguyen 2.7 (2)3.0 (3)7.5 (6)10 (10)13 (14)15 (15)16 (15)15/15
DD-CMA-ES-Akimoto Gharafi 4.4 (2)1.9 (2)4.6 (3)4.9 (4)5.4 (4)5.7 (4)5.8 (4)15/15
DD-CMA-ES-pycma Gharafi 4.9 (3)1.7 (2)5.3 (6)7.1 (10)7.5 (12)7.7 (12)7.9 (12)15/15
EXPLO2 Huntsman 8.0 (7)3.5 (3)670/15
oMads-2N Dahito 2.4 (1)0.82 (0.7)2.0 (3)2.0 (3)2.1 (3)2.1 (3)2.1 (3)15/15
oMads-Neg Dahito 2.7 (2)0.67 (0.7)1.2 (2)1.2 (2)1.2 (1)1.3 (1)1.3 (1)15/15

Δ fopt
f4, 2-D
1e1
22
1e0
344
1e-1
459
1e-2
496
1e-3
523
1e-5
544
1e-7
566
#succ
15/15
CMA-ES-Akimoto Gharafi 4.0 (1)3.1 (5)35 (21)66 (56)111 (119)132 (117)132 (113)15/15
CMA-ES-pycma Gharafi 1.7 (1)10 (15)66 (92)112 (120)126 (126)140 (137)140 (133)15/15
CMAES-APOP-KMA Nguyen 2.8 (2)9.4 (9)91 (94)114 (102)114 (96)114 (90)111 (87)15/15
CMAES-APOP-KP Nguyen 1.7 (2)7.1 (11)47 (62)71 (68)74 (65)74 (63)72 (61)15/15
CMAES-APOP-MA Nguyen 3.4 (2)8.6 (9)99 (111)104 (104)105 (92)109 (97)108 (90)15/15
DD-CMA-ES-Akimoto Gharafi 4.1 (4)5.0 (8)23 (21)97 (135)113 (150)124 (153)125 (148)15/15
DD-CMA-ES-pycma Gharafi 3.3 (1.0)5.3 (4)48 (40)84 (85)112 (77)126 (89)128 (92)15/15
EXPLO2 Huntsman 46 (57)670/15
oMads-2N Dahito 1.7 (1)0.75 (0.8)5.1 (7)4.7 (6)4.5 (6)4.4 (6)4.3 (6)15/15
oMads-Neg Dahito 3.1 (6)1.1 (0.9)2.8 (3)2.6 (3)2.5 (3)2.5 (3)2.4 (3)15/15

Δ fopt
f5, 2-D
1e1
4.0
1e0
4.0
1e-1
4.0
1e-2
4.0
1e-3
4.0
1e-5
4.0
1e-7
4.0
#succ
15/15
CMA-ES-Akimoto Gharafi 4.1 (2)7.1 (3)7.8 (3)7.8 (3)7.8 (3)7.8 (3)7.8 (3)15/15
CMA-ES-pycma Gharafi 4.3 (2)7.2 (4)7.5 (4)7.5 (4)7.5 (4)7.5 (4)7.5 (4)15/15
CMAES-APOP-KMA Nguyen 2.3 (0.4)5.1 (1)5.3 (1)5.3 (1)5.3 (1)5.3 (1)5.3 (1)15/15
CMAES-APOP-KP Nguyen 3.5 (2)6.3 (2)6.9 (2)6.9 (2)6.9 (2)6.9 (2)6.9 (2)15/15
CMAES-APOP-MA Nguyen 2.9 (1)5.4 (3)6.1 (3)6.1 (3)6.1 (3)6.1 (3)6.1 (3)15/15
DD-CMA-ES-Akimoto Gharafi 4.0 (2)7.2 (4)7.5 (4)7.7 (4)7.7 (4)7.7 (4)7.7 (4)15/15
DD-CMA-ES-pycma Gharafi 4.5 (3)10 (10)10 (10)10 (10)10 (10)10 (10)10 (10)15/15
EXPLO2 Huntsman 0.88 (0.2)1.4 (0.6)1.4 (0.6)3.6 (5)3.6 (5)119 (147)670/15
oMads-2N Dahito 1.1 (0.1)1.6 (0.4)1.6 (0.4)1.6 (0.4)1.6 (0.4)1.6 (0.4)★21.6 (0.4)★215/15
oMads-Neg Dahito 1.1 (0.1)2.2 (0.4)2.4 (0.5)2.4 (0.5)2.4 (0.5)2.4 (0.5)2.4 (0.5)15/15

Δ fopt
f6, 2-D
1e1
13
1e0
23
1e-1
41
1e-2
54
1e-3
67
1e-5
95
1e-7
124
#succ
15/15
CMA-ES-Akimoto Gharafi 2.4 (3)3.8 (3)4.1 (1)4.6 (1)4.5 (0.8)4.6 (0.9)4.7 (0.6)15/15
CMA-ES-pycma Gharafi 2.2 (2)3.5 (1)2.9 (1)3.5 (0.6)3.7 (0.6)3.8 (0.4)3.8 (0.4)15/15
CMAES-APOP-KMA Nguyen 2.0 (3)3.7 (1)3.7 (1)3.9 (1)4.0 (1)4.0 (0.9)3.9 (0.8)15/15
CMAES-APOP-KP Nguyen 2.3 (2)3.2 (2)3.7 (1)3.8 (0.8)3.8 (0.9)4.0 (0.7)4.0 (0.4)15/15
CMAES-APOP-MA Nguyen 2.9 (3)3.9 (4)3.3 (2)3.8 (1)4.0 (1)4.0 (0.9)4.2 (0.7)15/15
DD-CMA-ES-Akimoto Gharafi 2.4 (4)3.2 (3)3.6 (2)3.8 (1)3.9 (1)4.3 (1)4.3 (0.7)15/15
DD-CMA-ES-pycma Gharafi 2.1 (2)3.5 (2)3.0 (1)3.5 (2)3.4 (1)3.5 (0.6)3.7 (0.5)15/15
EXPLO2 Huntsman 4.7 (4)9.0 (9)670/15
oMads-2N Dahito 1.0 (0.8)1.4 (0.7)1.4 (0.6)1.4 (0.4)1.4 (0.4)1.6 (0.4)1.6 (0.4)15/15
oMads-Neg Dahito 0.89 (0.7)1.4 (0.8)1.3 (0.5)1.3 (0.4)1.4 (0.4)1.5 (0.3)1.6 (0.3)15/15

Δ fopt
f7, 2-D
1e1
3.0
1e0
21
1e-1
60
1e-2
193
1e-3
217
1e-5
217
1e-7
241
#succ
15/15
CMA-ES-Akimoto Gharafi 2.5 (3)1.8 (2)2.2 (1)1.2 (0.6)1.3 (0.9)1.3 (0.9)1.4 (1)15/15
CMA-ES-pycma Gharafi 1.7 (2)2.0 (2)3.3 (2)1.3 (0.8)1.3 (0.7)1.3 (0.7)1.3 (0.7)15/15
CMAES-APOP-KMA Nguyen 2.3 (2)1.7 (2)3.3 (1)1.4 (1)1.5 (1.0)1.5 (1.0)1.6 (0.8)15/15
CMAES-APOP-KP Nguyen 2.4 (2)1.4 (1)3.1 (2)1.5 (0.9)1.5 (0.8)1.5 (0.8)1.7 (0.9)15/15
CMAES-APOP-MA Nguyen 2.4 (3)2.4 (2)2.5 (2)1.4 (0.9)1.5 (1)1.5 (1)1.9 (2)15/15
DD-CMA-ES-Akimoto Gharafi 2.3 (3)1.9 (2)1.9 (1)0.92 (0.4)1.0 (0.8)1.0 (0.8)1.2 (0.7)15/15
DD-CMA-ES-pycma Gharafi 2.2 (2)2.3 (2)2.1 (2)0.93 (0.6)0.98 (0.6)0.98 (0.6)0.98 (0.5)15/15
EXPLO2 Huntsman 10 (8)4.8 (5)16 (13)670/15
oMads-2N Dahito 2.3 (2)2.2 (3)2.7 (4)1.1 (2)1.2 (2)1.2 (2)1.2 (1)15/15
oMads-Neg Dahito 2.2 (2)1.1 (0.8)2.8 (5)1.1 (1)1.1 (1)1.1 (1)1.1 (1)15/15

Δ fopt
f8, 2-D
1e1
5.0
1e0
12
1e-1
37
1e-2
46
1e-3
86
1e-5
94
1e-7
112
#succ
15/15
CMA-ES-Akimoto Gharafi 3.1 (3)6.1 (8)7.0 (6)8.6 (5)5.1 (2)5.7 (2)5.3 (2)15/15
CMA-ES-pycma Gharafi 3.5 (4)8.9 (16)7.0 (8)7.1 (7)4.5 (3)4.8 (3)4.5 (3)15/15
CMAES-APOP-KMA Nguyen 3.2 (3)5.5 (4)5.9 (5)7.2 (4)4.7 (2)5.0 (2)4.6 (2)15/15
CMAES-APOP-KP Nguyen 5.2 (6)7.1 (9)6.3 (5)7.4 (5)5.0 (2)5.4 (2)5.1 (1)15/15
CMAES-APOP-MA Nguyen 2.9 (4)8.1 (10)7.1 (8)8.3 (6)5.1 (3)5.4 (3)5.0 (2)15/15
DD-CMA-ES-Akimoto Gharafi 4.4 (5)11 (22)7.4 (9)7.4 (8)4.8 (4)5.4 (4)5.1 (3)15/15
DD-CMA-ES-pycma Gharafi 4.5 (4)9.4 (14)7.4 (5)7.5 (5)4.4 (2)4.8 (2)4.5 (2)15/15
EXPLO2 Huntsman 13 (17)19 (23)670/15
oMads-2N Dahito 1.8 (2)3.2 (3)2.2 (1)2.3 (1)1.5 (0.5)1.8 (0.6)2.0 (0.4)15/15
oMads-Neg Dahito 2.0 (3)3.0 (3)2.0 (1)2.2 (1)1.5 (0.8)1.7 (0.7)1.8 (0.6)15/15

Δ fopt
f9, 2-D
1e1
1
1e0
18
1e-1
30
1e-2
44
1e-3
68
1e-5
81
1e-7
92
#succ
15/15
CMA-ES-Akimoto Gharafi 1 (0)3.2 (3)6.3 (6)6.8 (3)5.2 (2)5.3 (2)5.6 (2)15/15
CMA-ES-pycma Gharafi 1 (0)2.7 (2)5.5 (4)5.3 (3)4.0 (2)4.1 (2)4.2 (2)15/15
CMAES-APOP-KMA Nguyen 1 (0)2.2 (1)7.0 (5)7.3 (3)5.2 (2)5.4 (2)5.4 (2)15/15
CMAES-APOP-KP Nguyen 1 (0)4.0 (5)8.2 (7)8.7 (5)6.6 (3)6.4 (2)6.3 (2)15/15
CMAES-APOP-MA Nguyen 1 (0)2.4 (3)7.7 (6)7.5 (5)5.8 (3)5.6 (3)5.6 (2)15/15
DD-CMA-ES-Akimoto Gharafi 1 (0)2.4 (2)6.4 (5)6.4 (4)4.9 (3)5.2 (2)5.1 (2)15/15
DD-CMA-ES-pycma Gharafi 1 (0)1.8 (1)3.8 (2)4.4 (1)3.5 (1)3.8 (0.9)3.9 (0.8)15/15
EXPLO2 Huntsman 60 (36)54 (64)670/15
oMads-2N Dahito 1 (0)1.0 (0.6)1.4 (0.9)1.9 (1)1.6 (0.7)1.8 (0.6)2.0 (0.6)15/15
oMads-Neg Dahito 1 (0)1.3 (2)1.5 (2)1.6 (2)1.4 (1)1.5 (0.9)1.9 (0.9)15/15

Δ fopt
f10, 2-D
1e1
30
1e0
46
1e-1
54
1e-2
61
1e-3
68
1e-5
82
1e-7
98
#succ
15/15
CMA-ES-Akimoto Gharafi 6.8 (4)6.1 (3)6.3 (1.0)6.2 (1)6.3 (0.6)5.9 (0.6)5.7 (0.5)15/15
CMA-ES-pycma Gharafi 5.1 (3)4.8 (1)5.0 (0.9)5.0 (0.8)4.8 (0.8)4.7 (0.7)4.5 (0.6)15/15
CMAES-APOP-KMA Nguyen 5.1 (4)5.9 (3)6.5 (0.9)6.4 (0.9)6.0 (0.9)5.6 (0.7)5.1 (0.5)15/15
CMAES-APOP-KP Nguyen 7.1 (7)8.3 (3)8.5 (2)8.2 (1)8.0 (0.9)7.3 (0.7)6.6 (0.6)15/15
CMAES-APOP-MA Nguyen 5.2 (4)6.1 (3)6.1 (1)6.2 (0.9)6.0 (0.6)5.5 (0.6)5.0 (0.6)15/15
DD-CMA-ES-Akimoto Gharafi 5.3 (4)6.2 (2)6.5 (0.9)6.2 (0.8)6.1 (0.7)5.7 (0.6)5.6 (0.5)15/15
DD-CMA-ES-pycma Gharafi 5.8 (3)5.2 (2)5.1 (1)4.9 (1)4.9 (1)4.8 (1)4.5 (0.9)15/15
EXPLO2 Huntsman 670/15
oMads-2N Dahito 2.1 (1)2.7 (2)2.6 (2)2.9 (1)3.1 (0.9)4.1 (1)4.7 (3)15/15
oMads-Neg Dahito 2.0 (2)2.4 (1)2.5 (1)2.5 (1)2.8 (1)3.6 (1)3.7 (1)15/15

Δ fopt
f11, 2-D
1e1
35
1e0
45
1e-1
50
1e-2
62
1e-3
67
1e-5
81
1e-7
97
#succ
15/15
CMA-ES-Akimoto Gharafi 5.4 (3)6.7 (2)7.0 (1)6.2 (1)6.2 (1)5.9 (1)5.7 (1.0)15/15
CMA-ES-pycma Gharafi 3.3 (3)4.6 (2)4.9 (1)4.7 (0.9)4.8 (0.9)4.6 (0.6)4.5 (0.4)15/15
CMAES-APOP-KMA Nguyen 4.2 (4)5.2 (4)7.0 (1)6.2 (0.8)6.1 (0.8)5.7 (0.7)5.2 (0.4)15/15
CMAES-APOP-KP Nguyen 5.4 (5)7.7 (4)8.4 (3)7.6 (2)7.5 (2)6.9 (1)6.5 (1)15/15
CMAES-APOP-MA Nguyen 4.5 (4)5.3 (3)6.0 (4)5.7 (2)5.8 (1)5.4 (0.8)5.1 (0.8)15/15
DD-CMA-ES-Akimoto Gharafi 5.9 (3)6.5 (3)6.8 (2)6.2 (1)6.2 (1)5.9 (0.7)5.6 (0.7)15/15
DD-CMA-ES-pycma Gharafi 4.1 (3)4.7 (2)5.2 (2)4.6 (1)4.6 (1)4.6 (0.9)4.5 (0.8)15/15
EXPLO2 Huntsman 670/15
oMads-2N Dahito 1.7 (1)2.3 (2)3.1 (2)3.1 (1)3.4 (1.0)3.5 (1.0)3.7 (1)15/15
oMads-Neg Dahito 1.8 (0.7)2.3 (1)2.7 (1.0)2.8 (0.8)3.0 (0.7)3.4 (1)3.8 (1.0)15/15

Δ fopt
f12, 2-D
1e1
35
1e0
46
1e-1
75
1e-2
94
1e-3
105
1e-5
153
1e-7
195
#succ
15/15
CMA-ES-Akimoto Gharafi 4.6 (4)5.6 (3)5.0 (2)5.2 (2)5.5 (3)5.1 (5)5.0 (5)15/15
CMA-ES-pycma Gharafi 3.8 (2)5.6 (4)6.4 (10)7.0 (10)7.0 (9)6.2 (8)6.0 (8)15/15
CMAES-APOP-KMA Nguyen 4.2 (4)9.2 (6)8.8 (6)8.6 (7)8.9 (7)7.5 (7)6.8 (7)15/15
CMAES-APOP-KP Nguyen 5.0 (6)13 (11)13 (16)12 (17)13 (19)11 (14)10 (11)15/15
CMAES-APOP-MA Nguyen 4.1 (4)9.0 (8)9.4 (12)11 (13)11 (13)15 (22)13 (20)15/15
DD-CMA-ES-Akimoto Gharafi 3.7 (2)9.5 (14)10 (15)10 (15)9.4 (14)8.3 (13)7.4 (10)15/15
DD-CMA-ES-pycma Gharafi 5.6 (4)6.8 (3)5.9 (3)5.7 (3)5.5 (3)4.6 (3)4.2 (2)15/15
EXPLO2 Huntsman 670/15
oMads-2N Dahito 1.5 (0.9)2.2 (2)2.3 (3)2.6 (3)2.8 (3)2.5 (2)2.5 (2)15/15
oMads-Neg Dahito 1.2 (0.7)1.5 (1)1.8 (1)2.1 (3)2.3 (3)2.2 (2)2.1 (2)15/15

Δ fopt
f13, 2-D
1e1
23
1e0
35
1e-1
46
1e-2
60
1e-3
71
1e-5
95
1e-7
122
#succ
15/15
CMA-ES-Akimoto Gharafi 2.7 (2)5.1 (2)5.5 (2)5.1 (1)5.5 (1)5.8 (0.9)5.7 (0.9)15/15
CMA-ES-pycma Gharafi 2.9 (2)3.5 (2)4.2 (1)4.0 (1)4.3 (0.8)4.4 (0.7)4.5 (0.5)15/15
CMAES-APOP-KMA Nguyen 3.6 (5)5.7 (3)5.8 (3)5.7 (1)6.1 (1)6.2 (1)5.9 (1)15/15
CMAES-APOP-KP Nguyen 3.9 (5)6.6 (4)7.2 (2)6.3 (2)6.8 (2)7.0 (2)7.0 (1)15/15
CMAES-APOP-MA Nguyen 3.8 (5)5.7 (3)6.2 (2)5.7 (2)6.0 (2)6.0 (1)6.2 (1)15/15
DD-CMA-ES-Akimoto Gharafi 2.5 (2)4.4 (3)5.1 (2)5.0 (2)5.4 (1)5.6 (1.0)5.7 (0.7)15/15
DD-CMA-ES-pycma Gharafi 3.0 (2)3.5 (1)4.1 (2)4.0 (0.9)4.1 (1.0)4.4 (0.6)4.3 (0.6)15/15
EXPLO2 Huntsman 21 (20)670/15
oMads-2N Dahito 1.4 (0.6)1.9 (1)3.0 (2)4.3 (3)6.4 (4)35 (41)1303 (1751)3/15
oMads-Neg Dahito 1.2 (0.5)2.1 (2)2.3 (1)2.5 (1)3.8 (4)18 (31)2.4e4 (3e4)1/15

Δ fopt
f14, 2-D
1e1
1
1e0
7.0
1e-1
16
1e-2
24
1e-3
38
1e-5
67
1e-7
90
#succ
15/15
CMA-ES-Akimoto Gharafi 2.7 (4)2.5 (3)3.5 (2)4.3 (2)4.4 (2)5.3 (1)5.8 (0.8)15/15
CMA-ES-pycma Gharafi 1.7 (2)2.3 (2)3.1 (1)3.9 (2)3.8 (1)4.0 (0.9)4.3 (0.8)15/15
CMAES-APOP-KMA Nguyen 1.8 (1)1.7 (2)3.0 (2)3.7 (1)3.8 (2)4.6 (2)5.6 (0.8)15/15
CMAES-APOP-KP Nguyen 1.9 (2)2.5 (2)3.4 (1)3.7 (1)4.5 (2)6.3 (2)6.9 (2)15/15
CMAES-APOP-MA Nguyen 2.1 (2)2.4 (2)2.5 (1)3.8 (2)4.1 (1)5.0 (1)5.8 (0.8)15/15
DD-CMA-ES-Akimoto Gharafi 2.3 (4)2.5 (2)2.8 (3)4.1 (1)4.5 (2)4.5 (1.0)5.3 (0.8)15/15
DD-CMA-ES-pycma Gharafi 1.9 (2)1.3 (1)2.7 (2)3.4 (1)3.6 (1)3.8 (1)4.1 (1.0)15/15
EXPLO2 Huntsman 3.6 (4)4.1 (4)20 (22)670/15
oMads-2N Dahito 1.8 (2)1.1 (1)1.7 (1)2.0 (0.9)1.9 (1)3.0 (0.9)6.4 (4)15/15
oMads-Neg Dahito 1.9 (2)1.1 (0.9)1.6 (0.8)1.6 (0.8)1.6 (0.5)2.5 (0.8)4.9 (4)15/15

Δ fopt
f15, 2-D
1e1
37
1e0
291
1e-1
1033
1e-2
1066
1e-3
1113
1e-5
1231
1e-7
1412
#succ
15/15
CMA-ES-Akimoto Gharafi 1.5 (2)2.5 (3)2.1 (3)2.5 (3)2.5 (3)2.4 (2)2.2 (2)15/15
CMA-ES-pycma Gharafi 1.2 (1)1.9 (2)1.7 (2)1.8 (2)1.8 (2)1.7 (2)1.6 (1)15/15
CMAES-APOP-KMA Nguyen 1.1 (0.8)1.8 (3)2.3 (3)2.4 (3)2.4 (3)2.4 (3)2.2 (3)15/15
CMAES-APOP-KP Nguyen 2.2 (0.7)2.0 (2)2.3 (3)2.5 (3)2.6 (3)2.5 (3)2.3 (3)15/15
CMAES-APOP-MA Nguyen 0.82 (0.7)2.5 (2)2.0 (2)2.8 (2)3.6 (4)3.7 (4)3.5 (4)15/15
DD-CMA-ES-Akimoto Gharafi 0.85 (0.6)2.0 (3)1.8 (1)2.5 (3)2.4 (3)2.3 (2)2.1 (2)15/15
DD-CMA-ES-pycma Gharafi 1.2 (0.9)3.2 (3)1.9 (1)1.9 (1)1.9 (1)1.8 (1)1.6 (0.9)15/15
EXPLO2 Huntsman 2.1 (3)670/15
oMads-2N Dahito 0.65 (0.5)2.0 (2)1.8 (3)1.8 (2)1.7 (2)1.6 (2)1.4 (2)15/15
oMads-Neg Dahito 0.59 (0.5)1.3 (1)1.5 (2)1.5 (2)1.4 (2)1.3 (1)1.2 (1)15/15

Δ fopt
f16, 2-D
1e1
9.0
1e0
50
1e-1
174
1e-2
326
1e-3
358
1e-5
409
1e-7
538
#succ
15/15
CMA-ES-Akimoto Gharafi 2.0 (1)3.4 (2)3.7 (3)2.2 (2)2.9 (3)2.8 (3)2.3 (2)15/15
CMA-ES-pycma Gharafi 4.2 (5)4.3 (8)2.5 (3)2.8 (4)3.2 (4)3.0 (4)2.4 (3)15/15
CMAES-APOP-KMA Nguyen 2.6 (2)4.2 (7)4.7 (5)3.2 (4)3.3 (5)3.8 (5)3.2 (4)15/15
CMAES-APOP-KP Nguyen 1.9 (2)5.0 (7)7.0 (7)4.5 (5)7.9 (5)7.4 (5)6.1 (4)15/15
CMAES-APOP-MA Nguyen 1.6 (1)4.6 (7)18 (14)12 (20)15 (20)14 (18)12 (15)15/15
DD-CMA-ES-Akimoto Gharafi 2.3 (2)3.6 (4)1.7 (2)1.4 (2)2.0 (2)2.0 (2)1.6 (1)15/15
DD-CMA-ES-pycma Gharafi 4.2 (3)2.7 (2)1.9 (3)2.1 (2)2.4 (3)2.4 (3)2.0 (2)15/15
EXPLO2 Huntsman 3.0 (2)6.4 (7)670/15
oMads-2N Dahito 3.6 (2)4.5 (9)2.1 (4)1.4 (2)1.3 (2)1.2 (2)1.0 (1)15/15
oMads-Neg Dahito 1.3 (1)1.5 (3)0.92 (1)0.65 (0.6)0.64 (0.6)1.2 (1)1.0 (0.8)15/15

Δ fopt
f17, 2-D
1e1
3.0
1e0
61
1e-1
133
1e-2
275
1e-3
396
1e-5
1086
1e-7
1657
#succ
15/15
CMA-ES-Akimoto Gharafi 1.3 (2)1.2 (0.8)1.5 (0.7)1.1 (0.4)1.2 (1.0)1.2 (1)1.0 (0.7)15/15
CMA-ES-pycma Gharafi 1.2 (1)2.2 (0.5)1.6 (0.3)1.1 (0.3)1.4 (1)1.3 (0.9)1.0 (0.7)15/15
CMAES-APOP-KMA Nguyen 2.2 (3)3.0 (5)2.3 (3)1.9 (2)1.7 (1)1.3 (0.8)1.2 (0.6)15/15
CMAES-APOP-KP Nguyen 1.2 (2)2.5 (6)1.9 (3)1.3 (2)1.4 (1)1.4 (0.8)1.3 (0.7)15/15
CMAES-APOP-MA Nguyen 1.1 (1)3.7 (7)3.2 (6)2.0 (3)2.2 (3)1.7 (2)1.5 (1)15/15
DD-CMA-ES-Akimoto Gharafi 1.3 (1)1.5 (0.7)2.0 (3)1.3 (1)1.4 (1)1.3 (0.9)1.1 (0.5)15/15
DD-CMA-ES-pycma Gharafi 1.2 (2)1.8 (3)1.8 (2)1.3 (1)1.6 (0.8)1.2 (0.7)0.98 (0.5)15/15
EXPLO2 Huntsman 2.1 (2)3.5 (3)670/15
oMads-2N Dahito 2.0 (2)1.4 (3)0.97 (1)1.2 (2)1.1 (1)1.2 (0.9)69 (68)7/15
oMads-Neg Dahito 11 (3)1.9 (3)1.3 (1)0.84 (0.7)0.90 (0.6)2.0 (2)88 (134)6/15

Δ fopt
f18, 2-D
1e1
19
1e0
134
1e-1
666
1e-2
1249
1e-3
1708
1e-5
2438
1e-7
2858
#succ
15/15
CMA-ES-Akimoto Gharafi 1.4 (2)2.5 (3)1.2 (0.7)1.00 (0.8)0.88 (0.6)0.96 (0.4)1.1 (0.5)15/15
CMA-ES-pycma Gharafi 1.3 (0.8)3.1 (3)1.1 (1)0.74 (0.8)0.65 (0.6)0.83 (0.5)0.81 (0.4)15/15
CMAES-APOP-KMA Nguyen 1.1 (1)5.4 (3)3.2 (4)2.6 (3)2.1 (3)1.9 (2)1.8 (2)15/15
CMAES-APOP-KP Nguyen 1.5 (2)7.0 (5)2.0 (1)1.4 (0.6)1.3 (0.5)1.2 (0.5)1.2 (0.6)15/15
CMAES-APOP-MA Nguyen 1.2 (0.9)4.6 (4)1.6 (1)2.0 (1)1.9 (1)1.7 (1)1.7 (1)15/15
DD-CMA-ES-Akimoto Gharafi 1.3 (1)3.1 (3)1.3 (2)1.0 (0.9)0.85 (0.7)0.94 (0.5)0.90 (0.4)15/15
DD-CMA-ES-pycma Gharafi 1.2 (0.6)3.0 (3)1.1 (1)0.81 (0.7)0.67 (0.5)0.97 (0.8)1.1 (0.7)15/15
EXPLO2 Huntsman 2.7 (3)7.4 (7)670/15
oMads-2N Dahito 0.90 (0.7)3.5 (5)1.3 (1)2.8 (4)3.9 (4)54 (54)489 (363)1/15
oMads-Neg Dahito 1.5 (0.7)2.1 (2)1.2 (1)2.7 (3)4.5 (4)174 (189)1039 (1942)1/15

Δ fopt
f19, 2-D
1e1
1
1e0
1
1e-1
26
1e-2
216
1e-3
227
1e-5
252
1e-7
276
#succ
15/15
CMA-ES-Akimoto Gharafi 1 (0)1 (0)10 (21)10 (12)10 (13)11 (11)11 (11)15/15
CMA-ES-pycma Gharafi 1 (0)1 (0)2.4 (2)7.3 (7)10 (12)9.2 (12)8.9 (10)15/15
CMAES-APOP-KMA Nguyen 1 (0)1 (0)2.9 (4)2.0 (3)24 (47)23 (45)22 (43)15/15
CMAES-APOP-KP Nguyen 1 (0)1 (0)6.6 (7)14 (7)14 (6)14 (8)14 (7)15/15
CMAES-APOP-MA Nguyen 1 (0)1 (0)11 (17)13 (10)18 (28)24 (34)29 (34)15/15
DD-CMA-ES-Akimoto Gharafi 1 (0)1 (0)17 (29)14 (14)15 (14)14 (12)14 (12)15/15
DD-CMA-ES-pycma Gharafi 1 (0)1 (0)19 (40)26 (42)30 (40)32 (57)30 (53)15/15
EXPLO2 Huntsman 10 (14)62 (54)19 (29)670/15
oMads-2N Dahito 1 (0)1 (0)2.2 (1)7.5 (9)7.1 (8)6.5 (7)6.0 (7)15/15
oMads-Neg Dahito 1 (0)1 (0)0.98 (0.8)6.2 (8)6.0 (8)5.4 (7)5.0 (6)15/15

Δ fopt
f20, 2-D
1e1
4.0
1e0
61
1e-1
365
1e-2
366
1e-3
366
1e-5
370
1e-7
375
#succ
15/15
CMA-ES-Akimoto Gharafi 1.8 (1)13 (16)6.9 (8)7.7 (8)8.2 (9)8.6 (10)8.9 (10)15/15
CMA-ES-pycma Gharafi 1.7 (2)13 (14)13 (12)15 (15)16 (16)16 (17)17 (17)15/15
CMAES-APOP-KMA Nguyen 1.6 (2)21 (31)23 (30)29 (30)32 (32)34 (32)34 (32)15/15
CMAES-APOP-KP Nguyen 1.3 (1)14 (17)8.8 (10)10 (11)11 (11)12 (11)12 (11)15/15
CMAES-APOP-MA Nguyen 1.4 (1)30 (10)23 (30)26 (32)31 (32)33 (32)35 (32)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (1)11 (12)9.0 (9)10 (9)11 (8)12 (8)13 (8)15/15
DD-CMA-ES-pycma Gharafi 1.6 (2)17 (22)10 (8)11 (9)12 (9)12 (10)13 (9)15/15
EXPLO2 Huntsman 4.1 (3)670/15
oMads-2N Dahito 0.75 (0.4)1.8 (3)3.4 (4)3.4 (4)3.5 (4)3.5 (4)3.5 (4)15/15
oMads-Neg Dahito 0.80 (0.2)6.5 (10)4.4 (4)4.4 (4)4.5 (4)4.5 (4)4.5 (4)15/15

Δ fopt
f21, 2-D
1e1
2.0
1e0
51
1e-1
174
1e-2
276
1e-3
290
1e-5
324
1e-7
330
#succ
15/15
CMA-ES-Akimoto Gharafi 0.77 (0.5)6.6 (12)7.3 (8)6.6 (14)8.7 (15)10 (14)10 (14)15/15
CMA-ES-pycma Gharafi 0.80 (0.8)3.8 (7)2.5 (2)5.2 (9)7.0 (9)6.9 (8)7.4 (9)15/15
CMAES-APOP-KMA Nguyen 0.87 (0.8)10 (17)4.2 (8)3.2 (5)4.5 (7)14 (22)14 (22)15/15
CMAES-APOP-KP Nguyen 0.90 (0.8)13 (32)15 (25)11 (25)11 (23)10 (21)10 (21)15/15
CMAES-APOP-MA Nguyen 0.83 (0.5)7.7 (9)7.6 (5)5.2 (3)5.5 (4)15 (20)19 (35)15/15
DD-CMA-ES-Akimoto Gharafi 0.73 (0.5)8.0 (7)7.2 (7)7.4 (8)7.7 (7)7.3 (7)7.7 (7)15/15
DD-CMA-ES-pycma Gharafi 0.83 (0.5)6.5 (9)6.0 (6)10 (12)10 (12)9.4 (13)10 (13)15/15
EXPLO2 Huntsman 1.4 (1)3.3 (3)1.9 (2)3.6 (5)3.4 (4)670/15
oMads-2N Dahito 0.93 (0.8)5.4 (7)2.9 (3)1.9 (2)1.8 (2)1.6 (1)1.6 (1)15/15
oMads-Neg Dahito 0.97 (0.8)4.5 (6)3.0 (3)2.0 (2)1.9 (2)1.7 (2)1.7 (2)15/15

Δ fopt
f22, 2-D
1e1
5.0
1e0
27
1e-1
168
1e-2
218
1e-3
249
1e-5
289
1e-7
306
#succ
15/15
CMA-ES-Akimoto Gharafi 0.96 (1)10 (10)6.2 (12)5.6 (9)5.5 (10)17 (9)17 (8)15/15
CMA-ES-pycma Gharafi 1.3 (1)10 (11)6.9 (4)12 (19)56 (18)82 (269)80 (267)15/15
CMAES-APOP-KMA Nguyen 0.79 (0.8)3.5 (9)15 (31)13 (24)18 (37)19 (33)19 (33)15/15
CMAES-APOP-KP Nguyen 1.2 (1)6.4 (10)7.5 (9)8.4 (10)8.3 (8)7.7 (8)7.6 (8)15/15
CMAES-APOP-MA Nguyen 1.1 (2)11 (10)12 (21)22 (42)21 (40)20 (34)20 (33)15/15
DD-CMA-ES-Akimoto Gharafi 1.3 (1)13 (38)4.9 (7)32 (21)32 (29)75 (162)72 (154)15/15
DD-CMA-ES-pycma Gharafi 1.1 (1)5.7 (10)8.5 (11)10 (17)8.8 (15)7.8 (13)25 (54)15/15
EXPLO2 Huntsman 2.2 (3)3.6 (3)5.9 (8)4.5 (5)4.0 (6)670/15
oMads-2N Dahito 1.5 (2)4.3 (4)1.4 (2)1.1 (1)0.97 (1)0.88 (0.9)0.88 (0.8)15/15
oMads-Neg Dahito 1.7 (2)6.3 (6)2.0 (3)1.6 (2)1.4 (2)1.3 (2)1.2 (2)15/15

Δ fopt
f23, 2-D
1e1
8.0
1e0
193
1e-1
234
1e-2
263
1e-3
299
1e-5
348
1e-7
379
#succ
15/15
CMA-ES-Akimoto Gharafi 1.9 (2)9.4 (8)13 (12)15 (15)14 (13)12 (12)12 (11)15/15
CMA-ES-pycma Gharafi 2.3 (2)5.4 (5)13 (15)12 (14)11 (12)10 (11)9.3 (10)15/15
CMAES-APOP-KMA Nguyen 1.6 (2)12 (16)104 (94)113 (96)102 (87)91 (78)86 (74)15/15
CMAES-APOP-KP Nguyen 2.1 (3)9.0 (12)81 (84)93 (88)84 (79)76 (70)72 (65)15/15
CMAES-APOP-MA Nguyen 2.0 (2)5.5 (8)85 (89)85 (96)77 (86)69 (77)65 (72)15/15
DD-CMA-ES-Akimoto Gharafi 2.0 (3)6.8 (7)6.3 (5)6.3 (5)5.8 (4)5.5 (4)6.4 (4)15/15
DD-CMA-ES-pycma Gharafi 1.9 (1)5.0 (5)8.0 (11)7.5 (11)6.9 (9)6.3 (8)6.3 (8)15/15
EXPLO2 Huntsman 1.9 (2)670/15
oMads-2N Dahito 1.2 (2)0.92 (1.0)1.3 (1)1.4 (1)1.5 (0.9)4106 (3411)2e50/15
oMads-Neg Dahito 1.3 (2)1.3 (1)1.3 (1)1.3 (1)1.4 (1.0)2524 (2655)2e50/15

Δ fopt
f24, 2-D
1e1
18
1e0
857
1e-1
8515
1e-2
23399
1e-3
24113
1e-5
24721
1e-7
24721
#succ
5/15
CMA-ES-Akimoto Gharafi 1.6 (2)86 (102)31 (47)39 (41)38 (40)47 (63)47 (48)9/15
CMA-ES-pycma Gharafi 1.4 (0.8)93 (151)492 (339)278 (321)582 (799)568 (770)568 (1337)1/15
CMAES-APOP-KMA Nguyen 3.0 (1)20 (43)7.2 (6)7.1 (11)7.1 (10)7.1 (8)7.1 (9)12/15
CMAES-APOP-KP Nguyen 1.4 (2)50 (62)10 (18)7.0 (8)8.0 (9)8.2 (9)8.2 (11)13/15
CMAES-APOP-MA Nguyen 1.1 (1)49 (65)6.9 (7)5.6 (5)6.1 (5)6.2 (5)6.3 (5)14/15
DD-CMA-ES-Akimoto Gharafi 1.4 (1)14 (9)24 (32)19 (18)19 (18)19 (17)19 (17)14/15
DD-CMA-ES-pycma Gharafi 1.2 (0.8)51 (34)92 (134)56 (96)72 (125)71 (83)71 (53)6/15
EXPLO2 Huntsman 1.1 (1)670/15
oMads-2N Dahito 3.6 (6)2.0 (2)0.63 (0.8)0.89 (0.9)0.86 (0.9)0.84 (0.9)0.85 (0.9)15/15
oMads-Neg Dahito 1.9 (2)1.2 (1)2.3 (3)1.3 (1)1.3 (1)1.3 (1)1.3 (1)15/15

Previous dimension | Dimension = 3 | Next dimension

Δ fopt
f1, 3-D
1e1
4.0
1e0
8.0
1e-1
8.0
1e-2
8.0
1e-3
8.0
1e-5
8.0
1e-7
8.0
#succ
15/15
CMA-ES-Akimoto Gharafi 1.5 (1)3.8 (3)11 (5)17 (3)23 (4)35 (7)47 (5)15/15
CMA-ES-pycma Gharafi 1.4 (2)3.9 (2)8.6 (3)14 (2)20 (5)30 (4)43 (6)15/15
CMAES-APOP-KMA Nguyen 1.7 (1)4.1 (3)7.6 (2)12 (5)16 (4)24 (3)32 (4)15/15
CMAES-APOP-KP Nguyen 2.1 (2)4.1 (3)8.6 (1)14 (4)21 (5)33 (5)45 (7)15/15
CMAES-APOP-MA Nguyen 1.2 (1)3.5 (2)8.4 (2)13 (3)17 (3)27 (4)34 (4)15/15
DD-CMA-ES-Akimoto Gharafi 1.9 (2)4.6 (4)11 (7)16 (5)20 (8)33 (6)42 (6)15/15
DD-CMA-ES-pycma Gharafi 1.8 (2)4.1 (3)8.5 (4)14 (5)19 (4)29 (4)38 (5)15/15
EXPLO2 Huntsman 2.3 (2)4.4 (2)7.8 (2)22 (13)185 (200)1000/15
oMads-2N Dahito 1.0 (0.8)1.3 (1)4.4 (3)6.0 (2)6.1 (2)12 (3)13 (2)15/15
oMads-Neg Dahito 1.2 (1)1.6 (1)4.2 (2)5.6 (2)6.1 (2)11 (4)12 (1)15/15

Δ fopt
f2, 3-D
1e1
38
1e0
42
1e-1
43
1e-2
44
1e-3
45
1e-5
47
1e-7
48
#succ
15/15
CMA-ES-Akimoto Gharafi 8.6 (4)11 (4)13 (2)14 (2)15 (2)16 (2)18 (2)15/15
CMA-ES-pycma Gharafi 6.7 (3)8.7 (2)10 (1)11 (1)12 (1)13 (1)15 (1)15/15
CMAES-APOP-KMA Nguyen 10 (4)12 (5)13 (4)15 (2)15 (2)16 (2)17 (1)15/15
CMAES-APOP-KP Nguyen 13 (6)16 (5)18 (2)18 (2)19 (2)21 (2)22 (3)15/15
CMAES-APOP-MA Nguyen 8.3 (4)10 (4)12 (4)14 (2)14 (2)15 (2)16 (2)15/15
DD-CMA-ES-Akimoto Gharafi 5.7 (2)7.0 (2)7.7 (2)8.6 (2)9.4 (1)11 (2)12 (2)15/15
DD-CMA-ES-pycma Gharafi 4.8 (2)5.8 (2)6.8 (2)7.6 (2)8.3 (2)10 (2)11 (2)15/15
EXPLO2 Huntsman 1000/15
oMads-2N Dahito 2.9 (0.9)3.2 (1)4.6 (2)5.8 (4)6.8 (3)8.9 (4)11 (4)15/15
oMads-Neg Dahito 2.8 (1)3.6 (2)4.8 (2)6.1 (3)6.7 (3)8.5 (3)10 (3)15/15

Δ fopt
f3, 3-D
1e1
38
1e0
822
1e-1
830
1e-2
835
1e-3
842
1e-5
847
1e-7
853
#succ
15/15
CMA-ES-Akimoto Gharafi 3.8 (3)2.6 (3)15 (18)16 (19)16 (19)16 (20)17 (21)15/15
CMA-ES-pycma Gharafi 2.3 (2)2.0 (2)11 (12)12 (14)12 (14)13 (14)13 (15)15/15
CMAES-APOP-KMA Nguyen 5.4 (11)7.2 (6)30 (33)30 (33)30 (34)31 (34)31 (34)15/15
CMAES-APOP-KP Nguyen 4.2 (2)4.8 (4)26 (28)27 (28)27 (28)28 (28)28 (28)15/15
CMAES-APOP-MA Nguyen 2.8 (2)5.5 (7)23 (26)28 (54)29 (54)30 (55)31 (55)15/15
DD-CMA-ES-Akimoto Gharafi 2.4 (1)1.6 (2)6.2 (4)6.6 (5)6.7 (5)7.0 (5)7.2 (5)15/15
DD-CMA-ES-pycma Gharafi 5.9 (12)2.0 (2)5.6 (4)7.9 (8)8.1 (8)8.5 (9)8.7 (9)15/15
EXPLO2 Huntsman 4.9 (7)1000/15
oMads-2N Dahito 3.1 (1)1.0 (1)8.5 (8)8.4 (8)8.4 (8)8.4 (8)8.5 (8)15/15
oMads-Neg Dahito 2.5 (1)0.82 (0.9)5.4 (9)5.4 (9)5.4 (9)5.4 (9)5.5 (9)15/15

Δ fopt
f4, 3-D
1e1
40
1e0
808
1e-1
866
1e-2
921
1e-3
952
1e-5
1015
1e-7
1044
#succ
15/15
CMA-ES-Akimoto Gharafi 3.5 (2)256 (249)8051 (1e4)1.2e4 (9708)1.1e4 (1e4)1.1e4 (9452)1.0e4 (1e4)2/15
CMA-ES-pycma Gharafi 5.4 (2)199 (224)1.3e4 (1e4)2.4e4 (2e4)2.3e4 (3e4)2.2e4 (3e4)2.1e4 (3e4)1/15
CMAES-APOP-KMA Nguyen 9.2 (10)175 (173)4661 (6756)4386 (4561)4245 (6619)3983 (5664)3875 (6082)2/15
CMAES-APOP-KP Nguyen 8.9 (14)110 (159)1040 (1386)995 (1591)965 (1075)906 (995)882 (700)7/15
CMAES-APOP-MA Nguyen 5.0 (3)206 (215)4815 (4698)9139 (9122)8842 (1e4)8297 (9608)8070 (7186)1/15
DD-CMA-ES-Akimoto Gharafi 8.6 (11)466 (496)1.3e4 (2e4)2.5e4 (2e4)2.4e4 (2e4)2.2e4 (3e4)2.2e4 (2e4)1/15
DD-CMA-ES-pycma Gharafi 5.6 (10)290 (357)2.5e4 (3e4)2.4e4 (2e4)2.3e4 (3e4)2.2e4 (3e4)2.1e4 (1e4)1/15
EXPLO2 Huntsman 1000/15
oMads-2N Dahito 4.4 (5)4.3 (4)31 (37)30 (35)29 (33)27 (31)26 (30)15/15
oMads-Neg Dahito 10 (16)4.5 (5)40 (48)38 (45)36 (43)34 (41)33 (39)15/15

Δ fopt
f5, 3-D
1e1
7.0
1e0
7.0
1e-1
7.0
1e-2
7.0
1e-3
7.0
1e-5
7.0
1e-7
7.0
#succ
15/15
CMA-ES-Akimoto Gharafi 3.2 (2)5.0 (2)5.3 (2)5.4 (2)5.4 (2)5.4 (2)5.4 (2)15/15
CMA-ES-pycma Gharafi 3.8 (4)6.2 (4)6.5 (4)6.5 (4)6.5 (4)6.5 (4)6.5 (4)15/15
CMAES-APOP-KMA Nguyen 3.2 (2)4.1 (2)4.3 (2)4.3 (2)4.3 (2)4.3 (2)4.3 (2)15/15
CMAES-APOP-KP Nguyen 3.4 (2)5.2 (2)5.3 (2)5.4 (2)5.4 (2)5.4 (2)5.4 (2)15/15
CMAES-APOP-MA Nguyen 3.0 (0.9)4.1 (2)4.4 (2)4.4 (2)4.4 (2)4.4 (2)4.4 (2)15/15
DD-CMA-ES-Akimoto Gharafi 3.7 (2)5.9 (2)6.4 (2)6.4 (2)6.4 (2)6.4 (2)6.4 (2)15/15
DD-CMA-ES-pycma Gharafi 3.3 (1)4.9 (2)5.2 (2)5.2 (2)5.2 (2)5.2 (2)5.2 (2)15/15
EXPLO2 Huntsman 1.8 (2)4.0 (4)5.3 (5)17 (19)58 (104)1000/15
oMads-2N Dahito 0.80 (0.3)1.1 (0.7)1.1 (0.7)1.1 (0.7)1.1 (0.7)1.1 (0.7)1.1 (0.7)15/15
oMads-Neg Dahito 0.90 (0.6)1.2 (0.6)1.3 (0.6)1.3 (0.6)1.3 (0.6)1.3 (0.6)1.3 (0.6)15/15

Δ fopt
f6, 3-D
1e1
34
1e0
56
1e-1
90
1e-2
117
1e-3
149
1e-5
215
1e-7
265
#succ
15/15
CMA-ES-Akimoto Gharafi 2.5 (2)3.4 (1)3.2 (0.5)3.4 (0.7)3.5 (0.5)3.2 (0.4)3.3 (0.5)15/15
CMA-ES-pycma Gharafi 2.1 (2)3.4 (1)2.9 (0.8)3.1 (0.5)3.1 (0.6)2.8 (0.4)3.0 (0.3)15/15
CMAES-APOP-KMA Nguyen 2.4 (2)3.1 (1)2.9 (1.0)2.8 (1.0)2.8 (0.8)2.8 (0.5)3.0 (0.4)15/15
CMAES-APOP-KP Nguyen 2.1 (1)2.6 (1)2.9 (0.7)3.1 (1)3.0 (1)3.0 (0.7)3.2 (0.5)15/15
CMAES-APOP-MA Nguyen 1.7 (1)3.0 (1)2.7 (0.8)2.8 (0.6)2.7 (0.5)2.6 (0.5)2.8 (0.4)15/15
DD-CMA-ES-Akimoto Gharafi 2.9 (2)3.4 (1)3.4 (0.8)3.4 (0.8)3.3 (0.7)3.1 (0.5)3.3 (0.4)15/15
DD-CMA-ES-pycma Gharafi 1.9 (1)3.0 (2)2.8 (0.9)2.8 (0.6)2.8 (0.4)2.7 (0.5)2.8 (0.4)15/15
EXPLO2 Huntsman 4.2 (6)13 (10)1000/15
oMads-2N Dahito 0.95 (0.6)1.8 (0.7)1.5 (0.8)1.5 (0.8)1.5 (0.5)1.5 (0.4)1.7 (0.4)15/15
oMads-Neg Dahito 1.2 (1)1.8 (1.0)1.5 (0.8)1.4 (0.6)1.5 (0.4)1.4 (0.3)1.6 (0.3)15/15

Δ fopt
f7, 3-D
1e1
11
1e0
65
1e-1
342
1e-2
464
1e-3
482
1e-5
482
1e-7
535
#succ
15/15
CMA-ES-Akimoto Gharafi 2.5 (2)1.3 (0.5)0.80 (0.6)0.76 (0.5)0.84 (0.5)0.84 (0.5)0.84 (0.5)15/15
CMA-ES-pycma Gharafi 2.4 (3)1.8 (2)0.66 (0.7)0.69 (0.5)0.79 (0.5)0.79 (0.5)0.74 (0.4)15/15
CMAES-APOP-KMA Nguyen 2.1 (2)1.8 (2)0.83 (0.9)0.81 (0.7)0.90 (0.7)0.90 (0.7)1.1 (0.9)15/15
CMAES-APOP-KP Nguyen 1.6 (1)1.6 (2)1.0 (0.8)1.3 (0.8)1.4 (0.8)1.4 (0.8)1.5 (0.8)15/15
CMAES-APOP-MA Nguyen 1.8 (2)2.9 (5)1.8 (2)2.0 (1)2.2 (2)2.2 (2)2.2 (2)15/15
DD-CMA-ES-Akimoto Gharafi 2.1 (2)1.5 (1)0.62 (0.5)0.76 (0.5)0.78 (0.4)0.78 (0.4)0.75 (0.4)15/15
DD-CMA-ES-pycma Gharafi 1.8 (2)2.1 (2)0.83 (0.7)0.77 (0.5)0.84 (0.5)0.84 (0.5)0.91 (0.5)15/15
EXPLO2 Huntsman 3.0 (1)2.1 (2)4.3 (4)1000/15
oMads-2N Dahito 1.4 (0.9)1.2 (1)0.89 (1)1.1 (1)1.9 (2)1.9 (2)1.8 (2)15/15
oMads-Neg Dahito 1.5 (1)1.5 (2)0.84 (1)0.76 (0.8)1.5 (1)1.5 (1)1.4 (1)15/15

Δ fopt
f8, 3-D
1e1
27
1e0
45
1e-1
152
1e-2
179
1e-3
188
1e-5
198
1e-7
208
#succ
15/15
CMA-ES-Akimoto Gharafi 2.4 (1)5.9 (6)3.5 (2)3.8 (2)3.9 (2)4.3 (2)4.5 (2)15/15
CMA-ES-pycma Gharafi 2.6 (2)7.3 (6)3.5 (2)3.7 (2)3.8 (2)4.2 (2)4.4 (2)15/15
CMAES-APOP-KMA Nguyen 1.7 (1)6.0 (4)3.5 (1)3.6 (1)3.8 (1)4.1 (1)4.3 (1)15/15
CMAES-APOP-KP Nguyen 2.3 (1)7.3 (9)4.0 (3)4.1 (3)4.3 (2)4.7 (2)4.9 (2)15/15
CMAES-APOP-MA Nguyen 1.5 (1)6.5 (5)3.7 (2)3.9 (1)4.1 (1)4.3 (1)4.4 (1)15/15
DD-CMA-ES-Akimoto Gharafi 1.8 (2)4.2 (3)2.5 (1)2.9 (0.7)3.2 (0.7)3.5 (0.7)3.9 (0.6)15/15
DD-CMA-ES-pycma Gharafi 2.0 (1)4.9 (5)2.9 (1)3.0 (1)3.1 (1)3.4 (1)3.6 (1)15/15
EXPLO2 Huntsman 4.7 (5)1000/15
oMads-2N Dahito 1.8 (1)3.3 (3)1.6 (1.0)1.8 (0.6)1.8 (0.7)2.1 (0.9)2.3 (0.8)15/15
oMads-Neg Dahito 1.7 (1)2.9 (2)1.6 (0.8)1.7 (0.6)1.8 (0.6)2.0 (0.6)2.0 (0.5)15/15

Δ fopt
f9, 3-D
1e1
21
1e0
65
1e-1
127
1e-2
149
1e-3
159
1e-5
169
1e-7
178
#succ
15/15
CMA-ES-Akimoto Gharafi 2.9 (1)3.9 (2)4.0 (1)4.3 (1)4.6 (2)5.0 (1)5.3 (1)15/15
CMA-ES-pycma Gharafi 1.9 (0.6)2.9 (2)3.2 (1)3.6 (2)3.8 (1)4.2 (1)4.5 (0.9)15/15
CMAES-APOP-KMA Nguyen 2.3 (1)4.0 (3)4.0 (2)4.2 (1)4.4 (1)4.7 (1)4.9 (1)15/15
CMAES-APOP-KP Nguyen 2.0 (2)3.5 (2)4.1 (2)4.5 (1)4.7 (1)5.1 (1)5.4 (1)15/15
CMAES-APOP-MA Nguyen 1.8 (1.0)2.9 (2)3.5 (1)3.8 (1)4.1 (0.9)4.3 (0.8)4.5 (0.9)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (1)3.9 (5)4.3 (2)4.4 (2)4.6 (2)5.0 (2)5.2 (2)15/15
DD-CMA-ES-pycma Gharafi 1.5 (0.4)2.8 (2)3.1 (1.0)3.3 (1)3.5 (0.9)3.8 (0.8)4.1 (0.9)15/15
EXPLO2 Huntsman 4.7 (3)1000/15
oMads-2N Dahito 0.80 (0.4)1.2 (0.9)1.5 (0.7)1.8 (0.7)1.8 (0.7)2.1 (0.7)2.4 (0.8)15/15
oMads-Neg Dahito 0.70 (0.3)0.78 (0.6)1.0 (0.6)1.3 (0.6)1.4 (0.5)1.7 (0.8)1.8 (0.7)15/15

Δ fopt
f10, 3-D
1e1
114
1e0
152
1e-1
168
1e-2
180
1e-3
194
1e-5
218
1e-7
242
#succ
15/15
CMA-ES-Akimoto Gharafi 2.9 (1)3.0 (0.7)3.2 (0.4)3.3 (0.4)3.4 (0.3)3.4 (0.4)3.5 (0.3)15/15
CMA-ES-pycma Gharafi 2.4 (1)2.3 (1)2.4 (1)2.6 (0.7)2.6 (0.6)2.7 (0.6)2.8 (0.5)15/15
CMAES-APOP-KMA Nguyen 3.3 (1)3.4 (1)3.4 (0.6)3.5 (0.6)3.4 (0.5)3.3 (0.5)3.3 (0.4)15/15
CMAES-APOP-KP Nguyen 5.2 (2)4.9 (0.8)4.8 (0.6)4.7 (0.7)4.6 (0.7)4.6 (0.4)4.5 (0.5)15/15
CMAES-APOP-MA Nguyen 3.5 (1)3.4 (1)3.4 (0.7)3.4 (0.6)3.3 (0.6)3.3 (0.5)3.3 (0.4)15/15
DD-CMA-ES-Akimoto Gharafi 3.2 (1)2.9 (1)3.2 (0.4)3.2 (0.3)3.3 (0.4)3.4 (0.5)3.4 (0.3)15/15
DD-CMA-ES-pycma Gharafi 2.2 (2)2.3 (1)2.4 (1.0)2.6 (0.8)2.6 (0.8)2.8 (0.7)2.9 (0.6)15/15
EXPLO2 Huntsman 1000/15
oMads-2N Dahito 1.8 (2)1.8 (1)2.1 (1)2.2 (0.9)2.3 (1)2.8 (1)3.9 (2)15/15
oMads-Neg Dahito 1.4 (0.8)1.6 (0.8)1.7 (0.8)1.9 (0.6)2.2 (1.0)2.5 (0.7)3.1 (2)15/15

Δ fopt
f11, 3-D
1e1
67
1e0
105
1e-1
227
1e-2
263
1e-3
277
1e-5
302
1e-7
327
#succ
15/15
CMA-ES-Akimoto Gharafi 4.0 (3)4.1 (1.0)2.2 (0.3)2.1 (0.2)2.2 (0.3)2.4 (0.2)2.5 (0.3)15/15
CMA-ES-pycma Gharafi 3.2 (2)3.1 (0.8)1.7 (0.3)1.6 (0.3)1.7 (0.3)1.9 (0.2)2.0 (0.2)15/15
CMAES-APOP-KMA Nguyen 4.5 (3)4.9 (2)2.7 (0.7)2.5 (0.6)2.6 (0.5)2.6 (0.4)2.6 (0.4)15/15
CMAES-APOP-KP Nguyen 6.1 (5)6.6 (2)3.5 (0.9)3.3 (0.7)3.3 (0.5)3.4 (0.3)3.4 (0.3)15/15
CMAES-APOP-MA Nguyen 5.8 (3)5.1 (2)2.8 (0.3)2.6 (0.3)2.6 (0.3)2.6 (0.3)2.7 (0.3)15/15
DD-CMA-ES-Akimoto Gharafi 4.5 (3)3.9 (1.0)2.3 (0.3)2.1 (0.3)2.2 (0.3)2.4 (0.2)2.5 (0.3)15/15
DD-CMA-ES-pycma Gharafi 3.5 (1)3.2 (1)1.7 (0.5)1.7 (0.3)1.8 (0.3)1.9 (0.3)2.0 (0.3)15/15
EXPLO2 Huntsman 10 (11)1000/15
oMads-2N Dahito 3.0 (2)3.3 (2)2.1 (1)2.3 (0.9)2.4 (1)2.9 (1)5.6 (5)15/15
oMads-Neg Dahito 3.1 (3)3.7 (2)2.1 (1)1.9 (0.9)2.1 (0.8)2.6 (1.0)3.9 (2)15/15

Δ fopt
f12, 3-D
1e1
65
1e0
168
1e-1
338
1e-2
401
1e-3
445
1e-5
696
1e-7
790
#succ
15/15
CMA-ES-Akimoto Gharafi 5.7 (1)4.4 (3)3.9 (4)4.0 (4)4.2 (4)3.4 (3)3.7 (3)15/15
CMA-ES-pycma Gharafi 5.9 (5)5.5 (7)4.1 (5)4.3 (5)4.2 (4)3.3 (3)3.7 (3)15/15
CMAES-APOP-KMA Nguyen 4.8 (3)3.8 (5)3.1 (3)3.1 (3)3.2 (3)2.6 (2)3.4 (2)15/15
CMAES-APOP-KP Nguyen 8.3 (10)5.8 (7)4.5 (4)4.5 (4)4.5 (4)3.5 (3)4.5 (6)15/15
CMAES-APOP-MA Nguyen 6.3 (6)5.3 (6)3.8 (4)3.7 (4)3.6 (3)3.2 (2)3.1 (2)15/15
DD-CMA-ES-Akimoto Gharafi 7.1 (11)5.9 (7)4.5 (4)4.5 (3)4.6 (3)3.7 (2)3.7 (3)15/15
DD-CMA-ES-pycma Gharafi 4.4 (0.7)3.2 (2)2.2 (3)2.3 (3)2.4 (3)1.9 (2)2.0 (2)15/15
EXPLO2 Huntsman 1000/15
oMads-2N Dahito 2.9 (3)2.1 (2)1.5 (2)1.6 (1)1.8 (1)1.6 (1)1.9 (1)15/15
oMads-Neg Dahito 1.9 (0.7)1.7 (2)1.4 (2)1.4 (1)1.6 (1)1.5 (0.9)1.7 (2)15/15

Δ fopt
f13, 3-D
1e1
49
1e0
85
1e-1
108
1e-2
136
1e-3
215
1e-5
281
1e-7
365
#succ
15/15
CMA-ES-Akimoto Gharafi 3.5 (1)3.5 (1)4.0 (1.0)4.1 (0.7)3.1 (0.5)3.4 (0.4)3.2 (0.3)15/15
CMA-ES-pycma Gharafi 3.7 (2)3.2 (0.9)3.5 (0.8)3.5 (0.7)2.6 (0.4)2.7 (0.4)2.6 (0.3)15/15
CMAES-APOP-KMA Nguyen 2.9 (2)3.6 (2)4.5 (2)4.8 (2)3.4 (1.0)3.7 (1)3.6 (0.8)15/15
CMAES-APOP-KP Nguyen 3.7 (2)5.0 (3)6.0 (2)6.1 (2)4.4 (0.9)4.6 (0.8)4.5 (0.6)15/15
CMAES-APOP-MA Nguyen 3.5 (2)4.3 (2)4.3 (1)4.3 (2)3.4 (0.8)3.6 (0.6)3.3 (0.5)15/15
DD-CMA-ES-Akimoto Gharafi 3.9 (2)3.9 (2)4.3 (1)4.1 (0.4)3.2 (0.5)3.4 (0.4)3.2 (0.5)15/15
DD-CMA-ES-pycma Gharafi 2.4 (1)3.0 (1)3.6 (1)3.6 (0.9)2.7 (0.6)2.8 (0.5)2.7 (0.4)15/15
EXPLO2 Huntsman 10 (12)1000/15
oMads-2N Dahito 1.8 (0.8)1.8 (0.9)3.2 (3)3.9 (5)5.2 (5)56 (55)3e50/15
oMads-Neg Dahito 1.3 (0.5)1.7 (1)3.4 (3)4.0 (4)4.3 (5)52 (72)3e50/15

Δ fopt
f14, 3-D
1e1
2.0
1e0
17
1e-1
28
1e-2
43
1e-3
71
1e-5
110
1e-7
194
#succ
15/15
CMA-ES-Akimoto Gharafi 1.3 (2)2.0 (2)3.1 (2)4.2 (2)3.9 (0.9)4.6 (0.8)3.9 (0.4)15/15
CMA-ES-pycma Gharafi 1.4 (2)2.2 (2)3.0 (0.8)3.7 (1.0)3.5 (0.7)3.9 (0.6)3.2 (0.3)15/15
CMAES-APOP-KMA Nguyen 1.5 (2)1.7 (1.0)2.5 (0.9)3.2 (0.9)3.3 (1)4.5 (1)4.1 (0.5)15/15
CMAES-APOP-KP Nguyen 1.6 (2)1.6 (1)2.9 (1)3.7 (0.9)3.9 (1)6.2 (2)5.7 (0.5)15/15
CMAES-APOP-MA Nguyen 2.3 (3)1.9 (2)2.9 (2)3.6 (1)3.8 (1)5.1 (0.8)4.2 (0.4)15/15
DD-CMA-ES-Akimoto Gharafi 1.7 (2)2.0 (1)3.2 (0.7)3.8 (1)3.5 (1)4.4 (0.8)3.8 (0.4)15/15
DD-CMA-ES-pycma Gharafi 2.1 (3)1.8 (1)3.1 (2)3.4 (1.0)3.1 (0.9)3.6 (0.6)3.1 (0.6)15/15
EXPLO2 Huntsman 4.1 (5)4.5 (3)17 (17)1000/15
oMads-2N Dahito 1 (0.8)1.1 (0.8)2.0 (1)2.1 (0.7)2.1 (0.8)4.0 (2)11 (10)14/15
oMads-Neg Dahito 1.3 (2)1.1 (0.6)1.9 (1)1.9 (0.4)1.8 (0.5)3.1 (1)11 (8)14/15

Δ fopt
f15, 3-D
1e1
121
1e0
1372
1e-1
6285
1e-2
8282
1e-3
8429
1e-5
8787
1e-7
9041
#succ
15/15
CMA-ES-Akimoto Gharafi 1.1 (0.7)1.9 (2)0.74 (0.4)0.59 (0.4)↑10.59 (0.4)↑10.59 (0.4)↑10.60 (0.4)↑115/15
CMA-ES-pycma Gharafi 1.0 (0.7)1.6 (2)0.98 (0.7)0.79 (0.6)0.79 (0.6)0.80 (0.6)0.80 (0.6)15/15
CMAES-APOP-KMA Nguyen 0.79 (0.5)1.7 (2)1.4 (1)1.2 (1)1.2 (1)1.2 (1.0)1.2 (1.0)15/15
CMAES-APOP-KP Nguyen 1.2 (0.8)2.3 (2)0.96 (1.0)0.82 (0.7)0.86 (0.7)0.89 (0.7)0.92 (0.7)15/15
CMAES-APOP-MA Nguyen 0.65 (0.6)2.5 (2)1.0 (1.0)0.97 (0.8)1.0 (0.8)1.1 (0.8)1.1 (0.8)15/15
DD-CMA-ES-Akimoto Gharafi 1.1 (0.6)1.5 (1)1.1 (0.6)0.89 (0.5)0.90 (0.5)0.90 (0.5)0.90 (0.5)15/15
DD-CMA-ES-pycma Gharafi 1.1 (0.5)1.8 (2)1.1 (1)0.90 (1)0.91 (1)0.91 (1)0.92 (1)15/15
EXPLO2 Huntsman 2.7 (3)1000/15
oMads-2N Dahito 1.3 (2)2.5 (3)2.0 (3)1.5 (2)1.5 (2)1.4 (2)1.4 (2)15/15
oMads-Neg Dahito 1.2 (1)1.8 (2)2.1 (3)1.6 (2)1.6 (2)1.5 (2)1.5 (2)15/15

Δ fopt
f16, 3-D
1e1
41
1e0
319
1e-1
582
1e-2
789
1e-3
1864
1e-5
3204
1e-7
3361
#succ
15/15
CMA-ES-Akimoto Gharafi 1.4 (2)3.1 (3)2.7 (2)2.1 (2)0.99 (0.7)0.63 (0.4)↑10.64 (0.4)15/15
CMA-ES-pycma Gharafi 1.4 (2)3.5 (3)2.2 (1)2.0 (1)0.88 (0.5)0.59 (0.3)↑10.60 (0.3)↑115/15
CMAES-APOP-KMA Nguyen 1.8 (3)2.3 (3)3.5 (4)3.8 (5)3.2 (3)2.1 (2)2.2 (2)15/15
CMAES-APOP-KP Nguyen 1.9 (3)3.3 (4)8.2 (18)8.8 (14)4.3 (6)2.9 (4)2.9 (4)15/15
CMAES-APOP-MA Nguyen 1.1 (1)8.2 (14)17 (21)19 (23)8.8 (10)5.6 (6)5.7 (6)15/15
DD-CMA-ES-Akimoto Gharafi 0.93 (0.7)1.6 (2)2.1 (1)1.8 (1)0.94 (1.0)0.67 (0.6)0.69 (0.6)15/15
DD-CMA-ES-pycma Gharafi 0.84 (2)2.7 (5)3.3 (4)3.0 (3)2.3 (2)1.6 (1)1.6 (1)15/15
EXPLO2 Huntsman 1.3 (2)1000/15
oMads-2N Dahito 0.70 (0.6)1.2 (2)1.1 (0.9)1.4 (2)0.86 (1)0.76 (0.9)0.95 (0.9)15/15
oMads-Neg Dahito 0.97 (0.8)0.88 (1.0)0.88 (1)1.0 (1)0.49 (0.4)↑10.49 (1)0.92 (1.0)15/15

Δ fopt
f17, 3-D
1e1
4.0
1e0
78
1e-1
282
1e-2
491
1e-3
1134
1e-5
2347
1e-7
3469
#succ
15/15
CMA-ES-Akimoto Gharafi 1.2 (2)0.96 (0.5)0.74 (0.2)↑11.1 (0.2)0.72 (0.6)0.99 (0.8)0.83 (0.6)15/15
CMA-ES-pycma Gharafi 1.5 (2)1.1 (0.6)0.80 (0.2)↑11.2 (1)1.0 (0.7)1.1 (0.5)1.0 (0.4)15/15
CMAES-APOP-KMA Nguyen 0.95 (1)0.83 (0.5)0.62 (0.2)↑01.2 (2)1.2 (0.9)1.1 (0.6)0.98 (0.5)15/15
CMAES-APOP-KP Nguyen 1.1 (2)2.1 (0.8)2.7 (4)2.1 (2)1.3 (1)1.2 (0.7)1.1 (0.6)15/15
CMAES-APOP-MA Nguyen 0.87 (0.9)1.6 (1)1.7 (3)1.8 (2)1.1 (1)1.4 (1)1.3 (0.8)15/15
DD-CMA-ES-Akimoto Gharafi 1.3 (2)0.93 (0.3)0.69 (0.3)↑11.2 (2)0.67 (0.8)0.91 (0.8)0.80 (0.6)15/15
DD-CMA-ES-pycma Gharafi 0.90 (1)1.1 (0.7)1.2 (2)1.7 (1)0.97 (0.6)1.2 (0.8)1.2 (0.8)15/15
EXPLO2 Huntsman 2.1 (2)4.3 (3)1000/15
oMads-2N Dahito 7.4 (3)1.9 (3)1.4 (1)1.6 (2)1.0 (0.9)2.1 (1)367 (454)0/15
oMads-Neg Dahito 1.9 (2)1.7 (2)1.4 (1)1.9 (2)1.3 (0.8)10 (8)574 (647)0/15

Δ fopt
f18, 3-D
1e1
40
1e0
145
1e-1
1289
1e-2
3084
1e-3
3523
1e-5
4738
1e-7
5527
#succ
15/15
CMA-ES-Akimoto Gharafi 0.78 (0.5)1.3 (0.4)0.66 (0.8)0.49 (0.4)↑00.56 (0.3)↑00.72 (0.3)0.69 (0.3)15/15
CMA-ES-pycma Gharafi 0.86 (1)1.8 (0.7)0.78 (0.8)0.53 (0.5)0.57 (0.4)↑10.67 (0.9)0.70 (0.8)15/15
CMAES-APOP-KMA Nguyen 1.2 (2)6.4 (6)1.9 (3)1.1 (1)1.1 (1)1.0 (0.8)1.0 (0.7)15/15
CMAES-APOP-KP Nguyen 0.99 (0.9)4.3 (7)1.3 (1)1.0 (1.0)1.1 (0.8)1.1 (0.6)1.0 (0.5)15/15
CMAES-APOP-MA Nguyen 0.73 (0.6)2.3 (3)2.3 (3)1.4 (1)1.5 (0.9)1.4 (0.6)1.4 (0.6)15/15
DD-CMA-ES-Akimoto Gharafi 1.1 (1)4.3 (7)0.83 (0.9)0.52 (0.4)↑10.54 (0.3)↑10.72 (0.4)0.83 (0.4)15/15
DD-CMA-ES-pycma Gharafi 2.2 (1)5.3 (5)1.6 (2)1.5 (1)1.5 (1)1.6 (2)1.6 (1)15/15
EXPLO2 Huntsman 2.0 (1)1000/15
oMads-2N Dahito 2.1 (5)2.3 (3)0.92 (0.8)1.8 (2)5.4 (8)424 (453)3e50/15
oMads-Neg Dahito 4.8 (6)3.5 (4)2.2 (2)2.7 (4)14 (17)3e50/15

Δ fopt
f19, 3-D
1e1
1
1e0
1
1e-1
109
1e-2
6764
1e-3
7367
1e-5
7399
1e-7
7441
#succ
15/15
CMA-ES-Akimoto Gharafi 1 (0)1 (0)73 (74)3.2 (3)3.6 (3)3.6 (3)3.6 (3)15/15
CMA-ES-pycma Gharafi 1 (0)1 (0)41 (55)2.6 (3)3.3 (3)3.4 (3)3.4 (3)15/15
CMAES-APOP-KMA Nguyen 1 (0)1 (0)36 (39)2.6 (2)3.3 (3)3.6 (4)3.8 (4)15/15
CMAES-APOP-KP Nguyen 1 (0)1 (0)82 (51)4.3 (8)5.1 (8)5.5 (8)5.8 (8)15/15
CMAES-APOP-MA Nguyen 1 (0)1 (0)33 (29)2.0 (1.0)2.2 (1)2.9 (1)3.1 (1)15/15
DD-CMA-ES-Akimoto Gharafi 1 (0)1 (0)49 (97)3.7 (4)4.4 (5)4.4 (5)4.5 (5)15/15
DD-CMA-ES-pycma Gharafi 1 (0)1 (0)58 (64)3.2 (3)3.2 (3)3.2 (3)3.3 (3)15/15
EXPLO2 Huntsman 21 (15)186 (189)1000/15
oMads-2N Dahito 1 (0)1 (0)0.51 (0.3)↑05.6 (10)8.6 (13)8.5 (13)8.5 (13)15/15
oMads-Neg Dahito 1 (0)1 (0)0.44 (0.2)↑05.6 (7)17 (26)17 (21)17 (23)13/15

Δ fopt
f20, 3-D
1e1
8.0
1e0
385
1e-1
2291
1e-2
2398
1e-3
2481
1e-5
2573
1e-7
2776
#succ
15/15
CMA-ES-Akimoto Gharafi 1.4 (1)2.5 (4)4.1 (4)4.1 (4)4.1 (4)4.1 (4)4.0 (4)15/15
CMA-ES-pycma Gharafi 1.0 (0.9)4.3 (7)4.4 (4)4.4 (4)4.4 (4)4.5 (4)4.3 (4)15/15
CMAES-APOP-KMA Nguyen 1.9 (2)6.9 (10)19 (18)20 (17)20 (17)20 (16)19 (15)15/15
CMAES-APOP-KP Nguyen 0.96 (0.9)4.3 (5)6.5 (9)6.6 (9)6.5 (8)6.5 (8)6.2 (7)15/15
CMAES-APOP-MA Nguyen 1.1 (1)5.8 (8)19 (14)20 (14)20 (13)20 (13)19 (12)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (0.9)6.9 (7)4.8 (4)4.8 (4)4.8 (4)4.8 (4)4.6 (4)15/15
DD-CMA-ES-pycma Gharafi 1.5 (0.8)4.3 (6)7.1 (6)7.4 (7)7.4 (7)7.4 (7)7.1 (6)15/15
EXPLO2 Huntsman 3.2 (2)3.8 (5)1000/15
oMads-2N Dahito 0.63 (0.6)0.96 (0.9)2.6 (2)2.5 (2)2.4 (2)2.4 (2)2.2 (2)15/15
oMads-Neg Dahito 0.99 (1)1.1 (0.9)2.0 (2)1.9 (2)1.8 (2)1.8 (2)1.7 (2)15/15

Δ fopt
f21, 3-D
1e1
6.0
1e0
184
1e-1
425
1e-2
439
1e-3
458
1e-5
469
1e-7
482
#succ
14/15
CMA-ES-Akimoto Gharafi 1.6 (1)6.7 (10)36 (66)50 (122)49 (117)49 (114)49 (111)15/15
CMA-ES-pycma Gharafi 1.2 (2)8.7 (19)7.2 (13)7.5 (13)7.5 (14)7.7 (14)7.7 (14)15/15
CMAES-APOP-KMA Nguyen 1.7 (2)5.4 (14)11 (23)13 (22)13 (21)13 (22)13 (22)15/15
CMAES-APOP-KP Nguyen 2.2 (2)4.6 (7)11 (14)11 (14)11 (13)11 (13)11 (12)15/15
CMAES-APOP-MA Nguyen 1.2 (1)5.3 (12)18 (35)19 (34)18 (33)19 (32)21 (32)15/15
DD-CMA-ES-Akimoto Gharafi 0.99 (0.9)4.3 (10)4.1 (5)4.1 (5)4.1 (5)4.2 (4)4.4 (5)15/15
DD-CMA-ES-pycma Gharafi 1.5 (2)7.7 (10)16 (27)16 (27)16 (27)17 (27)17 (27)15/15
EXPLO2 Huntsman 2.5 (3)1.4 (1)3.5 (3)1000/15
oMads-2N Dahito 1.8 (3)2.6 (3)2.1 (2)2.1 (2)2.0 (2)2.0 (2)2.0 (2)15/15
oMads-Neg Dahito 1.5 (2)3.1 (4)4.6 (5)4.5 (5)4.3 (5)4.3 (5)4.2 (4)15/15

Δ fopt
f22, 3-D
1e1
18
1e0
170
1e-1
354
1e-2
362
1e-3
384
1e-5
401
1e-7
414
#succ
15/15
CMA-ES-Akimoto Gharafi 1.4 (2)4.0 (7)33 (79)34 (78)63 (96)242 (267)236 (264)15/15
CMA-ES-pycma Gharafi 1.1 (0.9)4.5 (6)13 (15)17 (17)71 (50)103 (53)101 (53)15/15
CMAES-APOP-KMA Nguyen 0.68 (0.9)12 (24)34 (37)34 (37)33 (34)32 (33)31 (32)15/15
CMAES-APOP-KP Nguyen 0.92 (0.7)5.3 (8)10 (15)12 (14)19 (23)19 (22)19 (22)15/15
CMAES-APOP-MA Nguyen 1.5 (1)13 (3)27 (54)36 (53)41 (65)63 (86)62 (84)15/15
DD-CMA-ES-Akimoto Gharafi 1.7 (2)4.8 (6)17 (26)34 (61)34 (59)120 (56)117 (58)15/15
DD-CMA-ES-pycma Gharafi 0.96 (0.6)3.8 (5)21 (37)31 (43)59 (49)87 (251)85 (243)15/15
EXPLO2 Huntsman 2.7 (4)1.2 (0.8)2.0 (2)1000/15
oMads-2N Dahito 0.93 (0.7)1.4 (3)2.2 (2)2.2 (2)2.1 (2)2.0 (2)2.1 (2)15/15
oMads-Neg Dahito 2.5 (4)2.1 (2)2.4 (3)2.4 (3)2.3 (2)2.3 (2)2.3 (2)15/15

Δ fopt
f23, 3-D
1e1
3.0
1e0
407
1e-1
906
1e-2
1215
1e-3
2214
1e-5
2293
1e-7
2393
#succ
15/15
CMA-ES-Akimoto Gharafi 2.7 (2)6.7 (5)8.2 (8)6.8 (6)3.8 (4)4.0 (3)4.0 (3)15/15
CMA-ES-pycma Gharafi 4.2 (5)6.7 (6)4.9 (4)3.8 (3)2.2 (2)2.3 (2)2.3 (2)15/15
CMAES-APOP-KMA Nguyen 2.7 (3)11 (10)67 (12)55 (11)31 (7)31 (7)30 (7)15/15
CMAES-APOP-KP Nguyen 3.8 (4)8.7 (14)51 (46)43 (35)24 (19)24 (19)24 (18)15/15
CMAES-APOP-MA Nguyen 3.9 (5)10 (13)48 (36)41 (30)24 (20)24 (21)23 (20)15/15
DD-CMA-ES-Akimoto Gharafi 1.9 (2)6.2 (8)15 (13)11 (10)6.3 (6)6.3 (6)6.2 (5)15/15
DD-CMA-ES-pycma Gharafi 1.8 (1)10 (9)10 (10)8.3 (8)4.6 (4)4.6 (4)4.6 (4)15/15
EXPLO2 Huntsman 3.0 (2)1000/15
oMads-2N Dahito 4.4 (6)0.67 (0.8)1.2 (1)1.0 (1)0.89 (0.9)967 (751)3e50/15
oMads-Neg Dahito 3.8 (3)0.71 (0.8)1.7 (2)1.4 (1)0.94 (0.8)944 (1076)3e50/15

Δ fopt
f24, 3-D
1e1
97
1e0
10391
1e-1
1.0e5
1e-2
3.6e5
1e-3
3.6e5
1e-5
3.6e5
1e-7
3.6e5
#succ
2/15
CMA-ES-Akimoto Gharafi 1.4 (1)26 (26)13 (16)7.0 (5)7.0 (5)7.0 (6)7.1 (6)7/15
CMA-ES-pycma Gharafi 2.0 (1)168 (217)96 (73)2e60/15
CMAES-APOP-KMA Nguyen 1.6 (0.9)18 (22)4.1 (5)3.2 (3)3.2 (5)3.3 (4)3.3 (3)6/15
CMAES-APOP-KP Nguyen 1.7 (1)17 (16)3.2 (3)1.9 (3)2.2 (3)2.2 (2)2.2 (3)8/15
CMAES-APOP-MA Nguyen 1.4 (1)13 (14)2.1 (3)1.0 (1)1.1 (1)1.1 (0.9)1.1 (1)11/15
DD-CMA-ES-Akimoto Gharafi 1.3 (1)11 (24)15 (12)13 (13)13 (14)13 (20)13 (17)4/15
DD-CMA-ES-pycma Gharafi 2.1 (2)77 (145)30 (42)28 (29)28 (32)28 (33)28 (36)2/15
EXPLO2 Huntsman 3.8 (3)1000/15
oMads-2N Dahito 1.4 (3)1.4 (2)4.8 (7)5.6 (6)5.6 (5)5.6 (7)5.6 (6)2/15
oMads-Neg Dahito 2.4 (6)1.0 (1)5.6 (7)12 (16)12 (16)12 (15)12 (13)1/15

Previous dimension | Dimension = 5 | Next dimension

Δ fopt
f1, 5-D
1e1
11
1e0
12
1e-1
12
1e-2
12
1e-3
12
1e-5
12
1e-7
12
#succ
15/15
CMA-ES-Akimoto Gharafi 2.1 (2)7.5 (4)15 (3)22 (4)29 (3)41 (5)55 (6)15/15
CMA-ES-pycma Gharafi 1.6 (1)7.1 (2)13 (3)20 (3)27 (3)39 (4)50 (6)15/15
CMAES-APOP-KMA Nguyen 1.9 (1)5.9 (2)10 (2)15 (2)20 (2)29 (3)38 (4)15/15
CMAES-APOP-KP Nguyen 2.1 (2)7.7 (3)14 (5)22 (5)29 (6)41 (5)53 (8)15/15
CMAES-APOP-MA Nguyen 1.8 (1)6.7 (2)11 (2)15 (2)20 (3)30 (4)40 (3)15/15
DD-CMA-ES-Akimoto Gharafi 1.5 (1)7.5 (2)12 (2)18 (3)24 (4)35 (6)47 (5)15/15
DD-CMA-ES-pycma Gharafi 1.7 (1)5.8 (2)11 (2)16 (3)21 (3)31 (6)42 (7)15/15
EXPLO2 Huntsman 1.8 (1)5.1 (2)15 (13)1340/15
oMads-2N Dahito 1.2 (0.9)3.6 (3)9.2 (3)10 (3)10 (3)18 (4)18 (3)15/15
oMads-Neg Dahito 1.4 (0.9)3.3 (2)7.6 (3)8.7 (2)10 (2)16 (3)17 (3)15/15

Δ fopt
f2, 5-D
1e1
83
1e0
87
1e-1
88
1e-2
89
1e-3
90
1e-5
92
1e-7
94
#succ
15/15
CMA-ES-Akimoto Gharafi 9.3 (2)11 (3)12 (2)13 (2)14 (2)15 (2)17 (2)15/15
CMA-ES-pycma Gharafi 7.2 (2)8.5 (2)10 (2)11 (2)12 (2)13 (2)15 (2)15/15
CMAES-APOP-KMA Nguyen 10 (3)12 (3)14 (2)14 (2)15 (2)16 (2)16 (1)15/15
CMAES-APOP-KP Nguyen 11 (4)15 (4)17 (4)18 (2)19 (2)20 (2)21 (2)15/15
CMAES-APOP-MA Nguyen 9.4 (3)12 (3)13 (2)14 (2)15 (2)16 (1)17 (1)15/15
DD-CMA-ES-Akimoto Gharafi 5.2 (2)6.0 (1)6.9 (1)7.7 (1)8.4 (1)10 (1)11 (1)15/15
DD-CMA-ES-pycma Gharafi 4.0 (1)5.1 (1)6.0 (1)6.5 (0.9)7.2 (1)8.5 (1)10 (2)15/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 3.5 (0.9)5.1 (1)7.2 (2)8.7 (2)11 (2)13 (3)17 (3)15/15
oMads-Neg Dahito 3.6 (1)4.8 (3)6.3 (3)7.2 (3)7.9 (3)11 (4)13 (4)15/15

Δ fopt
f3, 5-D
1e1
716
1e0
1622
1e-1
1637
1e-2
1642
1e-3
1646
1e-5
1650
1e-7
1654
#succ
15/15
CMA-ES-Akimoto Gharafi 1.4 (1)6.2 (7)167 (184)177 (253)177 (253)177 (252)177 (252)15/15
CMA-ES-pycma Gharafi 0.79 (0.4)28 (74)853 (1290)1087 (1137)1085 (1382)1083 (1517)1081 (1375)10/15
CMAES-APOP-KMA Nguyen 1.1 (2)15 (12)77 (84)78 (83)78 (83)78 (83)79 (82)15/15
CMAES-APOP-KP Nguyen 1.1 (2)26 (25)120 (116)120 (116)120 (115)121 (115)121 (115)15/15
CMAES-APOP-MA Nguyen 0.60 (0.3)↑129 (28)165 (188)169 (187)169 (187)169 (186)170 (186)15/15
DD-CMA-ES-Akimoto Gharafi 0.83 (1)4.8 (4)103 (134)104 (135)104 (134)104 (134)104 (134)15/15
DD-CMA-ES-pycma Gharafi 0.42 (0.1)↑018 (19)308 (431)309 (429)308 (428)308 (427)308 (426)14/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 1.1 (2)16 (21)170 (223)170 (195)170 (183)169 (163)169 (152)13/15
oMads-Neg Dahito 1.1 (1)15 (13)128 (142)128 (142)127 (142)127 (141)127 (141)14/15

Δ fopt
f4, 5-D
1e1
809
1e0
1633
1e-1
1688
1e-2
1758
1e-3
1817
1e-5
1886
1e-7
1903
#succ
15/15
CMA-ES-Akimoto Gharafi 1.5 (1)2.3e4 (3e4)3e60/15
CMA-ES-pycma Gharafi 1.9 (2)3e60/15
CMAES-APOP-KMA Nguyen 1.6 (2)1e60/15
CMAES-APOP-KP Nguyen 2.3 (2)1e60/15
CMAES-APOP-MA Nguyen 3.5 (3)1e60/15
DD-CMA-ES-Akimoto Gharafi 2.0 (2)7022 (7644)3e60/15
DD-CMA-ES-pycma Gharafi 1.6 (1)3e60/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 1.6 (2)150 (181)796 (1011)764 (826)740 (801)713 (750)706 (568)5/15
oMads-Neg Dahito 1.6 (2)233 (285)5e50/15

Δ fopt
f5, 5-D
1e1
10
1e0
10
1e-1
10
1e-2
10
1e-3
10
1e-5
10
1e-7
10
#succ
15/15
CMA-ES-Akimoto Gharafi 4.2 (3)6.3 (3)6.3 (3)6.4 (3)6.4 (3)6.4 (3)6.4 (3)15/15
CMA-ES-pycma Gharafi 4.8 (2)6.4 (2)6.5 (2)6.5 (2)6.5 (2)6.5 (2)6.5 (2)15/15
CMAES-APOP-KMA Nguyen 3.3 (1)5.0 (2)5.0 (2)5.1 (2)5.1 (2)5.1 (2)5.1 (2)15/15
CMAES-APOP-KP Nguyen 4.5 (1)6.7 (4)7.3 (4)7.3 (4)7.3 (4)7.3 (4)7.3 (4)15/15
CMAES-APOP-MA Nguyen 3.2 (0.8)4.1 (1)4.2 (1)4.3 (1)4.3 (1)4.3 (1)4.3 (1)15/15
DD-CMA-ES-Akimoto Gharafi 4.9 (2)7.1 (3)7.5 (3)7.5 (3)7.5 (3)7.5 (3)7.5 (3)15/15
DD-CMA-ES-pycma Gharafi 4.0 (2)5.9 (2)6.1 (2)6.1 (2)6.1 (2)6.1 (2)6.1 (2)15/15
EXPLO2 Huntsman 3.2 (2)9.3 (6)88 (107)93 (101)188 (261)1340/15
oMads-2N Dahito 1.2 (0.7)3.3 (2)3.3 (2)3.3 (2)3.3 (2)3.3 (2)3.3 (2)15/15
oMads-Neg Dahito 2.3 (1)3.6 (2)3.6 (2)3.6 (2)3.6 (2)3.6 (2)3.6 (2)15/15

Δ fopt
f6, 5-D
1e1
114
1e0
214
1e-1
281
1e-2
404
1e-3
580
1e-5
1038
1e-7
1332
#succ
15/15
CMA-ES-Akimoto Gharafi 2.0 (0.8)1.8 (0.4)2.0 (0.5)1.9 (0.3)1.6 (0.3)1.2 (0.1)1.2 (0.1)15/15
CMA-ES-pycma Gharafi 1.6 (0.7)1.7 (0.4)1.9 (0.4)1.7 (0.3)1.5 (0.2)1.1 (0.1)1.1 (0.1)15/15
CMAES-APOP-KMA Nguyen 1.9 (1.0)1.8 (0.5)2.0 (0.4)1.8 (0.3)1.4 (0.3)1.1 (0.1)1.1 (0.1)15/15
CMAES-APOP-KP Nguyen 1.8 (0.6)1.7 (0.4)1.9 (0.4)1.7 (0.3)1.5 (0.3)1.2 (0.2)1.2 (0.2)15/15
CMAES-APOP-MA Nguyen 1.6 (0.6)1.7 (0.4)1.8 (0.3)1.7 (0.3)1.5 (0.2)1.1 (0.2)1.1 (0.2)15/15
DD-CMA-ES-Akimoto Gharafi 1.8 (1.0)1.8 (0.5)1.9 (0.4)1.7 (0.3)1.5 (0.3)1.2 (0.1)1.2 (0.1)15/15
DD-CMA-ES-pycma Gharafi 1.6 (0.7)1.5 (0.5)1.7 (0.4)1.5 (0.2)1.3 (0.3)1.0 (0.2)1.0 (0.1)15/15
EXPLO2 Huntsman 8.2 (9)1340/15
oMads-2N Dahito 1.3 (0.7)1.4 (0.5)1.6 (0.5)1.5 (0.6)1.5 (0.6)1.2 (0.4)1.3 (0.4)15/15
oMads-Neg Dahito 1.0 (0.6)1.6 (0.6)2.0 (0.7)1.9 (0.7)1.7 (0.4)1.4 (0.3)1.4 (0.3)15/15

Δ fopt
f7, 5-D
1e1
24
1e0
324
1e-1
1171
1e-2
1451
1e-3
1572
1e-5
1572
1e-7
1597
#succ
15/15
CMA-ES-Akimoto Gharafi 4.1 (2)1.0 (0.9)0.50 (0.3)↑00.48 (0.2)↑00.46 (0.2)↑00.46 (0.2)↑00.52 (0.2)↑015/15
CMA-ES-pycma Gharafi 2.8 (3)1.1 (1.0)0.58 (0.3)↑00.59 (0.3)↑00.61 (0.3)↑00.61 (0.3)↑00.63 (0.3)↑015/15
CMAES-APOP-KMA Nguyen 3.1 (3)1.2 (1)1.00 (0.6)1.0 (0.6)0.99 (0.6)0.99 (0.6)1.0 (0.5)15/15
CMAES-APOP-KP Nguyen 2.9 (2)1.1 (1)1.1 (0.5)1.1 (0.8)1.1 (0.8)1.1 (0.8)1.2 (0.8)15/15
CMAES-APOP-MA Nguyen 2.9 (1)1.9 (2)1.6 (0.3)1.9 (0.4)1.8 (0.3)1.8 (0.3)1.9 (0.4)15/15
DD-CMA-ES-Akimoto Gharafi 3.4 (2)0.98 (0.7)0.55 (0.4)↑10.56 (0.3)↑10.53 (0.3)↑10.53 (0.3)↑10.60 (0.5)15/15
DD-CMA-ES-pycma Gharafi 3.2 (2)1.1 (0.7)0.77 (0.5)0.73 (0.5)0.69 (0.4)0.69 (0.4)0.72 (0.4)15/15
EXPLO2 Huntsman 4.1 (4)1340/15
oMads-2N Dahito 1.8 (0.9)1.3 (1)1.5 (2)3.0 (5)3.7 (4)3.7 (4)3.7 (4)15/15
oMads-Neg Dahito 2.6 (1)1.3 (2)2.0 (3)4.5 (7)18 (18)18 (18)18 (18)15/15

Δ fopt
f8, 5-D
1e1
73
1e0
273
1e-1
336
1e-2
372
1e-3
391
1e-5
410
1e-7
422
#succ
15/15
CMA-ES-Akimoto Gharafi 2.9 (1)4.9 (5)5.1 (4)5.2 (4)5.2 (4)5.4 (4)5.7 (4)15/15
CMA-ES-pycma Gharafi 2.6 (1.0)4.2 (4)4.6 (4)4.7 (4)4.7 (3)5.0 (3)5.3 (3)15/15
CMAES-APOP-KMA Nguyen 2.0 (0.7)3.4 (2)4.1 (1)4.3 (1)4.4 (1)4.7 (1)4.9 (1)15/15
CMAES-APOP-KP Nguyen 2.6 (1)3.9 (2)4.7 (2)5.0 (1)5.2 (1)5.5 (1)5.8 (1)15/15
CMAES-APOP-MA Nguyen 2.2 (0.8)5.4 (8)6.1 (9)6.1 (9)6.2 (9)6.4 (9)6.6 (9)15/15
DD-CMA-ES-Akimoto Gharafi 2.7 (0.8)4.4 (5)4.8 (5)4.8 (4)4.9 (4)5.1 (4)5.4 (4)15/15
DD-CMA-ES-pycma Gharafi 1.9 (0.6)3.4 (4)3.8 (4)3.9 (3)3.9 (3)4.2 (3)4.4 (3)15/15
EXPLO2 Huntsman 26 (28)1340/15
oMads-2N Dahito 2.5 (1)2.3 (2)2.6 (1)2.7 (1)2.7 (1)2.9 (1)3.0 (1)15/15
oMads-Neg Dahito 2.3 (0.8)3.3 (3)3.4 (3)3.4 (3)3.4 (2)3.4 (2)3.5 (2)15/15

Δ fopt
f9, 5-D
1e1
35
1e0
127
1e-1
214
1e-2
263
1e-3
300
1e-5
335
1e-7
369
#succ
15/15
CMA-ES-Akimoto Gharafi 4.5 (2)6.3 (3)5.6 (2)5.3 (2)5.1 (1)5.1 (1)5.1 (1)15/15
CMA-ES-pycma Gharafi 4.2 (2)6.0 (2)5.2 (1)5.0 (1)4.7 (1)4.7 (1)4.7 (1)15/15
CMAES-APOP-KMA Nguyen 3.2 (1.0)6.9 (3)6.2 (2)5.8 (1)5.4 (1)5.4 (1)5.3 (0.9)15/15
CMAES-APOP-KP Nguyen 4.2 (0.8)6.7 (3)6.5 (2)6.3 (1)6.1 (1)6.1 (1)6.1 (1.0)15/15
CMAES-APOP-MA Nguyen 3.1 (1)4.9 (3)4.8 (2)4.6 (1)4.5 (1)4.4 (1.0)4.3 (0.9)15/15
DD-CMA-ES-Akimoto Gharafi 4.7 (1)10 (10)8.0 (6)7.3 (5)6.8 (5)6.7 (4)6.5 (4)15/15
DD-CMA-ES-pycma Gharafi 3.6 (2)7.7 (10)6.3 (6)5.9 (5)5.5 (4)5.5 (4)5.4 (4)15/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 1.1 (0.4)2.5 (2)2.9 (2)2.8 (1)2.6 (1.0)2.7 (0.9)2.7 (0.9)15/15
oMads-Neg Dahito 0.98 (0.5)2.6 (2)2.9 (2)2.8 (1)2.7 (1)2.6 (1.0)2.5 (0.9)15/15

Δ fopt
f10, 5-D
1e1
349
1e0
500
1e-1
574
1e-2
607
1e-3
626
1e-5
829
1e-7
880
#succ
15/15
CMA-ES-Akimoto Gharafi 2.1 (0.4)1.8 (0.4)1.9 (0.2)1.9 (0.2)2.0 (0.1)1.7 (0.1)1.7 (0.1)15/15
CMA-ES-pycma Gharafi 1.7 (0.7)1.5 (0.5)1.5 (0.3)1.6 (0.3)1.6 (0.3)1.4 (0.2)1.5 (0.2)15/15
CMAES-APOP-KMA Nguyen 2.4 (1.0)2.0 (0.5)1.9 (0.3)1.9 (0.2)2.0 (0.2)1.6 (0.1)1.6 (0.1)15/15
CMAES-APOP-KP Nguyen 3.4 (0.9)2.9 (0.6)2.8 (0.3)2.7 (0.3)2.8 (0.3)2.3 (0.2)2.3 (0.2)15/15
CMAES-APOP-MA Nguyen 2.6 (0.7)2.1 (0.5)2.0 (0.2)2.0 (0.2)2.1 (0.2)1.7 (0.1)1.7 (0.1)15/15
DD-CMA-ES-Akimoto Gharafi 2.2 (0.6)1.9 (0.4)1.9 (0.3)1.9 (0.3)2.0 (0.3)1.7 (0.2)1.8 (0.2)15/15
DD-CMA-ES-pycma Gharafi 1.9 (0.8)1.6 (0.5)1.5 (0.4)1.6 (0.2)1.7 (0.2)1.5 (0.2)1.6 (0.2)15/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 3.0 (2)3.1 (1)3.4 (1)3.6 (1)3.9 (1)3.7 (2)11 (11)15/15
oMads-Neg Dahito 2.1 (2)2.8 (2)2.9 (2)3.3 (1)4.4 (3)3.9 (2)11 (11)15/15

Δ fopt
f11, 5-D
1e1
143
1e0
202
1e-1
763
1e-2
977
1e-3
1177
1e-5
1467
1e-7
1673
#succ
15/15
CMA-ES-Akimoto Gharafi 4.6 (0.7)3.9 (0.7)1.2 (0.1)1.0 (0.1)0.91 (0.1)↑00.84 (0.1)↑00.83 (0.1)↑015/15
CMA-ES-pycma Gharafi 3.6 (1.0)3.3 (0.6)0.98 (0.2)0.85 (0.1)↑10.77 (0.1)↑00.73 (0.1)↑00.73 (0.1)↑015/15
CMAES-APOP-KMA Nguyen 5.7 (1)5.0 (0.7)1.4 (0.2)1.2 (0.1)1.1 (0.1)0.94 (0.1)0.89 (0.1)↑115/15
CMAES-APOP-KP Nguyen 8.0 (4)7.5 (1)2.2 (0.3)1.8 (0.2)1.6 (0.2)1.4 (0.1)1.3 (0.1)15/15
CMAES-APOP-MA Nguyen 5.2 (1)4.6 (0.8)1.4 (0.2)1.2 (0.2)1.0 (0.1)0.90 (0.1)0.85 (0.1)↑015/15
DD-CMA-ES-Akimoto Gharafi 3.8 (2)3.5 (1.0)1.1 (0.1)1.00 (0.1)0.89 (0.1)0.84 (0.1)↑00.83 (0.1)↑015/15
DD-CMA-ES-pycma Gharafi 3.6 (0.9)3.3 (0.8)1.00 (0.2)0.87 (0.2)↑10.81 (0.1)↑00.76 (0.1)↑00.75 (0.1)↑015/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 3.9 (3)6.1 (4)2.4 (1)2.5 (1)2.7 (1)3.2 (1)17 (27)15/15
oMads-Neg Dahito 3.1 (2)5.4 (4)2.2 (1)2.2 (1)2.2 (1)3.5 (2)29 (43)14/15

Δ fopt
f12, 5-D
1e1
108
1e0
268
1e-1
371
1e-2
413
1e-3
461
1e-5
1303
1e-7
1494
#succ
15/15
CMA-ES-Akimoto Gharafi 11 (9)8.2 (8)8.6 (9)9.0 (9)8.9 (8)4.9 (4)4.8 (4)15/15
CMA-ES-pycma Gharafi 10 (11)6.9 (9)7.0 (9)7.4 (9)7.4 (8)3.4 (3)3.4 (3)15/15
CMAES-APOP-KMA Nguyen 7.6 (7)6.4 (6)6.9 (6)7.4 (5)7.4 (5)3.8 (2)3.8 (2)15/15
CMAES-APOP-KP Nguyen 9.4 (5)7.6 (6)7.6 (6)7.8 (6)7.8 (6)3.4 (3)4.5 (2)15/15
CMAES-APOP-MA Nguyen 5.5 (4)4.5 (4)4.8 (5)5.2 (5)5.4 (5)2.4 (2)2.5 (2)15/15
DD-CMA-ES-Akimoto Gharafi 7.4 (6)6.3 (6)6.5 (7)7.1 (6)7.4 (6)3.5 (3)3.6 (3)15/15
DD-CMA-ES-pycma Gharafi 8.4 (10)6.9 (7)7.3 (6)7.5 (5)7.3 (5)3.2 (2)3.1 (2)15/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 5.4 (4)3.5 (4)3.5 (4)4.2 (4)4.9 (4)2.6 (2)13 (14)11/15
oMads-Neg Dahito 3.7 (3)2.8 (4)3.5 (4)4.0 (4)4.4 (4)2.2 (1)7.8 (12)11/15

Δ fopt
f13, 5-D
1e1
132
1e0
195
1e-1
250
1e-2
319
1e-3
1310
1e-5
1752
1e-7
2255
#succ
15/15
CMA-ES-Akimoto Gharafi 2.9 (1)3.2 (1)3.6 (0.9)3.5 (0.4)1.0 (0.2)1.0 (0.1)1.0 (0.1)15/15
CMA-ES-pycma Gharafi 2.3 (0.8)3.0 (1)3.6 (0.9)3.4 (0.6)0.97 (0.2)0.99 (0.2)1.00 (0.1)15/15
CMAES-APOP-KMA Nguyen 2.6 (2)3.5 (2)3.9 (2)4.6 (2)1.3 (0.4)1.5 (0.8)1.5 (0.9)15/15
CMAES-APOP-KP Nguyen 3.9 (3)5.3 (3)4.9 (2)4.6 (1)1.4 (0.4)1.5 (0.5)1.6 (0.3)15/15
CMAES-APOP-MA Nguyen 2.2 (2)3.6 (2)4.2 (2)5.2 (1)1.4 (0.5)1.4 (0.5)2.1 (2)15/15
DD-CMA-ES-Akimoto Gharafi 2.4 (1.0)2.9 (1)3.4 (1)3.3 (0.7)0.98 (0.2)1.0 (0.2)0.99 (0.1)15/15
DD-CMA-ES-pycma Gharafi 2.3 (1)2.9 (1)3.7 (0.8)3.4 (0.6)0.98 (0.1)0.98 (0.1)1.00 (0.2)15/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 2.0 (1.0)7.9 (8)19 (17)19 (21)12 (17)682 (671)5e50/15
oMads-Neg Dahito 1.5 (0.4)4.6 (6)14 (17)20 (19)11 (12)1243 (1665)5e50/15

Δ fopt
f14, 5-D
1e1
10
1e0
41
1e-1
58
1e-2
90
1e-3
139
1e-5
251
1e-7
476
#succ
15/15
CMA-ES-Akimoto Gharafi 1.1 (2)2.2 (0.9)3.4 (0.7)3.8 (0.9)3.7 (1.0)3.8 (0.5)2.9 (0.3)15/15
CMA-ES-pycma Gharafi 0.64 (0.7)1.8 (0.8)3.0 (1.0)3.2 (0.7)3.3 (0.6)3.2 (0.4)2.5 (0.2)15/15
CMAES-APOP-KMA Nguyen 0.43 (0.4)↑11.8 (0.8)2.5 (0.5)3.0 (0.5)3.3 (0.6)3.4 (0.5)2.8 (0.3)15/15
CMAES-APOP-KP Nguyen 0.68 (0.8)2.3 (1)3.4 (1)3.8 (0.8)4.5 (0.9)5.3 (0.9)4.2 (0.4)15/15
CMAES-APOP-MA Nguyen 0.75 (1)1.9 (0.8)2.5 (0.5)3.0 (0.6)3.5 (0.7)3.9 (0.5)3.0 (0.3)15/15
DD-CMA-ES-Akimoto Gharafi 0.52 (0.8)2.0 (1)3.0 (0.9)3.3 (0.7)3.4 (1.0)3.6 (0.7)2.8 (0.2)15/15
DD-CMA-ES-pycma Gharafi 0.78 (0.9)1.9 (0.8)2.6 (0.7)3.1 (1)3.2 (0.9)3.3 (0.6)2.6 (0.3)15/15
EXPLO2 Huntsman 1.5 (2)11 (16)1340/15
oMads-2N Dahito 0.53 (0.5)2.1 (1)3.4 (1)3.0 (0.8)3.1 (0.8)5.4 (2)58 (53)3/15
oMads-Neg Dahito 0.75 (0.8)1.7 (1)2.9 (1)2.5 (0.8)2.6 (0.9)4.7 (2)35 (27)3/15

Δ fopt
f15, 5-D
1e1
511
1e0
9310
1e-1
19369
1e-2
19743
1e-3
20073
1e-5
20769
1e-7
21359
#succ
14/15
CMA-ES-Akimoto Gharafi 1.8 (2)0.87 (0.9)0.94 (0.6)0.94 (0.6)0.94 (0.6)0.94 (0.6)0.95 (0.7)15/15
CMA-ES-pycma Gharafi 1.1 (0.4)1.1 (0.7)1.2 (0.7)1.2 (0.7)1.2 (0.7)1.2 (0.7)1.2 (0.7)15/15
CMAES-APOP-KMA Nguyen 1.1 (2)1.2 (0.7)0.80 (0.6)0.82 (0.6)0.83 (0.6)0.84 (0.6)0.85 (0.6)15/15
CMAES-APOP-KP Nguyen 2.2 (2)1.0 (0.6)0.94 (0.8)0.96 (0.7)0.97 (0.7)0.98 (0.7)1.00 (0.7)15/15
CMAES-APOP-MA Nguyen 1.6 (2)0.84 (0.7)0.67 (0.5)0.71 (0.5)0.73 (0.6)0.77 (0.6)0.79 (0.6)15/15
DD-CMA-ES-Akimoto Gharafi 1.2 (2)1.3 (0.9)0.92 (0.5)0.93 (0.5)0.93 (0.5)0.93 (0.5)0.93 (0.5)15/15
DD-CMA-ES-pycma Gharafi 2.1 (2)0.85 (0.3)0.99 (0.5)0.99 (0.5)1.00 (0.5)1.00 (0.5)1.0 (0.5)15/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 2.0 (2)9.2 (12)41 (60)40 (31)40 (51)38 (38)37 (32)7/15
oMads-Neg Dahito 3.1 (4)7.4 (8)24 (21)23 (32)23 (19)22 (29)21 (21)10/15

Δ fopt
f16, 5-D
1e1
120
1e0
612
1e-1
2662
1e-2
10163
1e-3
10449
1e-5
11644
1e-7
12095
#succ
15/15
CMA-ES-Akimoto Gharafi 2.2 (1)2.5 (4)1.6 (2)0.61 (0.7)0.65 (0.5)0.62 (0.5)0.62 (0.5)15/15
CMA-ES-pycma Gharafi 1.9 (1)2.5 (2)1.2 (1)0.53 (0.6)0.58 (0.6)0.57 (0.5)0.57 (0.5)15/15
CMAES-APOP-KMA Nguyen 2.0 (2)5.9 (7)3.8 (4)1.5 (2)1.8 (2)1.8 (2)1.8 (2)15/15
CMAES-APOP-KP Nguyen 2.4 (2)3.9 (5)3.8 (4)1.3 (1)1.3 (1)1.6 (1)1.6 (1)15/15
CMAES-APOP-MA Nguyen 1.7 (2)16 (38)7.7 (11)2.9 (3)3.0 (3)2.9 (3)2.9 (3)15/15
DD-CMA-ES-Akimoto Gharafi 2.0 (2)2.6 (3)1.8 (1)0.58 (0.3)↑10.66 (0.6)0.63 (0.5)0.63 (0.5)15/15
DD-CMA-ES-pycma Gharafi 2.0 (2)4.1 (5)1.3 (1)0.70 (0.7)0.79 (0.6)0.74 (0.6)0.75 (0.6)15/15
EXPLO2 Huntsman 4.9 (8)1340/15
oMads-2N Dahito 1.3 (0.5)2.3 (2)3.9 (5)2.5 (2)6.7 (7)32 (38)99 (95)3/15
oMads-Neg Dahito 1.4 (3)2.4 (2)5.0 (6)3.5 (4)10 (10)63 (71)180 (172)2/15

Δ fopt
f17, 5-D
1e1
5.0
1e0
215
1e-1
899
1e-2
2861
1e-3
3669
1e-5
6351
1e-7
7934
#succ
15/15
CMA-ES-Akimoto Gharafi 1.7 (3)0.77 (0.4)0.73 (0.2)↑10.39 (0.6)0.51 (0.5)0.81 (0.7)0.92 (0.6)15/15
CMA-ES-pycma Gharafi 1.6 (2)1.5 (0.3)0.97 (1)0.66 (0.6)0.67 (0.5)0.80 (0.8)1.0 (0.5)15/15
CMAES-APOP-KMA Nguyen 1.3 (1)0.63 (0.2)↑01.0 (2)0.53 (0.7)0.95 (0.6)0.90 (0.2)0.94 (0.2)15/15
CMAES-APOP-KP Nguyen 1.4 (1)0.75 (0.2)↑10.79 (0.2)↑10.84 (0.9)0.88 (0.7)1.0 (0.5)1.1 (0.1)15/15
CMAES-APOP-MA Nguyen 1 (1)0.71 (0.4)0.88 (2)0.79 (1)0.97 (1.0)1.2 (0.6)1.2 (0.1)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (2)0.80 (0.4)1.3 (2)0.70 (0.7)0.73 (0.6)1.0 (0.5)1.1 (0.4)15/15
DD-CMA-ES-pycma Gharafi 2.2 (4)0.60 (0.2)↑01.6 (2)1.2 (0.9)1.1 (0.9)1.3 (0.8)1.4 (0.8)15/15
EXPLO2 Huntsman 3.3 (3)1340/15
oMads-2N Dahito 2.1 (2)4.4 (7)2.6 (4)3.2 (4)14 (12)192 (160)5e50/15
oMads-Neg Dahito 19 (4)4.0 (3)6.3 (6)8.4 (11)40 (31)5e50/15

Δ fopt
f18, 5-D
1e1
103
1e0
378
1e-1
3968
1e-2
8451
1e-3
9280
1e-5
10905
1e-7
12469
#succ
15/15
CMA-ES-Akimoto Gharafi 0.97 (0.6)2.7 (0.2)0.55 (0.5)0.53 (0.5)0.66 (0.5)0.77 (0.3)0.88 (0.3)15/15
CMA-ES-pycma Gharafi 0.81 (0.4)3.9 (6)0.87 (0.9)0.76 (0.7)0.79 (0.5)0.80 (0.4)0.89 (0.6)15/15
CMAES-APOP-KMA Nguyen 0.70 (0.4)1.5 (0.4)0.58 (0.7)0.62 (0.6)0.68 (0.4)0.71 (0.4)0.78 (0.3)15/15
CMAES-APOP-KP Nguyen 0.83 (0.4)0.98 (0.4)0.67 (0.7)0.72 (0.8)0.80 (0.6)0.85 (0.5)0.87 (0.5)15/15
CMAES-APOP-MA Nguyen 0.63 (0.4)1.2 (0.6)0.60 (0.7)0.82 (0.1)↑10.91 (0.1)↑10.94 (0.1)0.94 (0.1)15/15
DD-CMA-ES-Akimoto Gharafi 0.77 (0.6)1.4 (0.4)0.62 (0.5)0.72 (0.5)0.75 (0.4)0.87 (0.3)0.92 (0.3)15/15
DD-CMA-ES-pycma Gharafi 0.70 (0.6)4.2 (7)1.7 (2)1.1 (0.8)1.1 (0.7)1.2 (0.5)1.1 (0.4)15/15
EXPLO2 Huntsman 2.3 (2)1340/15
oMads-2N Dahito 3.6 (7)5.7 (8)2.9 (4)12 (15)59 (71)5e50/15
oMads-Neg Dahito 2.7 (4)3.8 (5)2.5 (3)19 (17)758 (916)5e50/15

Δ fopt
f19, 5-D
1e1
1
1e0
1
1e-1
242
1e-2
1.0e5
1e-3
1.2e5
1e-5
1.2e5
1e-7
1.2e5
#succ
15/15
CMA-ES-Akimoto Gharafi 1 (0)1 (0)139 (111)0.91 (0.7)1.0 (0.5)1.0 (0.5)1.0 (0.5)15/15
CMA-ES-pycma Gharafi 1 (0)1 (0)126 (104)0.81 (0.8)0.92 (0.6)0.93 (0.6)0.94 (0.6)15/15
CMAES-APOP-KMA Nguyen 1 (0)1 (0)129 (80)0.71 (1)0.79 (1)0.82 (1)0.83 (1)15/15
CMAES-APOP-KP Nguyen 1 (0)1 (0)175 (210)1.4 (1)1.5 (1)1.5 (1)1.5 (1)15/15
CMAES-APOP-MA Nguyen 1 (0)1 (0)92 (67)0.46 (0.2)↑10.54 (0.5)0.57 (0.5)0.59 (0.5)15/15
DD-CMA-ES-Akimoto Gharafi 1 (0)1 (0)77 (62)0.82 (0.7)0.72 (0.6)0.73 (0.6)0.73 (0.6)15/15
DD-CMA-ES-pycma Gharafi 1 (0)1 (0)144 (214)0.94 (0.7)0.85 (0.6)0.86 (0.7)0.86 (0.7)15/15
EXPLO2 Huntsman 28 (18)1340/15
oMads-2N Dahito 1 (0)1 (0)0.46 (0.1)↑15e50/15
oMads-Neg Dahito 1 (0)1 (0)0.36 (0.1)↑034 (38)5e50/15

Δ fopt
f20, 5-D
1e1
16
1e0
851
1e-1
38111
1e-2
51362
1e-3
54470
1e-5
54861
1e-7
55313
#succ
14/15
CMA-ES-Akimoto Gharafi 1.9 (1)6.1 (4)0.99 (0.7)0.78 (0.5)0.74 (0.5)0.76 (0.5)0.78 (0.5)15/15
CMA-ES-pycma Gharafi 1.9 (1)5.7 (6)1.1 (0.7)0.89 (0.5)0.85 (0.5)0.88 (0.5)0.89 (0.5)15/15
CMAES-APOP-KMA Nguyen 1.5 (2)25 (28)31 (31)23 (26)22 (18)22 (17)22 (17)9/15
CMAES-APOP-KP Nguyen 1.5 (1)6.8 (4)1.9 (2)1.5 (1)1.5 (1)1.5 (1)1.5 (1)15/15
CMAES-APOP-MA Nguyen 2.0 (2)17 (20)19 (27)14 (16)13 (19)13 (20)13 (18)10/15
DD-CMA-ES-Akimoto Gharafi 1.4 (1)5.1 (5)0.88 (0.7)0.68 (0.5)0.65 (0.5)0.67 (0.5)0.68 (0.5)15/15
DD-CMA-ES-pycma Gharafi 1.5 (2)10 (9)1.4 (1)1.2 (1.0)1.2 (0.9)1.2 (0.9)1.2 (0.9)15/15
EXPLO2 Huntsman 2.1 (1)1340/15
oMads-2N Dahito 0.80 (0.6)3.0 (4)2.2 (3)1.6 (2)1.6 (2)1.5 (2)1.5 (2)15/15
oMads-Neg Dahito 1.2 (0.7)3.1 (4)3.8 (4)2.8 (3)2.7 (3)2.6 (3)2.6 (3)15/15

Δ fopt
f21, 5-D
1e1
41
1e0
1157
1e-1
1674
1e-2
1692
1e-3
1705
1e-5
1729
1e-7
1757
#succ
14/15
CMA-ES-Akimoto Gharafi 1.1 (1)4.1 (5)52 (52)52 (51)51 (51)51 (50)50 (50)15/15
CMA-ES-pycma Gharafi 3.7 (1)53 (6)148 (263)146 (260)145 (259)144 (256)141 (253)14/15
CMAES-APOP-KMA Nguyen 1.0 (0.7)4.8 (9)18 (24)18 (23)18 (23)18 (23)18 (23)15/15
CMAES-APOP-KP Nguyen 1.3 (1.0)11 (9)17 (16)17 (16)17 (16)17 (16)17 (15)15/15
CMAES-APOP-MA Nguyen 2.7 (0.9)19 (31)51 (32)51 (32)51 (32)50 (32)50 (32)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (1)8.1 (7)119 (29)194 (560)193 (556)190 (548)188 (540)14/15
DD-CMA-ES-pycma Gharafi 1.1 (1)31 (82)146 (95)145 (94)145 (94)144 (93)142 (92)14/15
EXPLO2 Huntsman 1.6 (0.9)0.82 (0.8)1.2 (1)1340/15
oMads-2N Dahito 0.83 (0.6)2.3 (2)2.7 (4)2.7 (3)2.7 (3)2.7 (3)2.7 (3)15/15
oMads-Neg Dahito 3.9 (1)2.7 (4)2.9 (4)2.9 (4)2.9 (4)2.9 (4)2.9 (4)15/15

Δ fopt
f22, 5-D
1e1
71
1e0
386
1e-1
938
1e-2
980
1e-3
1008
1e-5
1040
1e-7
1068
#succ
14/15
CMA-ES-Akimoto Gharafi 3.7 (9)11 (9)175 (206)217 (246)212 (240)208 (233)204 (227)15/15
CMA-ES-pycma Gharafi 5.1 (10)6.0 (8)204 (213)277 (582)269 (566)262 (549)255 (534)15/15
CMAES-APOP-KMA Nguyen 3.3 (8)25 (40)45 (35)43 (33)42 (32)42 (31)41 (30)15/15
CMAES-APOP-KP Nguyen 3.9 (10)23 (35)39 (48)38 (46)37 (45)36 (44)36 (42)15/15
CMAES-APOP-MA Nguyen 8.7 (11)24 (71)51 (83)51 (80)50 (78)49 (76)48 (74)15/15
DD-CMA-ES-Akimoto Gharafi 2.3 (0.9)28 (41)340 (225)328 (215)320 (209)312 (203)305 (198)14/15
DD-CMA-ES-pycma Gharafi 5.2 (9)15 (34)288 (185)365 (358)356 (351)347 (341)338 (332)14/15
EXPLO2 Huntsman 1.2 (1)5.1 (5)1340/15
oMads-2N Dahito 3.4 (4)5.1 (9)2.9 (3)2.9 (3)2.8 (3)2.8 (3)2.8 (3)15/15
oMads-Neg Dahito 2.8 (6)3.8 (4)2.8 (4)2.7 (4)2.7 (4)2.7 (4)2.8 (4)15/15

Δ fopt
f23, 5-D
1e1
3.0
1e0
518
1e-1
14249
1e-2
27890
1e-3
31654
1e-5
33030
1e-7
34256
#succ
15/15
CMA-ES-Akimoto Gharafi 3.1 (2)7.1 (7)19 (32)10 (16)8.5 (15)8.2 (14)9.5 (13)15/15
CMA-ES-pycma Gharafi 1.8 (2)9.4 (9)17 (15)9.4 (8)8.3 (7)8.0 (7)7.8 (6)14/15
CMAES-APOP-KMA Nguyen 1.9 (2)32 (45)8.1 (1)4.7 (0.7)4.3 (0.6)4.2 (0.6)4.1 (0.6)15/15
CMAES-APOP-KP Nguyen 2.4 (2)25 (28)10 (2)5.3 (1)4.7 (1)4.6 (1)4.5 (1)15/15
CMAES-APOP-MA Nguyen 2.6 (3)35 (65)7.6 (5)4.6 (2)4.3 (2)4.3 (2)4.2 (2)15/15
DD-CMA-ES-Akimoto Gharafi 2.7 (3)12 (13)6.7 (4)3.5 (2)3.1 (2)3.0 (2)3.0 (2)15/15
DD-CMA-ES-pycma Gharafi 2.0 (2)6.6 (6)5.8 (2)3.0 (1)2.7 (0.9)2.7 (0.9)2.6 (0.9)15/15
EXPLO2 Huntsman 2.8 (2)1340/15
oMads-2N Dahito 1.7 (2)0.66 (1.0)1.5 (1)3.9 (5)13 (20)5e50/15
oMads-Neg Dahito 1.5 (2)0.90 (1)0.53 (0.7)3.7 (4)11 (14)5e50/15

Δ fopt
f24, 5-D
1e1
1622
1e0
2.2e5
1e-1
6.4e6
1e-2
9.6e6
1e-3
9.6e6
1e-5
1.3e7
1e-7
1.3e7
#succ
3/15
CMA-ES-Akimoto Gharafi 1.8 (2)1.5 (2)0.33 (0.4)0.24 (0.2)0.25 (0.3)0.21 (0.2)0.27 (0.4)7/15
CMA-ES-pycma Gharafi 2.2 (2)46 (58)5.5 (5)3e60/15
CMAES-APOP-KMA Nguyen 1.5 (2)2.3 (2)0.32 (0.3)0.48 (0.6)0.48 (0.5)0.36 (0.5)0.36 (0.4)3/15
CMAES-APOP-KP Nguyen 6.1 (4)1.9 (2)0.29 (0.3)0.46 (0.4)0.46 (0.5)0.35 (0.4)0.35 (0.4)3/15
CMAES-APOP-MA Nguyen 3.3 (3)2.2 (2)0.19 (0.2)0.20 (0.3)0.20 (0.2)0.15 (0.2)0.15 (0.2)6/15
DD-CMA-ES-Akimoto Gharafi 1.6 (2)5.3 (7)0.37 (0.4)0.37 (0.4)0.37 (0.4)0.28 (0.3)0.28 (0.2)9/15
DD-CMA-ES-pycma Gharafi 1.9 (2)3.0 (6)2.8 (3)1.9 (2)1.8 (1)1.4 (2)1.4 (0.9)2/15
EXPLO2 Huntsman 1340/15
oMads-2N Dahito 1.5 (1)3.0 (4)5e50/15
oMads-Neg Dahito 0.86 (0.9)2.0 (2)5e50/15

Previous dimension | Dimension = 10 | Next dimension

Δ fopt
f1, 10-D
1e1
22
1e0
23
1e-1
23
1e-2
23
1e-3
23
1e-5
23
1e-7
23
#succ
15/15
CMA-ES-Akimoto Gharafi 5.1 (1.0)12 (2)18 (2)25 (2)31 (2)44 (3)57 (4)15/15
CMA-ES-pycma Gharafi 4.2 (2)11 (2)17 (3)24 (3)30 (3)43 (4)57 (4)15/15
CMAES-APOP-KMA Nguyen 3.6 (2)7.6 (2)12 (2)17 (2)21 (2)30 (3)38 (3)15/15
CMAES-APOP-KP Nguyen 4.3 (2)11 (2)16 (2)23 (3)29 (3)42 (4)54 (3)15/15
CMAES-APOP-MA Nguyen 3.1 (1)7.4 (1)11 (2)16 (2)20 (2)29 (2)37 (3)15/15
DD-CMA-ES-Akimoto Gharafi 4.4 (1)10 (2)16 (4)21 (4)28 (3)39 (4)51 (5)15/15
DD-CMA-ES-pycma Gharafi 3.3 (2)8.4 (2)14 (2)19 (2)25 (3)37 (4)48 (6)15/15
EXPLO2 Huntsman 2.1 (0.2)5.2 (2)2670/15
oMads-2N Dahito 2.7 (2)7.7 (4)13 (4)13 (3)13 (2)21 (4)22 (4)15/15
oMads-Neg Dahito 2.3 (2)6.5 (3)11 (3)12 (3)14 (4)20 (3)20 (3)15/15

Δ fopt
f2, 10-D
1e1
187
1e0
190
1e-1
191
1e-2
191
1e-3
193
1e-5
194
1e-7
195
#succ
15/15
CMA-ES-Akimoto Gharafi 12 (3)15 (2)16 (1)17 (2)18 (1)20 (2)21 (1)15/15
CMA-ES-pycma Gharafi 13 (2)15 (1)16 (1)17 (1)18 (1)19 (1)21 (1)15/15
CMAES-APOP-KMA Nguyen 14 (3)16 (2)18 (0.9)18 (0.9)19 (0.9)20 (0.9)21 (0.9)15/15
CMAES-APOP-KP Nguyen 21 (5)23 (4)25 (3)26 (3)26 (3)28 (2)29 (2)15/15
CMAES-APOP-MA Nguyen 13 (2)15 (2)17 (2)18 (2)18 (1)19 (1.0)20 (0.9)15/15
DD-CMA-ES-Akimoto Gharafi 4.8 (1)5.9 (1)6.7 (1)7.5 (1)8.2 (1)10 (2)11 (2)15/15
DD-CMA-ES-pycma Gharafi 5.0 (0.8)6.2 (1)6.9 (1)7.8 (1.0)8.4 (1.0)10 (0.9)11 (1)15/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 5.9 (1)8.2 (3)10 (4)13 (4)16 (5)20 (5)24 (5)15/15
oMads-Neg Dahito 4.8 (2)6.9 (3)11 (5)14 (8)16 (8)20 (7)24 (7)15/15

Δ fopt
f3, 10-D
1e1
1739
1e0
3600
1e-1
3609
1e-2
3636
1e-3
3642
1e-5
3646
1e-7
3651
#succ
15/15
CMA-ES-Akimoto Gharafi 2.0 (1)289 (330)2659 (3284)2640 (2323)2636 (2932)2633 (2788)2630 (3631)6/15
CMA-ES-pycma Gharafi 2.7 (1)647 (698)6260 (7473)6214 (8104)6204 (8337)6198 (6744)6190 (7289)3/15
CMAES-APOP-KMA Nguyen 6.9 (6)94 (48)930 (710)924 (956)923 (942)922 (1252)921 (831)7/15
CMAES-APOP-KP Nguyen 6.4 (7)181 (194)854 (1037)848 (902)847 (640)847 (823)846 (877)7/15
CMAES-APOP-MA Nguyen 6.3 (6)125 (152)863 (592)857 (817)856 (601)856 (934)855 (873)7/15
DD-CMA-ES-Akimoto Gharafi 2.7 (2)377 (426)1805 (1522)1793 (1531)1790 (2850)1789 (2302)1787 (2168)8/15
DD-CMA-ES-pycma Gharafi 3.0 (2)369 (490)1324 (1882)1315 (1079)1313 (1854)1312 (1170)1311 (1524)10/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 6.4 (8)1e60/15
oMads-Neg Dahito 8.1 (12)1992 (2396)1e60/15

Δ fopt
f4, 10-D
1e1
2234
1e0
3626
1e-1
3660
1e-2
3695
1e-3
3707
1e-5
3744
1e-7
28767
#succ
12/15
CMA-ES-Akimoto Gharafi 3.9 (4)5e60/15
CMA-ES-pycma Gharafi 6.9 (5)5e60/15
CMAES-APOP-KMA Nguyen 7.5 (6)2e60/15
CMAES-APOP-KP Nguyen 9.0 (7)2e60/15
CMAES-APOP-MA Nguyen 10 (5)2e60/15
DD-CMA-ES-Akimoto Gharafi 3.7 (3)5e60/15
DD-CMA-ES-pycma Gharafi 7.1 (6)5e60/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 11 (12)1e60/15
oMads-Neg Dahito 21 (23)1e60/15

Δ fopt
f5, 10-D
1e1
20
1e0
20
1e-1
20
1e-2
20
1e-3
20
1e-5
20
1e-7
20
#succ
15/15
CMA-ES-Akimoto Gharafi 5.3 (1)6.3 (1)6.4 (1)6.5 (1)6.5 (1)6.5 (1)6.5 (1)15/15
CMA-ES-pycma Gharafi 5.0 (1)6.3 (1)6.7 (1)6.7 (1)6.7 (1)6.7 (1)6.7 (1)15/15
CMAES-APOP-KMA Nguyen 3.7 (0.7)4.6 (0.8)4.7 (0.8)4.7 (0.8)4.7 (0.8)4.7 (0.8)4.7 (0.8)15/15
CMAES-APOP-KP Nguyen 4.6 (0.7)6.0 (1)6.1 (1)6.1 (1)6.1 (1)6.1 (1)6.1 (1)15/15
CMAES-APOP-MA Nguyen 4.2 (1.0)4.9 (0.8)5.0 (0.8)5.0 (0.8)5.0 (0.8)5.0 (0.8)5.0 (0.8)15/15
DD-CMA-ES-Akimoto Gharafi 4.9 (2)6.6 (2)6.8 (2)6.8 (2)6.8 (2)6.8 (2)6.8 (2)15/15
DD-CMA-ES-pycma Gharafi 5.4 (1)6.5 (2)7.0 (2)7.0 (2)7.0 (2)7.0 (2)7.0 (2)15/15
EXPLO2 Huntsman 6.6 (2)2670/15
oMads-2N Dahito 3.8 (2)4.1 (2)4.4 (2)4.4 (2)4.4 (2)4.4 (2)4.4 (2)15/15
oMads-Neg Dahito 4.3 (2)5.2 (2)6.0 (4)6.3 (4)6.5 (4)7.1 (4)7.5 (4)15/15

Δ fopt
f6, 10-D
1e1
412
1e0
623
1e-1
826
1e-2
1039
1e-3
1292
1e-5
1841
1e-7
2370
#succ
15/15
CMA-ES-Akimoto Gharafi 1.7 (0.4)1.7 (0.3)1.8 (0.2)1.8 (0.2)1.7 (0.2)1.6 (0.1)1.6 (0.1)15/15
CMA-ES-pycma Gharafi 1.7 (0.5)1.7 (0.4)1.8 (0.3)1.7 (0.3)1.7 (0.2)1.6 (0.2)1.6 (0.2)15/15
CMAES-APOP-KMA Nguyen 1.5 (0.4)1.5 (0.3)1.5 (0.2)1.5 (0.2)1.5 (0.2)1.4 (0.1)1.4 (0.2)15/15
CMAES-APOP-KP Nguyen 1.7 (0.4)1.7 (0.3)1.7 (0.2)1.7 (0.2)1.7 (0.2)1.6 (0.3)1.6 (0.3)15/15
CMAES-APOP-MA Nguyen 1.5 (0.5)1.7 (0.3)1.7 (0.3)1.6 (0.3)1.6 (0.2)1.5 (0.2)1.4 (0.2)15/15
DD-CMA-ES-Akimoto Gharafi 1.8 (0.5)1.8 (0.4)1.8 (0.3)1.9 (0.3)1.8 (0.2)1.7 (0.2)1.7 (0.1)15/15
DD-CMA-ES-pycma Gharafi 1.5 (0.4)1.6 (0.3)1.6 (0.3)1.6 (0.3)1.6 (0.2)1.5 (0.2)1.5 (0.2)15/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 1.9 (0.7)2.2 (0.6)2.5 (0.5)2.8 (0.7)3.0 (0.6)3.2 (0.7)3.4 (0.4)15/15
oMads-Neg Dahito 1.4 (0.4)2.1 (0.5)2.4 (0.5)2.5 (0.5)2.6 (0.5)2.9 (0.7)3.0 (0.7)15/15

Δ fopt
f7, 10-D
1e1
172
1e0
1611
1e-1
4195
1e-2
5099
1e-3
5141
1e-5
5141
1e-7
5389
#succ
15/15
CMA-ES-Akimoto Gharafi 1.9 (0.6)0.92 (1)0.65 (0.5)0.71 (0.3)0.73 (0.3)0.73 (0.3)0.71 (0.3)15/15
CMA-ES-pycma Gharafi 1.7 (0.4)0.84 (0.6)0.69 (0.5)0.73 (0.3)0.73 (0.3)0.73 (0.3)0.71 (0.3)15/15
CMAES-APOP-KMA Nguyen 1.4 (0.4)1.3 (0.8)0.87 (0.8)0.90 (0.5)0.91 (0.5)0.91 (0.5)0.91 (0.5)15/15
CMAES-APOP-KP Nguyen 1.8 (0.6)1.5 (1)0.99 (0.3)1.1 (0.2)1.1 (0.3)1.1 (0.3)1.1 (0.3)15/15
CMAES-APOP-MA Nguyen 1.3 (0.6)1.8 (1)1.4 (0.2)1.4 (0.2)1.4 (0.2)1.4 (0.2)1.4 (0.3)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (0.6)1.0 (0.9)0.85 (0.3)0.77 (0.2)↑10.80 (0.3)0.80 (0.3)0.79 (0.2)↑115/15
DD-CMA-ES-pycma Gharafi 1.3 (0.5)1.1 (1)0.75 (0.3)0.73 (0.3)0.73 (0.3)0.73 (0.3)0.71 (0.3)↑015/15
EXPLO2 Huntsman 23 (29)2670/15
oMads-2N Dahito 2.3 (2)5.9 (9)33 (24)493 (475)875 (1031)875 (1021)835 (789)3/15
oMads-Neg Dahito 2.2 (2)8.0 (9)43 (59)1e60/15

Δ fopt
f8, 10-D
1e1
326
1e0
921
1e-1
1114
1e-2
1217
1e-3
1267
1e-5
1315
1e-7
1343
#succ
15/15
CMA-ES-Akimoto Gharafi 2.0 (0.5)4.5 (4)4.4 (3)4.3 (3)4.4 (3)4.5 (3)4.6 (3)15/15
CMA-ES-pycma Gharafi 1.8 (0.6)3.8 (0.9)3.9 (0.7)3.9 (0.7)4.0 (0.7)4.1 (0.6)4.2 (0.6)15/15
CMAES-APOP-KMA Nguyen 1.5 (0.8)3.3 (0.8)3.4 (0.7)3.4 (0.6)3.4 (0.6)3.5 (0.6)3.6 (0.5)15/15
CMAES-APOP-KP Nguyen 2.1 (1)4.8 (1)4.9 (1)5.0 (1.0)5.0 (0.9)5.1 (0.8)5.3 (0.8)15/15
CMAES-APOP-MA Nguyen 1.5 (0.5)4.5 (0.6)4.6 (0.6)4.7 (0.5)4.7 (0.5)4.8 (0.5)5.0 (0.5)15/15
DD-CMA-ES-Akimoto Gharafi 2.3 (1)4.4 (4)4.2 (3)4.2 (3)4.2 (3)4.3 (3)4.4 (3)15/15
DD-CMA-ES-pycma Gharafi 1.7 (0.6)3.8 (4)3.7 (3)3.7 (3)3.7 (3)3.8 (3)4.0 (3)15/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 2.9 (1)4.3 (3)4.3 (3)4.4 (3)4.5 (3)4.6 (3)4.7 (2)15/15
oMads-Neg Dahito 2.2 (0.7)3.4 (3)3.5 (3)3.6 (2)3.6 (2)3.7 (2)3.7 (2)15/15

Δ fopt
f9, 10-D
1e1
200
1e0
648
1e-1
857
1e-2
993
1e-3
1065
1e-5
1138
1e-7
1185
#succ
15/15
CMA-ES-Akimoto Gharafi 3.0 (0.9)5.1 (0.7)4.7 (0.6)4.4 (0.5)4.3 (0.4)4.4 (0.4)4.4 (0.4)15/15
CMA-ES-pycma Gharafi 2.8 (0.9)5.4 (0.8)5.1 (0.5)4.7 (0.4)4.6 (0.5)4.7 (0.4)4.8 (0.3)15/15
CMAES-APOP-KMA Nguyen 1.7 (0.3)5.8 (0.7)5.5 (0.5)5.2 (0.5)5.1 (0.4)5.0 (0.3)5.0 (0.3)15/15
CMAES-APOP-KP Nguyen 3.8 (1)6.0 (2)5.7 (1)5.3 (1)5.2 (1)5.2 (1)5.3 (1)15/15
CMAES-APOP-MA Nguyen 1.8 (0.4)6.1 (1)5.7 (0.9)5.4 (0.8)5.3 (0.7)5.2 (0.7)5.2 (0.7)15/15
DD-CMA-ES-Akimoto Gharafi 2.7 (0.7)6.3 (6)5.7 (4)5.3 (4)5.2 (3)5.2 (3)5.2 (3)15/15
DD-CMA-ES-pycma Gharafi 2.2 (0.9)5.7 (5)5.2 (4)4.9 (3)4.8 (3)4.8 (3)4.9 (3)15/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 0.85 (0.5)3.5 (1)3.6 (1.0)3.6 (0.8)3.7 (0.7)3.7 (0.7)3.7 (0.7)15/15
oMads-Neg Dahito 0.88 (0.5)3.0 (1)3.0 (0.9)2.9 (0.8)2.9 (0.7)2.9 (0.6)2.8 (0.6)15/15

Δ fopt
f10, 10-D
1e1
1835
1e0
2172
1e-1
2455
1e-2
2728
1e-3
2802
1e-5
4543
1e-7
4739
#succ
15/15
CMA-ES-Akimoto Gharafi 1.3 (0.2)1.3 (0.1)1.3 (0.1)1.2 (0.1)1.2 (0.1)0.83 (0.1)↑00.86 (0.1)↑015/15
CMA-ES-pycma Gharafi 1.2 (0.3)1.2 (0.2)1.2 (0.2)1.2 (0.2)1.2 (0.1)0.81 (0.1)↑00.85 (0.1)↑015/15
CMAES-APOP-KMA Nguyen 1.3 (0.4)1.3 (0.2)1.4 (0.1)1.3 (0.1)1.3 (0.1)0.84 (0.1)↑00.84 (0.0)↑015/15
CMAES-APOP-KP Nguyen 2.2 (0.4)2.1 (0.3)2.0 (0.2)1.9 (0.1)1.9 (0.1)1.2 (0.1)1.2 (0.1)15/15
CMAES-APOP-MA Nguyen 1.4 (0.4)1.5 (0.2)1.4 (0.1)1.3 (0.1)1.3 (0.1)0.84 (0.0)↑00.85 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 1.3 (0.3)1.3 (0.2)1.3 (0.1)1.2 (0.1)1.3 (0.1)0.84 (0.1)↑00.87 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 1.2 (0.2)1.3 (0.2)1.3 (0.1)1.2 (0.1)1.3 (0.1)0.85 (0.1)↑00.88 (0.1)↑015/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 5.0 (3)9.1 (3)13 (9)16 (8)45 (54)511 (654)1e60/15
oMads-Neg Dahito 4.2 (3)7.1 (4)11 (5)13 (6)31 (34)1456 (1752)1e60/15

Δ fopt
f11, 10-D
1e1
266
1e0
1041
1e-1
2602
1e-2
2954
1e-3
3338
1e-5
4092
1e-7
4843
#succ
15/15
CMA-ES-Akimoto Gharafi 6.0 (0.7)1.8 (0.2)0.81 (0.1)↑00.77 (0.1)↑00.73 (0.1)↑00.68 (0.0)↑00.64 (0.0)↑015/15
CMA-ES-pycma Gharafi 5.2 (0.9)1.6 (0.2)0.72 (0.1)↑00.70 (0.1)↑00.66 (0.0)↑00.62 (0.0)↑00.58 (0.0)↑015/15
CMAES-APOP-KMA Nguyen 6.8 (1)2.0 (0.2)0.88 (0.1)↑00.84 (0.1)↑00.78 (0.0)↑00.70 (0.0)↑00.63 (0.0)↑015/15
CMAES-APOP-KP Nguyen 12 (2)3.7 (0.3)1.6 (0.1)1.5 (0.1)1.4 (0.1)1.2 (0.1)1.1 (0.1)15/15
CMAES-APOP-MA Nguyen 6.7 (1)2.0 (0.2)0.87 (0.1)↑10.81 (0.1)↑00.76 (0.1)↑00.69 (0.0)↑00.63 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 6.0 (0.7)1.8 (0.2)0.84 (0.1)↑00.81 (0.0)↑00.77 (0.0)↑00.71 (0.0)↑00.66 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 5.6 (0.8)1.8 (0.2)0.79 (0.1)↑00.76 (0.1)↑00.71 (0.1)↑00.66 (0.1)↑00.61 (0.1)↑015/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 11 (6)6.3 (2)4.0 (1)4.9 (2)6.1 (3)14 (10)403 (479)1/15
oMads-Neg Dahito 7.3 (4)4.5 (3)3.0 (1)3.8 (2)5.4 (3)15 (17)870 (1027)0/15

Δ fopt
f12, 10-D
1e1
515
1e0
896
1e-1
1240
1e-2
1390
1e-3
1569
1e-5
3660
1e-7
5154
#succ
15/15
CMA-ES-Akimoto Gharafi 3.3 (3)4.1 (3)4.4 (2)4.7 (2)4.7 (2)2.5 (1)2.0 (0.9)15/15
CMA-ES-pycma Gharafi 3.1 (2)3.6 (3)3.9 (3)4.3 (3)4.4 (2)2.4 (0.9)2.0 (0.7)15/15
CMAES-APOP-KMA Nguyen 1.9 (1)3.0 (3)3.4 (3)3.8 (2)3.9 (2)2.2 (0.9)1.8 (0.6)15/15
CMAES-APOP-KP Nguyen 4.8 (4)5.7 (5)6.0 (5)6.3 (4)6.3 (3)3.3 (1)2.7 (1.0)15/15
CMAES-APOP-MA Nguyen 1.6 (0.2)2.8 (2)3.4 (3)3.7 (2)3.9 (2)2.2 (0.8)1.8 (0.6)15/15
DD-CMA-ES-Akimoto Gharafi 3.2 (2)3.2 (3)3.4 (4)3.8 (4)4.0 (3)2.3 (1)1.9 (0.9)15/15
DD-CMA-ES-pycma Gharafi 2.9 (2)3.5 (4)3.9 (4)4.1 (3)4.2 (3)2.3 (1)1.9 (0.9)15/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 2.5 (0.5)2.8 (3)3.8 (4)4.9 (4)6.3 (4)63 (83)2790 (3735)0/15
oMads-Neg Dahito 1.7 (0.1)2.1 (3)3.2 (4)4.9 (4)6.6 (6)71 (86)2731 (1261)1/15

Δ fopt
f13, 10-D
1e1
387
1e0
596
1e-1
797
1e-2
1014
1e-3
4587
1e-5
6208
1e-7
7779
#succ
15/15
CMA-ES-Akimoto Gharafi 2.0 (0.7)3.0 (1)2.9 (1)3.4 (1)0.82 (0.3)0.84 (0.2)0.92 (0.2)15/15
CMA-ES-pycma Gharafi 2.0 (0.5)3.0 (2)3.2 (1)3.5 (1)0.89 (0.3)1.0 (0.4)1.0 (0.4)15/15
CMAES-APOP-KMA Nguyen 2.1 (2)3.2 (2)4.2 (3)5.4 (3)1.4 (0.9)1.7 (0.9)1.9 (0.8)15/15
CMAES-APOP-KP Nguyen 2.2 (0.4)4.7 (3)7.2 (4)7.0 (3)1.9 (1.0)2.0 (1.0)2.1 (0.9)15/15
CMAES-APOP-MA Nguyen 2.6 (2)2.8 (2)5.5 (6)5.7 (5)1.6 (1)2.0 (1)2.0 (1)15/15
DD-CMA-ES-Akimoto Gharafi 2.2 (2)3.1 (1)3.4 (0.9)3.0 (0.6)0.79 (0.2)0.80 (0.2)↑10.80 (0.2)↑115/15
DD-CMA-ES-pycma Gharafi 2.2 (1)3.9 (2)3.5 (1.0)3.4 (0.7)0.87 (0.2)0.87 (0.2)1.00 (0.4)15/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 3.0 (2)10 (12)34 (65)101 (101)76 (73)1e60/15
oMads-Neg Dahito 4.2 (4)18 (20)41 (42)58 (58)51 (58)1e60/15

Δ fopt
f14, 10-D
1e1
37
1e0
98
1e-1
133
1e-2
205
1e-3
392
1e-5
687
1e-7
4305
#succ
15/15
CMA-ES-Akimoto Gharafi 1.2 (1)2.6 (0.4)3.6 (0.6)3.8 (0.5)3.2 (0.4)3.3 (0.3)0.80 (0.1)↑015/15
CMA-ES-pycma Gharafi 1.3 (1)2.5 (0.8)3.4 (0.8)3.6 (0.5)3.0 (0.3)3.2 (0.4)0.76 (0.1)↑015/15
CMAES-APOP-KMA Nguyen 1.3 (1.0)1.9 (0.4)2.4 (0.4)2.8 (0.6)2.6 (0.4)3.1 (0.3)0.82 (0.1)↑015/15
CMAES-APOP-KP Nguyen 1.2 (0.9)2.5 (0.5)3.2 (0.6)3.6 (0.8)3.7 (0.5)5.0 (0.7)1.3 (0.1)15/15
CMAES-APOP-MA Nguyen 0.82 (0.9)1.9 (0.5)2.6 (0.3)2.8 (0.4)2.6 (0.4)3.1 (0.3)0.81 (0.1)↑015/15
DD-CMA-ES-Akimoto Gharafi 1.1 (0.8)2.4 (0.7)3.2 (0.8)3.3 (0.6)2.8 (0.5)3.2 (0.4)0.78 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 0.99 (0.5)2.1 (0.7)2.9 (0.6)3.1 (0.5)2.7 (0.4)3.0 (0.4)0.75 (0.1)↑015/15
EXPLO2 Huntsman 1.2 (0.9)2670/15
oMads-2N Dahito 1.3 (1)3.5 (1)4.4 (0.7)4.4 (0.9)4.2 (1)10 (4)1678 (2205)0/15
oMads-Neg Dahito 0.96 (0.9)3.1 (1)3.8 (0.7)3.6 (0.7)3.4 (0.9)8.2 (2)3331 (2962)0/15

Δ fopt
f15, 10-D
1e1
4774
1e0
39246
1e-1
73643
1e-2
74669
1e-3
75790
1e-5
77814
1e-7
79834
#succ
12/15
CMA-ES-Akimoto Gharafi 0.83 (0.5)1.1 (0.6)0.97 (0.6)0.97 (0.6)0.97 (0.6)0.98 (0.6)0.98 (0.6)15/15
CMA-ES-pycma Gharafi 0.99 (0.7)1.0 (0.5)0.93 (0.2)0.94 (0.3)0.94 (0.2)0.95 (0.2)0.96 (0.2)15/15
CMAES-APOP-KMA Nguyen 1.8 (2)0.52 (0.2)↑00.47 (0.3)↑10.49 (0.3)↑10.50 (0.3)↑10.50 (0.3)↑10.50 (0.3)↑115/15
CMAES-APOP-KP Nguyen 1.7 (2)0.75 (0.5)0.59 (0.4)0.60 (0.4)0.61 (0.4)0.60 (0.4)0.60 (0.4)15/15
CMAES-APOP-MA Nguyen 2.2 (3)0.67 (0.2)↑00.44 (0.3)↑00.46 (0.3)↑00.47 (0.3)↑00.49 (0.3)↑00.49 (0.3)↑015/15
DD-CMA-ES-Akimoto Gharafi 0.95 (0.9)1.0 (0.6)0.79 (0.5)0.79 (0.5)0.80 (0.5)0.80 (0.5)0.81 (0.5)15/15
DD-CMA-ES-pycma Gharafi 1.2 (1)1.0 (0.6)0.90 (0.1)↑10.90 (0.1)↑10.90 (0.1)↑10.91 (0.1)↑10.92 (0.1)↑115/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 49 (64)1e60/15
oMads-Neg Dahito 44 (29)1e60/15

Δ fopt
f16, 10-D
1e1
425
1e0
7029
1e-1
15779
1e-2
45669
1e-3
51151
1e-5
65798
1e-7
71570
#succ
15/15
CMA-ES-Akimoto Gharafi 3.0 (2)0.67 (0.9)0.85 (0.7)0.48 (0.3)↑10.61 (0.4)0.51 (0.3)↑10.49 (0.3)↑115/15
CMA-ES-pycma Gharafi 2.4 (0.8)0.65 (0.6)1.0 (0.9)0.56 (0.3)↑10.60 (0.6)0.49 (0.5)0.46 (0.5)15/15
CMAES-APOP-KMA Nguyen 1.6 (1)2.5 (3)2.3 (2)0.92 (0.9)0.93 (0.9)0.75 (0.7)0.70 (0.7)15/15
CMAES-APOP-KP Nguyen 2.5 (2)4.4 (5)2.7 (2)1.2 (2)1.1 (1)0.92 (1)0.87 (1.0)15/15
CMAES-APOP-MA Nguyen 2.7 (2)1.3 (2)3.6 (5)1.4 (2)1.3 (1)1.1 (1)1.0 (1)15/15
DD-CMA-ES-Akimoto Gharafi 2.1 (1)0.80 (0.5)1.00 (0.7)0.55 (0.5)0.74 (0.5)0.69 (0.6)0.65 (0.6)15/15
DD-CMA-ES-pycma Gharafi 2.5 (1)0.96 (0.9)0.81 (0.6)0.92 (2)0.92 (1)0.84 (1)0.80 (1)15/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 0.67 (0.7)3.4 (5)37 (46)310 (296)1e60/15
oMads-Neg Dahito 1.4 (2)2.1 (2)75 (60)1e60/15

Δ fopt
f17, 10-D
1e1
26
1e0
429
1e-1
2203
1e-2
6329
1e-3
9851
1e-5
20190
1e-7
26503
#succ
15/15
CMA-ES-Akimoto Gharafi 1.2 (1)0.87 (0.2)0.59 (0.1)↑10.88 (1)1.2 (1.0)0.97 (0.4)1.1 (0.4)15/15
CMA-ES-pycma Gharafi 0.63 (2)0.96 (0.4)0.85 (2)0.74 (0.7)0.83 (0.7)0.96 (0.3)1.1 (0.4)15/15
CMAES-APOP-KMA Nguyen 0.55 (0.6)0.62 (0.3)0.70 (2)0.71 (0.7)0.81 (0.5)0.76 (0.2)↑10.71 (0.1)↑015/15
CMAES-APOP-KP Nguyen 0.86 (1)1.1 (0.6)0.97 (2)1.1 (1.0)1.4 (0.4)0.94 (0.2)0.88 (0.1)15/15
CMAES-APOP-MA Nguyen 0.78 (0.9)0.65 (0.2)↑10.48 (0.1)↑10.81 (1)0.99 (0.8)0.99 (0.1)0.90 (0.1)↑115/15
DD-CMA-ES-Akimoto Gharafi 0.66 (0.9)2.0 (1)0.83 (2)1.2 (1)1.5 (0.7)0.97 (0.3)1.0 (0.4)15/15
DD-CMA-ES-pycma Gharafi 0.64 (1)2.2 (0.4)0.99 (1)1.5 (1)1.4 (0.7)1.1 (0.5)1.1 (0.4)15/15
EXPLO2 Huntsman 0.97 (0.8)2670/15
oMads-2N Dahito 1.8 (2)93 (91)133 (164)1135 (1476)1e60/15
oMads-Neg Dahito 1.2 (1)100 (122)239 (307)2312 (2212)1e60/15

Δ fopt
f18, 10-D
1e1
238
1e0
836
1e-1
7012
1e-2
15928
1e-3
27536
1e-5
37234
1e-7
42708
#succ
15/15
CMA-ES-Akimoto Gharafi 0.97 (0.5)1.5 (0.5)1.1 (1)0.79 (0.5)0.85 (0.3)0.75 (0.3)0.80 (0.2)15/15
CMA-ES-pycma Gharafi 0.94 (0.4)2.1 (5)1.0 (0.9)1.1 (0.9)0.82 (0.6)0.79 (0.5)0.92 (0.3)15/15
CMAES-APOP-KMA Nguyen 0.62 (0.3)↑00.67 (0.1)↑00.90 (0.6)0.85 (0.7)0.55 (0.4)0.53 (0.3)↑10.55 (0.3)↑115/15
CMAES-APOP-KP Nguyen 0.86 (0.5)1.7 (0.5)0.87 (0.8)0.80 (0.1)↑00.56 (0.0)↑00.55 (0.0)↑00.61 (0.1)↑115/15
CMAES-APOP-MA Nguyen 0.58 (0.1)↑01.9 (5)0.86 (0.9)0.89 (0.1)0.60 (0.1)↑00.57 (0.1)↑00.58 (0.1)↑015/15
DD-CMA-ES-Akimoto Gharafi 0.91 (0.4)1.0 (0.6)1.1 (1)1.1 (0.8)0.88 (0.4)0.89 (0.3)0.90 (0.3)15/15
DD-CMA-ES-pycma Gharafi 0.79 (0.7)4.0 (5)1.3 (2)1.7 (1)1.3 (0.6)1.1 (0.4)1.2 (0.9)15/15
EXPLO2 Huntsman 7.8 (9)2670/15
oMads-2N Dahito 57 (56)181 (283)634 (960)1e60/15
oMads-Neg Dahito 52 (54)502 (620)1e60/15

Δ fopt
f19, 10-D
1e1
1
1e0
1
1e-1
10609
1e-2
9.8e5
1e-3
1.4e6
1e-5
1.4e6
1e-7
1.4e6
#succ
15/15
CMA-ES-Akimoto Gharafi 1 (0)1 (0)10 (8)0.59 (0.2)↑10.52 (0.2)↑00.52 (0.2)↑00.52 (0.2)↑015/15
CMA-ES-pycma Gharafi 1 (0)1 (0)12 (8)0.55 (0.2)↑00.51 (0.3)↑00.51 (0.3)↑10.52 (0.3)↑115/15
CMAES-APOP-KMA Nguyen 1 (0)1 (0)13 (11)1.3 (1)2.2 (3)2.2 (2)2.2 (2)7/15
CMAES-APOP-KP Nguyen 1 (0)1 (0)19 (22)1.6 (1)1.9 (2)1.9 (2)1.9 (2)8/15
CMAES-APOP-MA Nguyen 1 (0)1 (0)12 (4)0.37 (0.2)↑10.67 (0.9)0.67 (0.8)0.67 (0.7)13/15
DD-CMA-ES-Akimoto Gharafi 1 (0)1 (0)11 (7)0.55 (0.3)↑00.50 (0.3)↑00.50 (0.3)↑00.50 (0.3)↑015/15
DD-CMA-ES-pycma Gharafi 1 (0)1 (0)12 (9)0.65 (0.4)0.64 (0.3)↑00.64 (0.3)↑00.65 (0.3)↑115/15
EXPLO2 Huntsman 28 (18)2670/15
oMads-2N Dahito 1 (0)1 (0)0.02 (6e-3)↑01e60/15
oMads-Neg Dahito 1 (0)1 (0)0.02 (3e-3)↑01e60/15

Δ fopt
f20, 10-D
1e1
32
1e0
15426
1e-1
5.5e5
1e-2
5.7e5
1e-3
5.7e5
1e-5
5.8e5
1e-7
5.9e5
#succ
15/15
CMA-ES-Akimoto Gharafi 3.8 (2)1.1 (1)0.35 (0.2)↑00.34 (0.2)↑00.34 (0.2)↑00.35 (0.2)↑00.35 (0.2)↑015/15
CMA-ES-pycma Gharafi 3.5 (2)2.0 (1)0.88 (0.4)0.90 (0.4)0.90 (0.4)0.91 (0.4)0.91 (0.4)15/15
CMAES-APOP-KMA Nguyen 2.6 (1)13 (13)2e60/15
CMAES-APOP-KP Nguyen 3.8 (1)2.3 (2)2.1 (2)2.5 (2)2.5 (3)2.4 (2)2.4 (3)12/15
CMAES-APOP-MA Nguyen 2.5 (1)10 (7)2e60/15
DD-CMA-ES-Akimoto Gharafi 3.3 (2)1.2 (0.4)0.34 (0.2)↑00.34 (0.2)↑00.34 (0.2)↑00.35 (0.2)↑00.35 (0.2)↑015/15
DD-CMA-ES-pycma Gharafi 2.8 (1)1.7 (0.9)0.83 (0.4)0.85 (0.4)0.85 (0.4)0.85 (0.4)0.86 (0.4)15/15
EXPLO2 Huntsman 2.0 (0.6)2670/15
oMads-2N Dahito 3.1 (2)0.60 (0.8)1e60/15
oMads-Neg Dahito 2.5 (1)0.59 (0.7)1e60/15

Δ fopt
f21, 10-D
1e1
130
1e0
2236
1e-1
4392
1e-2
4487
1e-3
4618
1e-5
5074
1e-7
11329
#succ
15/15
CMA-ES-Akimoto Gharafi 4.2 (9)57 (65)44 (58)43 (57)42 (55)38 (50)17 (23)15/15
CMA-ES-pycma Gharafi 1.5 (0.6)52 (58)153 (152)150 (148)146 (144)134 (131)60 (59)14/15
CMAES-APOP-KMA Nguyen 4.7 (9)18 (24)33 (23)32 (23)32 (22)29 (20)13 (9)15/15
CMAES-APOP-KP Nguyen 3.7 (9)7.7 (11)15 (35)15 (34)15 (33)13 (30)6.1 (14)15/15
CMAES-APOP-MA Nguyen 2.8 (0.7)19 (40)27 (36)27 (35)26 (34)24 (31)11 (14)15/15
DD-CMA-ES-Akimoto Gharafi 13 (10)15 (12)65 (162)63 (159)62 (154)57 (141)25 (63)15/15
DD-CMA-ES-pycma Gharafi 6.5 (10)140 (332)109 (225)107 (220)104 (214)95 (195)43 (87)15/15
EXPLO2 Huntsman 3.0 (3)2670/15
oMads-2N Dahito 4.4 (8)5.5 (10)4.1 (6)4.1 (5)4.0 (5)3.7 (5)1.7 (2)15/15
oMads-Neg Dahito 5.7 (11)6.0 (10)8.8 (14)8.6 (14)8.4 (13)7.7 (12)3.5 (5)15/15

Δ fopt
f22, 10-D
1e1
98
1e0
2839
1e-1
6353
1e-2
6620
1e-3
6798
1e-5
8296
1e-7
10351
#succ
15/15
CMA-ES-Akimoto Gharafi 169 (15)74 (49)1501 (1489)1441 (2366)1403 (2175)1150 (1282)922 (1052)6/15
CMA-ES-pycma Gharafi 22 (34)191 (462)1879 (1188)1803 (1852)1756 (1545)1439 (2019)1153 (1241)5/15
CMAES-APOP-KMA Nguyen 57 (73)26 (28)140 (182)135 (202)131 (330)108 (154)86 (120)12/15
CMAES-APOP-KP Nguyen 65 (80)21 (25)135 (139)130 (133)127 (130)104 (106)83 (85)14/15
CMAES-APOP-MA Nguyen 63 (80)22 (26)244 (276)235 (288)229 (319)188 (199)150 (170)10/15
DD-CMA-ES-Akimoto Gharafi 6.8 (13)95 (145)475 (489)456 (1032)444 (513)364 (573)292 (293)11/15
DD-CMA-ES-pycma Gharafi 7.7 (12)114 (153)805 (1083)773 (937)752 (894)617 (528)494 (830)9/15
EXPLO2 Huntsman 3.6 (3)2670/15
oMads-2N Dahito 8.3 (8)3.3 (3)2.5 (3)2.5 (3)2.5 (3)2.2 (3)1.8 (2)15/15
oMads-Neg Dahito 12 (13)2.6 (5)4.6 (5)4.5 (4)4.4 (4)3.7 (4)3.1 (3)15/15

Δ fopt
f23, 10-D
1e1
3.0
1e0
915
1e-1
16425
1e-2
1.8e5
1e-3
2.0e5
1e-5
2.1e5
1e-7
2.1e5
#succ
15/15
CMA-ES-Akimoto Gharafi 2.1 (2)9.1 (7)12 (5)12 (15)11 (12)11 (14)11 (16)12/15
CMA-ES-pycma Gharafi 1.7 (2)21 (23)16 (22)16 (28)19 (26)19 (38)18 (25)9/15
CMAES-APOP-KMA Nguyen 2.1 (2)165 (135)16 (10)1.8 (0.7)1.7 (0.7)1.6 (0.6)1.6 (0.6)15/15
CMAES-APOP-KP Nguyen 1.5 (2)65 (127)17 (2)1.7 (0.4)1.6 (0.3)1.5 (0.3)1.5 (0.3)15/15
CMAES-APOP-MA Nguyen 1.4 (1)213 (119)19 (6)1.9 (0.6)1.7 (0.5)1.7 (0.5)1.7 (0.5)15/15
DD-CMA-ES-Akimoto Gharafi 1.7 (2)14 (15)24 (51)13 (16)15 (17)14 (18)14 (20)12/15
DD-CMA-ES-pycma Gharafi 2.2 (2)13 (10)58 (153)22 (28)20 (38)19 (27)19 (42)9/15
EXPLO2 Huntsman 1.5 (1)2670/15
oMads-2N Dahito 1.1 (1)0.91 (1)13 (22)1e60/15
oMads-Neg Dahito 0.96 (0.7)0.82 (0.9)11 (11)1e60/15

Δ fopt
f24, 10-D
1e1
98761
1e0
1.0e6
1e-1
7.5e7
1e-2
7.5e7
1e-3
7.5e7
1e-5
7.5e7
1e-7
7.5e7
#succ
1/15
CMA-ES-Akimoto Gharafi 4.0 (6)1.5 (2)0.22 (0.3)0.30 (0.5)0.30 (0.4)0.30 (0.3)0.30 (0.3)3/15
CMA-ES-pycma Gharafi 38 (49)15 (16)5e60/15
CMAES-APOP-KMA Nguyen 7.4 (5)0.74 (0.5)0.39 (0.4)2e60/15
CMAES-APOP-KP Nguyen 9.0 (8)0.99 (0.7)0.39 (1.0)0.39 (0.4)0.39 (0.4)0.39 (0.3)0.39 (0.5)1/15
CMAES-APOP-MA Nguyen 5.5 (7)0.64 (0.6)0.09 (0.1)0.13 (0.2)0.13 (0.2)0.13 (0.2)0.13 (0.1)3/15
DD-CMA-ES-Akimoto Gharafi 12 (19)2.2 (2)0.14 (0.2)0.14 (0.1)0.14 (0.2)0.14 (0.1)0.14 (0.2)6/15
DD-CMA-ES-pycma Gharafi 3.1 (2)3.1 (6)0.30 (0.3)0.30 (0.2)0.30 (0.4)0.30 (0.4)0.30 (0.5)3/15
EXPLO2 Huntsman 2670/15
oMads-2N Dahito 6.7 (5)1e60/15
oMads-Neg Dahito 4.9 (7)1e60/15

Previous dimension | Dimension = 20 | Next dimension

Δ fopt
f1, 20-D
1e1
43
1e0
43
1e-1
43
1e-2
43
1e-3
43
1e-5
43
1e-7
43
#succ
15/15
CMA-ES-Akimoto Gharafi 7.3 (1)13 (2)19 (2)26 (2)33 (2)45 (4)58 (3)15/15
CMA-ES-pycma Gharafi 6.7 (1)13 (1)19 (1)26 (2)32 (2)45 (2)58 (3)15/15
CMAES-APOP-KMA Nguyen 4.6 (0.4)8.5 (0.5)12 (1.0)16 (1)20 (1)28 (2)36 (2)15/15
CMAES-APOP-KP Nguyen 6.5 (2)13 (2)19 (2)25 (2)31 (3)44 (4)56 (4)15/15
CMAES-APOP-MA Nguyen 4.7 (0.9)8.6 (1)12 (1)16 (1)20 (1)28 (2)36 (2)15/15
DD-CMA-ES-Akimoto Gharafi 6.5 (1)12 (2)18 (2)24 (2)30 (2)41 (3)53 (3)15/15
DD-CMA-ES-pycma Gharafi 5.6 (1)11 (1)16 (2)22 (2)27 (2)38 (2)49 (2)15/15
EXPLO2 Huntsman 2.3 (0.6)86 (64)5010/15
oMads-2N Dahito 3.3 (2)12 (3)16 (5)16 (5)17 (7)27 (6)30 (9)15/15
oMads-Neg Dahito 3.4 (1)12 (3)16 (3)16 (3)19 (6)28 (4)31 (5)15/15

Δ fopt
f2, 20-D
1e1
385
1e0
386
1e-1
387
1e-2
388
1e-3
390
1e-5
391
1e-7
393
#succ
15/15
CMA-ES-Akimoto Gharafi 20 (2)23 (3)27 (4)28 (3)29 (2)31 (2)32 (2)15/15
CMA-ES-pycma Gharafi 20 (4)25 (4)27 (2)29 (2)31 (2)32 (2)33 (2)15/15
CMAES-APOP-KMA Nguyen 22 (4)26 (3)28 (2)29 (1)30 (1)30 (1)31 (1)15/15
CMAES-APOP-KP Nguyen 33 (5)40 (3)44 (3)45 (2)46 (2)47 (1)48 (1)15/15
CMAES-APOP-MA Nguyen 21 (3)24 (3)27 (4)28 (2)29 (1)31 (2)31 (2)15/15
DD-CMA-ES-Akimoto Gharafi 5.5 (0.7)6.4 (0.9)7.2 (0.8)7.9 (0.9)8.5 (0.9)10 (1)11 (1.0)15/15
DD-CMA-ES-pycma Gharafi 6.1 (1)7.2 (1)7.9 (1)8.5 (1.0)9.1 (1.0)10 (0.9)12 (0.9)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 10 (1)12 (2)15 (2)18 (2)20 (2)25 (3)30 (4)15/15
oMads-Neg Dahito 7.0 (1)9.4 (1.0)13 (2)15 (3)17 (2)22 (3)25 (3)15/15

Δ fopt
f3, 20-D
1e1
5066
1e0
7626
1e-1
7635
1e-2
7637
1e-3
7643
1e-5
7646
1e-7
7651
#succ
15/15
CMA-ES-Akimoto Gharafi 6.4 (2)1e70/15
CMA-ES-pycma Gharafi 14 (6)1.8e4 (1e4)1e70/15
CMAES-APOP-KMA Nguyen 16 (3)7390 (7082)4e60/15
CMAES-APOP-KP Nguyen 19 (4)4e60/15
CMAES-APOP-MA Nguyen 22 (10)4e60/15
DD-CMA-ES-Akimoto Gharafi 7.8 (6)1e70/15
DD-CMA-ES-pycma Gharafi 5.3 (5)1.9e4 (2e4)1e70/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 272 (286)2e60/15
oMads-Neg Dahito 670 (544)2e60/15

Δ fopt
f4, 20-D
1e1
4722
1e0
7628
1e-1
7666
1e-2
7686
1e-3
7700
1e-5
7758
1e-7
1.4e5
#succ
9/15
CMA-ES-Akimoto Gharafi 1e70/15
CMA-ES-pycma Gharafi 1e70/15
CMAES-APOP-KMA Nguyen 4e60/15
CMAES-APOP-KP Nguyen 4e60/15
CMAES-APOP-MA Nguyen 4e60/15
DD-CMA-ES-Akimoto Gharafi 1e70/15
DD-CMA-ES-pycma Gharafi 1.4e4 (2e4)1e70/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 642 (753)2e60/15
oMads-Neg Dahito 2008 (1910)2e60/15

Δ fopt
f5, 20-D
1e1
41
1e0
41
1e-1
41
1e-2
41
1e-3
41
1e-5
41
1e-7
41
#succ
15/15
CMA-ES-Akimoto Gharafi 4.9 (1)5.7 (1)5.9 (1)5.9 (1)5.9 (1)5.9 (1)5.9 (1)15/15
CMA-ES-pycma Gharafi 4.9 (0.9)5.7 (1)5.9 (1)5.9 (1)5.9 (1)5.9 (1)5.9 (1)15/15
CMAES-APOP-KMA Nguyen 4.4 (0.9)5.2 (1)5.4 (1)5.5 (1)5.5 (1)5.5 (1)5.5 (1)15/15
CMAES-APOP-KP Nguyen 5.2 (0.9)5.9 (1)6.1 (1)6.1 (1)6.1 (1)6.1 (1)6.1 (1)15/15
CMAES-APOP-MA Nguyen 4.0 (0.7)4.9 (1.0)5.0 (0.9)5.0 (0.9)5.0 (0.9)5.0 (0.9)5.0 (0.9)15/15
DD-CMA-ES-Akimoto Gharafi 5.0 (1.0)5.9 (1)6.0 (1)6.0 (1)6.0 (1)6.0 (1)6.0 (1)15/15
DD-CMA-ES-pycma Gharafi 5.0 (1)5.8 (0.9)6.0 (0.9)6.1 (0.9)6.1 (0.9)6.1 (0.9)6.1 (0.9)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 3.9 (1)4.8 (2)4.9 (3)4.9 (3)4.9 (3)4.9 (3)4.9 (3)15/15
oMads-Neg Dahito 4.4 (2)5.0 (2)5.0 (2)5.0 (2)5.0 (2)5.0 (2)5.0 (2)15/15

Δ fopt
f6, 20-D
1e1
1296
1e0
2343
1e-1
3413
1e-2
4255
1e-3
5220
1e-5
6728
1e-7
8409
#succ
15/15
CMA-ES-Akimoto Gharafi 1.5 (0.3)1.2 (0.2)1.1 (0.2)1.2 (0.2)1.2 (0.2)1.2 (0.1)1.2 (0.1)15/15
CMA-ES-pycma Gharafi 1.4 (0.2)1.2 (0.1)1.0 (0.1)1.1 (0.1)1.1 (0.1)1.1 (0.1)1.1 (0.2)15/15
CMAES-APOP-KMA Nguyen 1.3 (0.3)1.0 (0.2)0.92 (0.1)0.91 (0.1)0.91 (0.2)0.93 (0.1)0.95 (0.1)15/15
CMAES-APOP-KP Nguyen 1.6 (0.2)1.3 (0.1)1.2 (0.1)1.2 (0.1)1.1 (0.1)1.1 (0.1)1.2 (0.1)15/15
CMAES-APOP-MA Nguyen 1.5 (0.3)1.1 (0.3)1.0 (0.2)1.0 (0.2)1.0 (0.1)1.0 (0.2)1.0 (0.1)15/15
DD-CMA-ES-Akimoto Gharafi 1.5 (0.3)1.3 (0.4)1.2 (0.3)1.2 (0.2)1.2 (0.2)1.3 (0.2)1.3 (0.2)15/15
DD-CMA-ES-pycma Gharafi 1.4 (0.3)1.2 (0.2)1.1 (0.2)1.1 (0.1)1.1 (0.2)1.1 (0.2)1.1 (0.2)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 2.0 (0.7)2.0 (0.4)2.1 (0.3)2.2 (0.3)2.3 (0.3)2.5 (0.2)2.6 (0.3)15/15
oMads-Neg Dahito 1.6 (0.4)1.6 (0.4)1.8 (0.3)1.9 (0.3)1.9 (0.3)2.2 (0.2)2.5 (0.3)15/15

Δ fopt
f7, 20-D
1e1
1351
1e0
4274
1e-1
9503
1e-2
16523
1e-3
16524
1e-5
16524
1e-7
16969
#succ
15/15
CMA-ES-Akimoto Gharafi 0.90 (0.1)↑11.8 (1.0)1.3 (0.7)0.92 (0.3)0.92 (0.3)0.92 (0.3)0.91 (0.3)15/15
CMA-ES-pycma Gharafi 1.3 (1)2.2 (2)1.6 (1.0)1.1 (0.4)1.1 (0.4)1.1 (0.4)1.1 (0.4)15/15
CMAES-APOP-KMA Nguyen 1.7 (2)2.4 (0.4)1.4 (0.2)0.97 (0.2)0.97 (0.2)0.97 (0.2)0.96 (0.2)15/15
CMAES-APOP-KP Nguyen 1.2 (0.2)2.8 (0.3)1.6 (0.2)1.1 (0.1)1.1 (0.1)1.1 (0.1)1.1 (0.1)15/15
CMAES-APOP-MA Nguyen 2.3 (3)3.4 (0.7)1.8 (0.4)1.1 (0.3)1.2 (0.3)1.2 (0.3)1.1 (0.3)15/15
DD-CMA-ES-Akimoto Gharafi 1.2 (1)2.9 (2)1.8 (0.5)1.2 (0.3)1.2 (0.3)1.2 (0.3)1.2 (0.3)15/15
DD-CMA-ES-pycma Gharafi 1.2 (1)4.0 (2)2.2 (0.6)1.3 (0.4)1.3 (0.4)1.3 (0.4)1.3 (0.4)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 9.1 (13)2e60/15
oMads-Neg Dahito 11 (10)2e60/15

Δ fopt
f8, 20-D
1e1
2039
1e0
3871
1e-1
4040
1e-2
4148
1e-3
4219
1e-5
4371
1e-7
4484
#succ
15/15
CMA-ES-Akimoto Gharafi 3.6 (0.8)3.5 (0.4)3.8 (0.4)3.8 (0.4)3.9 (0.4)3.9 (0.4)3.9 (0.4)15/15
CMA-ES-pycma Gharafi 3.2 (0.6)3.2 (0.3)3.5 (0.3)3.6 (0.3)3.6 (0.3)3.7 (0.3)3.7 (0.3)15/15
CMAES-APOP-KMA Nguyen 3.0 (0.7)4.6 (0.9)4.8 (0.9)5.0 (0.8)5.0 (0.8)5.0 (0.8)5.0 (0.8)15/15
CMAES-APOP-KP Nguyen 3.8 (0.7)5.6 (0.5)6.1 (0.5)6.3 (0.5)6.4 (0.5)6.4 (0.5)6.4 (0.4)15/15
CMAES-APOP-MA Nguyen 2.7 (0.5)4.9 (0.5)5.3 (0.7)5.4 (0.5)5.5 (0.5)5.4 (0.5)5.4 (0.5)15/15
DD-CMA-ES-Akimoto Gharafi 3.2 (1)4.1 (3)4.3 (3)4.4 (3)4.4 (3)4.4 (3)4.4 (3)15/15
DD-CMA-ES-pycma Gharafi 3.0 (0.5)3.2 (0.2)3.3 (0.2)3.4 (0.2)3.4 (0.2)3.5 (0.2)3.5 (0.2)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 4.8 (1)5.3 (3)5.8 (3)6.1 (3)6.3 (3)6.3 (3)6.3 (3)15/15
oMads-Neg Dahito 3.5 (1)3.2 (1)3.9 (1.0)4.2 (0.9)4.4 (0.8)4.5 (0.9)4.5 (0.9)15/15

Δ fopt
f9, 20-D
1e1
1716
1e0
3102
1e-1
3277
1e-2
3379
1e-3
3455
1e-5
3594
1e-7
3727
#succ
15/15
CMA-ES-Akimoto Gharafi 4.0 (1)4.2 (0.7)4.5 (0.6)4.5 (0.6)4.6 (0.5)4.6 (0.5)4.6 (0.5)15/15
CMA-ES-pycma Gharafi 3.8 (0.6)4.4 (0.4)4.7 (0.4)4.7 (0.4)4.8 (0.3)4.8 (0.3)4.8 (0.3)15/15
CMAES-APOP-KMA Nguyen 3.2 (0.6)7.5 (15)7.9 (15)8.1 (16)8.1 (16)8.0 (15)7.8 (15)15/15
CMAES-APOP-KP Nguyen 4.7 (1)10 (18)11 (19)11 (19)11 (19)11 (19)11 (18)15/15
CMAES-APOP-MA Nguyen 3.4 (0.6)6.1 (0.4)6.5 (0.4)6.6 (0.4)6.6 (0.3)6.5 (0.3)6.4 (0.3)15/15
DD-CMA-ES-Akimoto Gharafi 3.4 (0.6)4.2 (0.4)4.4 (0.4)4.5 (0.3)4.5 (0.3)4.5 (0.3)4.5 (0.3)15/15
DD-CMA-ES-pycma Gharafi 3.4 (0.9)4.2 (0.4)4.5 (0.4)4.6 (0.4)4.6 (0.4)4.6 (0.4)4.6 (0.4)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 2.7 (0.9)4.2 (0.6)4.9 (0.8)5.2 (0.9)5.4 (0.8)5.4 (0.8)5.4 (0.8)15/15
oMads-Neg Dahito 2.5 (0.8)3.8 (0.9)4.4 (0.9)4.8 (0.9)5.1 (0.9)5.4 (0.9)7.6 (4)15/15

Δ fopt
f10, 20-D
1e1
7413
1e0
8661
1e-1
10735
1e-2
13641
1e-3
14920
1e-5
17073
1e-7
17476
#succ
15/15
CMA-ES-Akimoto Gharafi 1.1 (0.2)1.1 (0.1)0.98 (0.1)0.82 (0.0)↑00.78 (0.0)↑00.72 (0.0)↑00.74 (0.0)↑015/15
CMA-ES-pycma Gharafi 1.2 (0.2)1.2 (0.2)1.0 (0.1)0.86 (0.0)↑00.81 (0.0)↑00.74 (0.0)↑00.76 (0.0)↑015/15
CMAES-APOP-KMA Nguyen 1.00 (0.2)1.1 (0.1)0.97 (0.1)0.81 (0.1)↑00.76 (0.0)↑00.70 (0.0)↑00.70 (0.0)↑015/15
CMAES-APOP-KP Nguyen 1.8 (0.2)1.8 (0.1)1.5 (0.1)1.2 (0.1)1.2 (0.1)1.1 (0.0)1.1 (0.0)15/15
CMAES-APOP-MA Nguyen 1.1 (0.1)1.1 (0.2)0.94 (0.1)0.79 (0.1)↑00.75 (0.1)↑00.69 (0.0)↑00.70 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 1.2 (0.2)1.2 (0.2)1.0 (0.1)0.82 (0.1)↑00.79 (0.1)↑00.73 (0.0)↑00.75 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 1.2 (0.2)1.2 (0.1)1.1 (0.1)0.86 (0.0)↑00.81 (0.0)↑00.75 (0.0)↑00.76 (0.0)↑015/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 13 (7)22 (14)29 (12)44 (36)2e60/15
oMads-Neg Dahito 8.0 (4)20 (10)29 (15)55 (58)1892 (1776)2e60/15

Δ fopt
f11, 20-D
1e1
1002
1e0
2228
1e-1
6278
1e-2
8586
1e-3
9762
1e-5
12285
1e-7
14831
#succ
15/15
CMA-ES-Akimoto Gharafi 4.2 (0.4)2.1 (0.2)0.83 (0.1)↑00.66 (0.0)↑00.61 (0.0)↑00.54 (0.0)↑00.49 (0.0)↑015/15
CMA-ES-pycma Gharafi 4.4 (0.4)2.2 (0.1)0.85 (0.0)↑00.66 (0.0)↑00.62 (0.0)↑00.55 (0.0)↑00.49 (0.0)↑015/15
CMAES-APOP-KMA Nguyen 4.3 (0.4)2.1 (0.1)0.82 (0.0)↑00.64 (0.0)↑00.59 (0.0)↑00.51 (0.0)↑00.45 (0.0)↑015/15
CMAES-APOP-KP Nguyen 10 (1.0)5.1 (0.3)1.9 (0.1)1.5 (0.1)1.4 (0.1)1.2 (0.0)1.0 (0.0)15/15
CMAES-APOP-MA Nguyen 4.2 (0.3)2.1 (0.1)0.81 (0.1)↑00.63 (0.0)↑00.58 (0.0)↑00.50 (0.0)↑00.45 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 4.2 (0.5)2.2 (0.2)0.84 (0.1)↑00.67 (0.0)↑00.62 (0.0)↑00.55 (0.0)↑00.50 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 4.3 (0.6)2.2 (0.2)0.87 (0.1)↑00.68 (0.1)↑00.64 (0.0)↑00.56 (0.0)↑00.51 (0.0)↑015/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 11 (4)13 (4)6.7 (2)6.8 (1.0)7.9 (3)12 (7)1907 (1719)0/15
oMads-Neg Dahito 13 (4)13 (4)6.4 (1)6.4 (1)7.4 (1)35 (38)2e60/15

Δ fopt
f12, 20-D
1e1
1042
1e0
1938
1e-1
2740
1e-2
3156
1e-3
4140
1e-5
12407
1e-7
13827
#succ
15/15
CMA-ES-Akimoto Gharafi 2.2 (0.3)2.5 (2)2.7 (2)3.1 (2)2.8 (1)1.2 (0.4)1.3 (0.4)15/15
CMA-ES-pycma Gharafi 2.6 (2)3.1 (2)3.3 (2)3.4 (2)3.0 (2)1.3 (0.4)1.4 (0.3)15/15
CMAES-APOP-KMA Nguyen 1.6 (1)2.4 (2)3.0 (2)3.2 (1)2.8 (1)1.2 (0.5)1.3 (0.4)15/15
CMAES-APOP-KP Nguyen 2.6 (2)3.4 (3)3.9 (3)4.3 (3)3.9 (2)1.7 (0.7)1.8 (0.6)15/15
CMAES-APOP-MA Nguyen 1.7 (1)2.6 (2)3.0 (2)3.2 (2)2.9 (1)1.3 (0.4)1.3 (0.4)15/15
DD-CMA-ES-Akimoto Gharafi 2.6 (2)3.7 (2)3.9 (2)4.0 (1)3.6 (1)1.5 (0.4)1.5 (0.4)15/15
DD-CMA-ES-pycma Gharafi 2.2 (2)3.2 (2)3.8 (1)3.8 (1)3.4 (0.7)1.4 (0.3)1.5 (0.3)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 5.8 (8)12 (12)19 (19)43 (39)299 (247)2e60/15
oMads-Neg Dahito 7.9 (12)20 (19)47 (46)190 (188)1565 (2086)2e60/15

Δ fopt
f13, 20-D
1e1
652
1e0
2021
1e-1
2751
1e-2
3507
1e-3
18749
1e-5
24455
1e-7
30201
#succ
15/15
CMA-ES-Akimoto Gharafi 2.8 (0.9)2.7 (2)2.6 (1)3.1 (2)0.70 (0.3)1.0 (0.8)1.0 (0.6)15/15
CMA-ES-pycma Gharafi 2.9 (1)3.8 (2)4.0 (2)3.3 (2)0.77 (0.4)1.1 (0.6)1.4 (0.5)15/15
CMAES-APOP-KMA Nguyen 1.5 (0.2)3.1 (3)5.7 (3)7.0 (3)1.7 (0.9)1.7 (0.6)2.0 (0.8)15/15
CMAES-APOP-KP Nguyen 3.1 (3)4.1 (4)6.0 (5)7.8 (5)1.8 (0.9)1.9 (0.2)2.2 (0.3)15/15
CMAES-APOP-MA Nguyen 4.1 (3)7.0 (5)6.3 (4)6.4 (4)2.0 (1)2.8 (0.5)2.5 (0.5)15/15
DD-CMA-ES-Akimoto Gharafi 3.0 (3)2.8 (2)3.5 (2)3.7 (2)0.77 (0.4)1.0 (0.4)1.1 (0.6)15/15
DD-CMA-ES-pycma Gharafi 3.5 (4)2.9 (2)4.3 (2)4.3 (3)1.1 (0.5)1.1 (0.5)1.0 (0.5)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 12 (18)19 (29)69 (82)159 (150)210 (221)2e60/15
oMads-Neg Dahito 13 (12)17 (19)35 (34)64 (65)147 (139)2e60/15

Δ fopt
f14, 20-D
1e1
75
1e0
239
1e-1
304
1e-2
451
1e-3
932
1e-5
1648
1e-7
15661
#succ
15/15
CMA-ES-Akimoto Gharafi 2.9 (0.8)2.5 (0.4)3.3 (0.4)3.8 (0.4)3.1 (0.2)3.8 (0.3)0.65 (0.0)↑015/15
CMA-ES-pycma Gharafi 2.5 (1)2.4 (0.4)3.3 (0.4)3.8 (0.3)3.2 (0.3)3.8 (0.2)0.67 (0.1)↑015/15
CMAES-APOP-KMA Nguyen 2.0 (1)1.8 (0.4)2.2 (0.4)2.8 (0.4)2.7 (0.3)3.5 (0.4)0.65 (0.0)↑015/15
CMAES-APOP-KP Nguyen 2.7 (0.9)2.5 (0.5)3.3 (0.4)4.0 (0.5)4.0 (0.5)6.1 (0.5)1.2 (0.1)15/15
CMAES-APOP-MA Nguyen 1.7 (0.7)1.7 (0.3)2.2 (0.4)2.7 (0.2)2.6 (0.2)3.4 (0.3)0.61 (0.1)↑015/15
DD-CMA-ES-Akimoto Gharafi 3.1 (0.9)2.4 (0.3)3.1 (0.2)3.6 (0.5)3.0 (0.3)3.7 (0.3)0.67 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 2.2 (0.9)2.0 (0.3)2.7 (0.2)3.4 (0.5)3.0 (0.4)3.7 (0.3)0.66 (0.0)↑015/15
EXPLO2 Huntsman 3.7 (2)5010/15
oMads-2N Dahito 2.1 (1)4.1 (1)5.2 (0.9)6.2 (1)6.4 (1)20 (6)2e60/15
oMads-Neg Dahito 2.8 (2)3.3 (1)4.2 (0.9)4.9 (1)4.8 (1)22 (14)2e60/15

Δ fopt
f15, 20-D
1e1
30378
1e0
1.5e5
1e-1
3.1e5
1e-2
3.2e5
1e-3
3.2e5
1e-5
4.5e5
1e-7
4.6e5
#succ
15/15
CMA-ES-Akimoto Gharafi 0.83 (0.3)1.0 (0.5)0.71 (0.2)↑00.71 (0.2)↑00.71 (0.2)↑00.52 (0.1)↑00.53 (0.1)↑015/15
CMA-ES-pycma Gharafi 0.87 (0.6)1.0 (0.5)0.73 (0.2)0.74 (0.2)0.75 (0.2)0.55 (0.2)↑00.56 (0.2)↑015/15
CMAES-APOP-KMA Nguyen 1.6 (0.7)0.46 (0.2)↑00.26 (0.1)↑00.27 (0.1)↑00.26 (0.1)↑00.19 (0.1)↑00.19 (0.1)↑015/15
CMAES-APOP-KP Nguyen 2.0 (0.4)0.50 (0.1)↑00.31 (0.2)↑00.31 (0.2)↑00.31 (0.2)↑00.23 (0.1)↑00.23 (0.1)↑015/15
CMAES-APOP-MA Nguyen 2.2 (0.5)0.51 (0.1)↑00.27 (0.1)↑00.27 (0.1)↑00.27 (0.1)↑00.19 (0.0)↑00.20 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 0.97 (0.4)0.84 (0.2)0.55 (0.2)↑00.55 (0.2)↑00.55 (0.2)↑00.41 (0.1)↑00.41 (0.1)↑015/15
DD-CMA-ES-pycma Gharafi 0.93 (0.4)1.1 (0.5)0.66 (0.2)↑00.66 (0.2)↑00.66 (0.2)↑00.49 (0.1)↑00.49 (0.1)↑015/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 2e60/15
oMads-Neg Dahito 2e60/15

Δ fopt
f16, 20-D
1e1
1384
1e0
27265
1e-1
77015
1e-2
1.4e5
1e-3
1.9e5
1e-5
2.0e5
1e-7
2.2e5
#succ
15/15
CMA-ES-Akimoto Gharafi 1.7 (0.7)0.89 (0.6)0.72 (0.2)↑10.74 (0.6)0.74 (0.5)0.84 (0.5)0.78 (0.4)15/15
CMA-ES-pycma Gharafi 1.7 (1.0)0.80 (0.7)0.76 (0.8)0.88 (0.9)1.0 (0.7)1.0 (0.7)1.0 (0.6)15/15
CMAES-APOP-KMA Nguyen 1.1 (0.6)3.6 (4)1.6 (1)1.1 (1.0)0.85 (0.7)0.85 (0.7)0.78 (0.6)15/15
CMAES-APOP-KP Nguyen 2.2 (1)4.7 (3)2.1 (1)1.2 (0.6)1.1 (0.8)1.1 (0.8)1.1 (0.7)15/15
CMAES-APOP-MA Nguyen 3.8 (0.4)8.2 (4)3.1 (1)1.8 (0.8)1.4 (1)1.4 (1.0)1.3 (0.9)15/15
DD-CMA-ES-Akimoto Gharafi 1.4 (0.5)0.68 (0.6)0.91 (0.6)0.77 (0.7)0.83 (0.8)1.0 (0.8)0.96 (0.7)15/15
DD-CMA-ES-pycma Gharafi 1.3 (0.4)0.88 (0.4)0.93 (0.6)0.84 (0.6)1.2 (1.0)1.2 (1.0)1.1 (0.9)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 0.86 (0.7)24 (30)2e60/15
oMads-Neg Dahito 1.3 (2)30 (22)2e60/15

Δ fopt
f17, 20-D
1e1
63
1e0
1030
1e-1
4005
1e-2
12242
1e-3
30677
1e-5
56288
1e-7
80472
#succ
15/15
CMA-ES-Akimoto Gharafi 1.2 (2)1.0 (0.4)1.3 (2)1.5 (1)1.3 (0.8)1.2 (0.4)1.0 (0.3)15/15
CMA-ES-pycma Gharafi 0.95 (1.0)0.81 (0.3)0.82 (1)0.86 (0.6)0.76 (0.4)0.87 (0.3)0.85 (0.3)15/15
CMAES-APOP-KMA Nguyen 0.78 (0.6)0.58 (0.1)↑00.99 (2)1.8 (1)0.96 (0.4)0.78 (0.3)0.64 (0.2)↑015/15
CMAES-APOP-KP Nguyen 1.4 (0.9)0.83 (0.1)↑11.1 (2)1.3 (1)1.2 (0.2)0.95 (0.1)0.78 (0.1)↑115/15
CMAES-APOP-MA Nguyen 0.69 (0.5)0.62 (0.2)↑02.6 (4)2.2 (2)1.3 (0.2)0.98 (0.2)0.80 (0.2)↑115/15
DD-CMA-ES-Akimoto Gharafi 1.3 (2)0.87 (0.2)2.3 (2)2.1 (0.9)1.1 (0.4)0.93 (0.3)1.1 (0.5)15/15
DD-CMA-ES-pycma Gharafi 1.0 (1)0.93 (0.2)2.0 (2)1.7 (0.9)1.2 (0.8)1.2 (0.8)0.99 (0.6)15/15
EXPLO2 Huntsman 1.3 (0.7)5010/15
oMads-2N Dahito 155 (130)2e60/15
oMads-Neg Dahito 1.5 (1)2.7e4 (3e4)2e60/15

Δ fopt
f18, 20-D
1e1
621
1e0
3972
1e-1
19561
1e-2
28555
1e-3
67569
1e-5
1.3e5
1e-7
1.5e5
#succ
15/15
CMA-ES-Akimoto Gharafi 0.87 (0.2)0.47 (0.2)↑01.1 (0.9)1.7 (0.9)1.3 (0.5)0.87 (0.3)0.83 (0.3)15/15
CMA-ES-pycma Gharafi 0.83 (0.1)↑10.71 (0.3)0.86 (0.5)1.1 (0.4)0.94 (0.3)0.83 (0.5)0.80 (0.4)15/15
CMAES-APOP-KMA Nguyen 0.59 (0.2)↑00.32 (0.2)↑00.85 (0.6)1.0 (0.2)0.52 (0.1)↑00.43 (0.1)↑10.44 (0.1)↑115/15
CMAES-APOP-KP Nguyen 0.92 (0.4)0.54 (0.1)↑11.6 (0.5)1.4 (0.3)0.69 (0.1)↑00.54 (0.1)↑10.58 (0.3)↑115/15
CMAES-APOP-MA Nguyen 0.63 (0.2)↑02.0 (4)1.2 (0.9)1.2 (0.4)0.63 (0.2)↑00.54 (0.3)↑10.56 (0.3)↑115/15
DD-CMA-ES-Akimoto Gharafi 0.74 (0.2)↑10.92 (2)1.1 (0.9)1.4 (0.2)0.96 (0.7)0.86 (0.4)0.85 (0.3)15/15
DD-CMA-ES-pycma Gharafi 0.83 (0.4)2.2 (4)1.4 (0.5)1.6 (1.0)1.1 (0.7)1.3 (0.5)1.3 (0.4)15/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 4219 (4154)2e60/15
oMads-Neg Dahito 4100 (5061)2e60/15

Δ fopt
f19, 20-D
1e1
1
1e0
1
1e-1
3.4e5
1e-2
4.7e6
1e-3
6.2e6
1e-5
6.7e6
1e-7
6.7e6
#succ
15/15
CMA-ES-Akimoto Gharafi 1 (0)1 (0)1.2 (0.8)0.48 (0.1)↑00.49 (0.3)↑00.58 (0.3)↑10.58 (0.3)↑114/15
CMA-ES-pycma Gharafi 1 (0)1 (0)1.2 (0.5)0.57 (0.2)↑10.53 (0.2)↑00.56 (0.2)↑00.57 (0.2)↑015/15
CMAES-APOP-KMA Nguyen 1 (0)1 (0)2.7 (6)6.0 (9)9.1 (7)8.5 (8)8.4 (8)1/15
CMAES-APOP-KP Nguyen 1 (0)1 (0)1.4 (1)3.0 (2)4.7 (5)4.4 (3)4.4 (5)2/15
CMAES-APOP-MA Nguyen 1 (0)1 (0)0.98 (0.4)0.46 (0.6)1.2 (1)1.1 (1)1.1 (1)6/15
DD-CMA-ES-Akimoto Gharafi 1 (0)1 (0)1.1 (0.6)0.63 (0.4)0.74 (0.3)0.92 (1.0)0.92 (0.9)13/15
DD-CMA-ES-pycma Gharafi 1 (0)1 (0)1.3 (0.5)0.52 (0.2)↑00.51 (0.2)↑00.53 (0.2)↑00.53 (0.2)↑015/15
EXPLO2 Huntsman 48 (30)5010/15
oMads-2N Dahito 1 (0)1 (0)3.5e-3 (1e-3)↑02e60/15
oMads-Neg Dahito 1 (0)1 (0)2.8e-3 (6e-4)↑02e60/15

Δ fopt
f20, 20-D
1e1
82
1e0
46150
1e-1
3.1e6
1e-2
5.5e6
1e-3
5.5e6
1e-5
5.6e6
1e-7
5.6e6
#succ
14/15
CMA-ES-Akimoto Gharafi 4.0 (0.7)1.2 (0.5)0.22 (0.2)↑00.16 (0.1)↑00.16 (0.1)↑00.16 (0.1)↑00.16 (0.1)↑015/15
CMA-ES-pycma Gharafi 3.6 (0.7)3.6 (2)0.92 (0.4)0.75 (0.2)↑00.76 (0.2)↑00.76 (0.2)↑00.76 (0.2)↑015/15
CMAES-APOP-KMA Nguyen 2.4 (0.7)16 (10)4e60/15
CMAES-APOP-KP Nguyen 3.7 (1)3.8 (3)4.3 (4)4e60/15
CMAES-APOP-MA Nguyen 2.3 (0.6)11 (1)4e60/15
DD-CMA-ES-Akimoto Gharafi 2.9 (0.9)1.4 (0.5)0.23 (0.2)↑00.20 (0.1)↑00.20 (0.1)↑00.21 (0.1)↑00.21 (0.1)↑015/15
DD-CMA-ES-pycma Gharafi 2.9 (0.8)2.2 (0.9)0.67 (0.4)0.56 (0.2)↑00.56 (0.2)↑00.56 (0.2)↑00.56 (0.2)↑015/15
EXPLO2 Huntsman 3.5 (2)5010/15
oMads-2N Dahito 5.6 (3)0.38 (0.5)2e60/15
oMads-Neg Dahito 4.1 (1)0.80 (1.0)2e60/15

Δ fopt
f21, 20-D
1e1
561
1e0
6541
1e-1
14103
1e-2
14318
1e-3
14643
1e-5
15567
1e-7
17589
#succ
15/15
CMA-ES-Akimoto Gharafi 7.3 (5)93 (230)92 (138)91 (136)89 (133)84 (126)75 (111)15/15
CMA-ES-pycma Gharafi 3.1 (5)102 (171)170 (238)168 (234)164 (229)155 (216)137 (191)14/15
CMAES-APOP-KMA Nguyen 7.9 (19)18 (24)20 (24)20 (24)20 (23)18 (22)16 (19)15/15
CMAES-APOP-KP Nguyen 10 (6)35 (23)26 (23)26 (23)25 (22)24 (21)21 (19)15/15
CMAES-APOP-MA Nguyen 5.3 (9)27 (47)26 (28)26 (28)26 (27)24 (25)21 (22)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (4)91 (70)55 (65)54 (64)53 (63)50 (59)45 (52)15/15
DD-CMA-ES-pycma Gharafi 3.8 (4)173 (126)132 (258)130 (254)127 (248)120 (234)106 (207)14/15
EXPLO2 Huntsman 2.3 (2)5010/15
oMads-2N Dahito 3.7 (7)6.3 (8)7.2 (10)7.2 (10)7.0 (10)6.7 (9)6.0 (8)15/15
oMads-Neg Dahito 4.1 (6)6.2 (12)3.8 (5)3.7 (5)3.7 (5)3.5 (5)3.2 (4)15/15

Δ fopt
f22, 20-D
1e1
467
1e0
5580
1e-1
23491
1e-2
24163
1e-3
24948
1e-5
26847
1e-7
1.3e5
#succ
12/15
CMA-ES-Akimoto Gharafi 23 (38)198 (255)2876 (2899)2796 (2691)2708 (3132)2516 (3649)501 (666)2/15
CMA-ES-pycma Gharafi 5.1 (5)541 (899)1e70/15
CMAES-APOP-KMA Nguyen 6.3 (13)23 (26)189 (170)184 (231)178 (214)166 (190)33 (52)8/15
CMAES-APOP-KP Nguyen 16 (9)86 (131)390 (524)379 (503)367 (487)341 (379)68 (62)5/15
CMAES-APOP-MA Nguyen 38 (52)117 (201)438 (529)426 (692)412 (441)383 (511)76 (74)5/15
DD-CMA-ES-Akimoto Gharafi 8.0 (13)343 (922)6291 (1e4)6116 (6578)5924 (5756)5505 (6186)1096 (1140)1/15
DD-CMA-ES-pycma Gharafi 122 (31)335 (896)6190 (6281)6017 (4761)5828 (7016)5416 (6893)1079 (983)1/15
EXPLO2 Huntsman 3.6 (3)5010/15
oMads-2N Dahito 12 (12)15 (6)17 (29)17 (28)16 (28)15 (26)3.1 (5)15/15
oMads-Neg Dahito 13 (25)14 (16)17 (12)17 (11)16 (11)15 (10)3.0 (2)14/15

Δ fopt
f23, 20-D
1e1
3.0
1e0
1614
1e-1
67457
1e-2
3.7e5
1e-3
4.9e5
1e-5
8.1e5
1e-7
8.4e5
#succ
15/15
CMA-ES-Akimoto Gharafi 2.3 (2)11 (7)9.2 (3)13 (20)32 (48)24 (20)23 (30)6/15
CMA-ES-pycma Gharafi 1.6 (1)272 (376)19 (25)49 (48)141 (179)85 (106)83 (88)2/15
CMAES-APOP-KMA Nguyen 2.2 (4)359 (237)10 (3)2.2 (0.5)2.1 (0.9)1.2 (0.6)1.2 (0.5)15/15
CMAES-APOP-KP Nguyen 1.8 (2)106 (202)12 (2)2.6 (1)2.5 (3)1.5 (2)1.5 (2)15/15
CMAES-APOP-MA Nguyen 2.1 (2)334 (236)10 (2)2.1 (0.3)2.1 (0.9)1.3 (0.5)1.2 (0.5)15/15
DD-CMA-ES-Akimoto Gharafi 3.3 (3)14 (13)6.2 (4)15 (20)45 (48)27 (28)26 (46)6/15
DD-CMA-ES-pycma Gharafi 2.8 (3)68 (114)13 (3)112 (109)134 (159)81 (92)78 (96)2/15
EXPLO2 Huntsman 2.3 (3)5010/15
oMads-2N Dahito 2.3 (4)0.99 (1.0)16 (21)2e60/15
oMads-Neg Dahito 2.1 (3)0.35 (0.1)↑018 (23)2e60/15

Δ fopt
f24, 20-D
1e1
1.3e6
1e0
7.5e6
1e-1
5.2e7
1e-2
5.2e7
1e-3
5.2e7
1e-5
5.2e7
1e-7
5.2e7
#succ
3/15
CMA-ES-Akimoto Gharafi 0.73 (0.9)0.58 (0.6)2.8 (3)1e70/15
CMA-ES-pycma Gharafi 13 (13)20 (19)1e70/15
CMAES-APOP-KMA Nguyen 1.5 (1)0.28 (0.3)4e60/15
CMAES-APOP-KP Nguyen 1.2 (1)0.24 (0.2)↑14e60/15
CMAES-APOP-MA Nguyen 2.7 (5)0.49 (0.4)1.1 (1)1.1 (2)1.1 (2)1.1 (0.9)1.1 (1)1/15
DD-CMA-ES-Akimoto Gharafi 0.48 (0.7)0.44 (0.4)↑01e70/15
DD-CMA-ES-pycma Gharafi 1.3 (2)0.77 (0.8)1.4 (1)1.4 (2)1.4 (2)1.4 (0.7)1.4 (2)2/15
EXPLO2 Huntsman 5010/15
oMads-2N Dahito 2e60/15
oMads-Neg Dahito 2e60/15

Previous dimension | Dimension = 40 | First dimension

Δ fopt
f1, 40-D
1e1
83
1e0
83
1e-1
83
1e-2
83
1e-3
83
1e-5
83
1e-7
83
#succ
15/15
CMA-ES-Akimoto Gharafi 8.7 (1)15 (1)21 (1)27 (1)33 (2)46 (2)59 (3)15/15
CMA-ES-pycma Gharafi 8.5 (1)14 (1)20 (1)26 (1)32 (2)45 (2)57 (2)15/15
CMAES-APOP-KMA Nguyen 5.4 (0.5)9.1 (0.8)13 (0.9)16 (1)20 (1)27 (1)34 (1)15/15
CMAES-APOP-KP Nguyen 8.5 (0.7)14 (1)20 (0.7)26 (1)32 (1.0)44 (2)56 (2)15/15
CMAES-APOP-MA Nguyen 5.5 (0.5)9.1 (0.6)13 (0.9)16 (1)20 (0.9)27 (1)34 (1)15/15
DD-CMA-ES-Akimoto Gharafi 7.9 (1.0)14 (1)19 (1)25 (1)30 (2)42 (2)53 (2)15/15
DD-CMA-ES-pycma Gharafi 7.5 (0.7)13 (1)18 (1)23 (1)29 (2)39 (2)50 (2)15/15
EXPLO2 Huntsman 88 (118)10010/15

Δ fopt
f2, 40-D
1e1
796
1e0
797
1e-1
799
1e-2
799
1e-3
800
1e-5
802
1e-7
804
#succ
15/15
CMA-ES-Akimoto Gharafi 35 (3)42 (4)47 (3)51 (4)53 (2)56 (1)57 (1)15/15
CMA-ES-pycma Gharafi 37 (3)45 (4)51 (5)55 (4)58 (4)61 (3)63 (2)15/15
CMAES-APOP-KMA Nguyen 33 (4)39 (5)44 (4)47 (4)50 (4)53 (2)54 (1)15/15
CMAES-APOP-KP Nguyen 59 (9)67 (5)74 (3)80 (4)85 (3)88 (2)89 (2)15/15
CMAES-APOP-MA Nguyen 33 (2)40 (3)44 (2)47 (3)49 (3)53 (3)54 (2)15/15
DD-CMA-ES-Akimoto Gharafi 6.3 (1)7.4 (1)8.2 (1.0)8.8 (1)9.5 (0.9)11 (1)12 (1)15/15
DD-CMA-ES-pycma Gharafi 6.8 (0.8)7.8 (0.6)8.5 (0.8)9.2 (0.9)10 (0.9)11 (0.8)12 (0.9)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f3, 40-D
1e1
15526
1e0
15602
1e-1
15612
1e-2
15641
1e-3
15646
1e-5
15651
1e-7
15656
#succ
15/15
CMA-ES-Akimoto Gharafi 1624 (2330)2e70/15
CMA-ES-pycma Gharafi 5557 (7503)2e70/15
CMAES-APOP-KMA Nguyen 2287 (3396)8e60/15
CMAES-APOP-KP Nguyen 1288 (1238)8e60/15
CMAES-APOP-MA Nguyen 8e60/15
DD-CMA-ES-Akimoto Gharafi 1548 (1877)2e70/15
DD-CMA-ES-pycma Gharafi 485 (644)2e70/15
EXPLO2 Huntsman 10010/15

Δ fopt
f4, 40-D
1e1
15536
1e0
15601
1e-1
15659
1e-2
15678
1e-3
15703
1e-5
15733
1e-7
2.8e5
#succ
6/15
CMA-ES-Akimoto Gharafi 2e70/15
CMA-ES-pycma Gharafi 2e70/15
CMAES-APOP-KMA Nguyen 8e60/15
CMAES-APOP-KP Nguyen 8e60/15
CMAES-APOP-MA Nguyen 8e60/15
DD-CMA-ES-Akimoto Gharafi 2e70/15
DD-CMA-ES-pycma Gharafi 2e70/15
EXPLO2 Huntsman 10010/15

Δ fopt
f5, 40-D
1e1
98
1e0
116
1e-1
120
1e-2
121
1e-3
121
1e-5
121
1e-7
121
#succ
15/15
CMA-ES-Akimoto Gharafi 4.4 (0.6)4.5 (0.5)4.5 (0.3)4.4 (0.3)4.4 (0.3)4.4 (0.3)4.4 (0.3)15/15
CMA-ES-pycma Gharafi 4.7 (0.8)4.5 (0.7)4.5 (0.5)4.5 (0.5)4.5 (0.5)4.5 (0.5)4.5 (0.5)15/15
CMAES-APOP-KMA Nguyen 4.2 (0.6)4.1 (0.5)4.1 (0.4)4.0 (0.4)4.0 (0.4)4.0 (0.4)4.0 (0.4)15/15
CMAES-APOP-KP Nguyen 4.9 (0.7)4.5 (0.7)4.5 (0.6)4.5 (0.6)4.5 (0.6)4.5 (0.6)4.5 (0.6)15/15
CMAES-APOP-MA Nguyen 4.2 (0.8)4.2 (0.4)4.0 (0.4)4.0 (0.4)4.0 (0.4)4.0 (0.4)4.0 (0.4)15/15
DD-CMA-ES-Akimoto Gharafi 4.5 (0.5)4.3 (0.7)4.3 (0.7)4.3 (0.7)4.3 (0.7)4.3 (0.7)4.3 (0.7)15/15
DD-CMA-ES-pycma Gharafi 4.5 (0.5)4.3 (0.4)4.3 (0.3)4.3 (0.3)4.3 (0.3)4.3 (0.3)4.3 (0.3)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f6, 40-D
1e1
3507
1e0
5523
1e-1
7168
1e-2
9470
1e-3
11538
1e-5
15007
1e-7
19222
#succ
15/15
CMA-ES-Akimoto Gharafi 1.6 (0.2)1.5 (0.2)1.5 (0.2)1.5 (0.2)1.5 (0.2)1.5 (0.2)1.5 (0.2)15/15
CMA-ES-pycma Gharafi 1.5 (0.2)1.4 (0.2)1.4 (0.2)1.3 (0.2)1.3 (0.2)1.3 (0.2)1.3 (0.2)15/15
CMAES-APOP-KMA Nguyen 1.3 (0.3)1.2 (0.1)1.2 (0.2)1.1 (0.2)1.1 (0.2)1.2 (0.2)1.1 (0.1)15/15
CMAES-APOP-KP Nguyen 1.6 (0.1)1.4 (0.1)1.4 (0.1)1.3 (0.2)1.3 (0.1)1.4 (0.1)1.3 (0.1)15/15
CMAES-APOP-MA Nguyen 1.2 (0.2)1.1 (0.1)1.2 (0.1)1.1 (0.1)1.1 (0.1)1.1 (0.1)1.1 (0.1)15/15
DD-CMA-ES-Akimoto Gharafi 1.6 (0.2)1.6 (0.2)1.6 (0.3)1.5 (0.2)1.6 (0.2)1.7 (0.3)1.7 (0.3)15/15
DD-CMA-ES-pycma Gharafi 1.4 (0.2)1.3 (0.2)1.4 (0.2)1.3 (0.1)1.3 (0.1)1.3 (0.1)1.3 (0.1)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f7, 40-D
1e1
10698
1e0
17839
1e-1
41037
1e-2
66294
1e-3
66294
1e-5
66294
1e-7
68145
#succ
15/15
CMA-ES-Akimoto Gharafi 1.4 (0.8)2.6 (0.8)1.4 (0.4)0.92 (0.3)0.92 (0.3)0.92 (0.3)0.90 (0.3)15/15
CMA-ES-pycma Gharafi 1.2 (0.6)3.5 (1)2.0 (0.6)1.3 (0.4)1.3 (0.4)1.3 (0.4)1.3 (0.4)15/15
CMAES-APOP-KMA Nguyen 2.3 (0.7)2.0 (0.4)1.1 (0.2)0.74 (0.2)↑00.74 (0.2)↑00.74 (0.2)↑00.73 (0.2)↑015/15
CMAES-APOP-KP Nguyen 2.7 (2)2.6 (0.6)1.4 (0.3)0.97 (0.1)0.98 (0.1)0.98 (0.1)0.97 (0.1)15/15
CMAES-APOP-MA Nguyen 2.9 (0.8)2.2 (0.4)1.1 (0.2)0.76 (0.2)↑10.76 (0.2)↑10.76 (0.2)↑10.74 (0.2)↑115/15
DD-CMA-ES-Akimoto Gharafi 1.0 (0.8)2.9 (1.0)1.6 (0.5)1.0 (0.3)1.0 (0.3)1.0 (0.3)1.0 (0.3)15/15
DD-CMA-ES-pycma Gharafi 1.1 (0.7)4.3 (2)2.3 (0.9)1.5 (0.6)1.5 (0.6)1.5 (0.6)1.5 (0.6)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f8, 40-D
1e1
7080
1e0
10655
1e-1
11012
1e-2
11265
1e-3
11430
1e-5
11701
1e-7
11969
#succ
15/15
CMA-ES-Akimoto Gharafi 5.8 (0.6)5.4 (0.4)5.5 (0.4)5.5 (0.4)5.5 (0.4)5.5 (0.4)5.5 (0.4)15/15
CMA-ES-pycma Gharafi 5.6 (0.5)6.4 (4)6.5 (4)6.5 (4)6.5 (4)6.4 (3)6.4 (3)15/15
CMAES-APOP-KMA Nguyen 4.8 (0.4)13 (28)13 (30)13 (29)13 (29)13 (29)13 (28)15/15
CMAES-APOP-KP Nguyen 7.1 (0.7)11 (0.8)11 (0.8)11 (0.8)11 (0.8)11 (0.7)11 (0.7)15/15
CMAES-APOP-MA Nguyen 4.7 (0.4)4.2 (0.3)4.3 (0.3)4.3 (0.3)4.3 (0.3)4.3 (0.3)4.3 (0.3)15/15
DD-CMA-ES-Akimoto Gharafi 4.6 (0.8)4.3 (0.6)4.3 (0.5)4.4 (0.5)4.4 (0.5)4.4 (0.5)4.4 (0.5)15/15
DD-CMA-ES-pycma Gharafi 4.2 (0.5)4.9 (3)5.0 (3)5.0 (3)5.0 (3)5.0 (3)5.0 (3)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f9, 40-D
1e1
6122
1e0
12982
1e-1
13300
1e-2
13496
1e-3
13651
1e-5
13909
1e-7
14142
#succ
15/15
CMA-ES-Akimoto Gharafi 6.8 (0.4)4.6 (0.3)4.7 (0.3)4.8 (0.3)4.8 (0.3)4.8 (0.2)4.8 (0.2)15/15
CMA-ES-pycma Gharafi 6.7 (0.9)5.5 (2)5.6 (2)5.7 (2)5.7 (2)5.7 (2)5.7 (2)15/15
CMAES-APOP-KMA Nguyen 5.5 (0.4)7.4 (0.2)7.7 (0.2)7.7 (0.2)7.7 (0.2)7.7 (0.2)7.7 (0.2)15/15
CMAES-APOP-KP Nguyen 8.5 (0.8)8.8 (0.5)9.1 (0.5)9.1 (0.5)9.2 (0.5)9.2 (0.4)9.1 (0.4)15/15
CMAES-APOP-MA Nguyen 5.4 (0.5)18 (56)19 (56)19 (56)19 (56)18 (55)18 (54)15/15
DD-CMA-ES-Akimoto Gharafi 6.8 (0.8)5.0 (2)5.1 (2)5.1 (2)5.1 (2)5.2 (2)5.2 (2)15/15
DD-CMA-ES-pycma Gharafi 6.5 (0.3)5.2 (3)5.3 (3)5.4 (3)5.4 (3)5.4 (3)5.4 (3)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f10, 40-D
1e1
25536
1e0
29762
1e-1
35676
1e-2
51579
1e-3
56007
1e-5
65128
1e-7
70824
#succ
15/15
CMA-ES-Akimoto Gharafi 1.2 (0.1)1.2 (0.1)1.1 (0.1)0.79 (0.0)↑00.76 (0.0)↑00.68 (0.0)↑00.65 (0.0)↑015/15
CMA-ES-pycma Gharafi 1.2 (0.1)1.2 (0.1)1.1 (0.1)0.85 (0.0)↑00.82 (0.0)↑00.76 (0.0)↑00.72 (0.0)↑015/15
CMAES-APOP-KMA Nguyen 1.1 (0.1)1.1 (0.1)1.00 (0.1)0.74 (0.0)↑00.71 (0.0)↑00.65 (0.0)↑00.62 (0.0)↑015/15
CMAES-APOP-KP Nguyen 1.8 (0.2)1.9 (0.2)1.7 (0.1)1.3 (0.1)1.2 (0.0)1.1 (0.0)1.0 (0.0)15/15
CMAES-APOP-MA Nguyen 1.0 (0.1)1.1 (0.1)0.98 (0.1)0.73 (0.0)↑00.71 (0.0)↑00.65 (0.0)↑00.62 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 1.1 (0.1)1.2 (0.1)1.1 (0.1)0.79 (0.1)↑00.77 (0.0)↑00.69 (0.0)↑00.65 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 1.3 (0.1)1.3 (0.1)1.2 (0.1)0.89 (0.1)↑00.86 (0.0)↑00.78 (0.0)↑00.73 (0.0)↑015/15
EXPLO2 Huntsman 10010/15

Δ fopt
f11, 40-D
1e1
2368
1e0
4855
1e-1
11681
1e-2
25315
1e-3
29749
1e-5
38949
1e-7
48211
#succ
15/15
CMA-ES-Akimoto Gharafi 4.9 (0.2)2.6 (0.1)1.2 (0.0)0.57 (0.0)↑00.51 (0.0)↑00.42 (9e-3)↑00.37 (8e-3)↑015/15
CMA-ES-pycma Gharafi 5.8 (0.2)3.1 (0.1)1.3 (0.0)0.66 (0.0)↑00.58 (0.0)↑00.48 (1e-2)↑00.42 (7e-3)↑015/15
CMAES-APOP-KMA Nguyen 4.7 (0.2)2.4 (0.1)1.1 (0.0)0.52 (0.0)↑00.46 (0.0)↑00.38 (0.0)↑00.32 (9e-3)↑015/15
CMAES-APOP-KP Nguyen 14 (0.7)7.4 (0.3)3.2 (0.1)1.6 (0.0)1.4 (0.0)1.1 (0.0)0.95 (0.0)↑015/15
CMAES-APOP-MA Nguyen 4.7 (0.2)2.5 (0.1)1.1 (0.0)0.52 (0.0)↑00.46 (0.0)↑00.38 (0.0)↑00.32 (8e-3)↑015/15
DD-CMA-ES-Akimoto Gharafi 4.9 (0.2)2.6 (0.1)1.2 (0.0)0.58 (0.0)↑00.52 (0.0)↑00.44 (0.0)↑00.38 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 5.7 (0.4)3.1 (0.2)1.4 (0.1)0.68 (0.0)↑00.60 (0.0)↑00.50 (0.0)↑00.43 (0.0)↑015/15
EXPLO2 Huntsman 10010/15

Δ fopt
f12, 40-D
1e1
4169
1e0
7452
1e-1
9174
1e-2
10751
1e-3
13146
1e-5
22758
1e-7
25192
#succ
15/15
CMA-ES-Akimoto Gharafi 1.5 (1)1.6 (1)2.0 (0.8)2.1 (0.6)2.0 (0.5)1.4 (0.3)1.5 (0.2)15/15
CMA-ES-pycma Gharafi 1.4 (1)1.8 (1)2.0 (1)2.1 (0.7)2.0 (0.5)1.5 (0.3)1.5 (0.3)15/15
CMAES-APOP-KMA Nguyen 0.94 (0.8)1.3 (1)1.7 (0.8)1.8 (0.7)1.7 (0.6)1.3 (0.3)1.3 (0.3)15/15
CMAES-APOP-KP Nguyen 1.8 (1)2.1 (2)2.7 (2)2.8 (1)2.7 (1)2.0 (0.7)2.1 (0.6)15/15
CMAES-APOP-MA Nguyen 0.87 (1)1.1 (0.9)1.6 (0.9)1.6 (0.6)1.6 (0.5)1.2 (0.3)1.3 (0.3)15/15
DD-CMA-ES-Akimoto Gharafi 1.2 (0.8)1.3 (0.9)1.6 (1.0)1.8 (0.6)1.7 (0.4)1.3 (0.3)1.4 (0.3)15/15
DD-CMA-ES-pycma Gharafi 1.9 (1)1.9 (2)2.4 (1)2.4 (1)2.2 (0.8)1.6 (0.5)1.6 (0.5)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f13, 40-D
1e1
2029
1e0
6916
1e-1
8734
1e-2
11861
1e-3
71936
1e-5
98467
1e-7
1.2e5
#succ
15/15
CMA-ES-Akimoto Gharafi 2.3 (2)2.7 (2)4.0 (4)3.8 (3)0.72 (0.4)1.2 (1.0)1.4 (0.9)15/15
CMA-ES-pycma Gharafi 3.0 (4)2.0 (2)4.1 (4)4.7 (3)1.2 (0.7)1.8 (0.8)1.9 (0.9)15/15
CMAES-APOP-KMA Nguyen 1.4 (0.2)2.6 (3)3.5 (4)8.6 (9)2.0 (2)2.2 (2)6.7 (7)15/15
CMAES-APOP-KP Nguyen 2.2 (0.1)3.5 (3)8.8 (10)13 (9)2.6 (1)2.7 (2)10 (11)14/15
CMAES-APOP-MA Nguyen 3.8 (7)2.9 (3)6.0 (3)8.6 (8)2.6 (2)2.6 (0.9)4.2 (2)15/15
DD-CMA-ES-Akimoto Gharafi 2.1 (2)2.1 (2)3.3 (3)5.6 (6)0.99 (0.9)1.4 (0.9)1.7 (1)15/15
DD-CMA-ES-pycma Gharafi 1.9 (0.1)3.5 (4)7.7 (4)7.0 (3)1.3 (0.5)1.7 (0.9)2.1 (0.9)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f14, 40-D
1e1
304
1e0
616
1e-1
777
1e-2
1105
1e-3
2207
1e-5
4825
1e-7
57711
#succ
15/15
CMA-ES-Akimoto Gharafi 2.5 (0.5)2.4 (0.2)3.0 (0.2)3.7 (0.3)3.4 (0.3)3.6 (0.2)0.58 (0.0)↑015/15
CMA-ES-pycma Gharafi 2.0 (0.4)2.2 (0.2)2.9 (0.4)3.6 (0.4)3.6 (0.3)4.2 (0.3)0.65 (0.0)↑015/15
CMAES-APOP-KMA Nguyen 1.4 (0.2)1.5 (0.2)1.9 (0.2)2.6 (0.3)2.8 (0.2)3.6 (0.2)0.55 (0.0)↑015/15
CMAES-APOP-KP Nguyen 2.0 (0.5)2.1 (0.2)2.7 (0.3)3.7 (0.4)4.2 (0.4)6.4 (0.3)1.1 (0.1)15/15
CMAES-APOP-MA Nguyen 1.3 (0.2)1.5 (0.2)1.8 (0.2)2.5 (0.2)2.8 (0.2)3.5 (0.2)0.56 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 2.2 (0.4)2.2 (0.2)2.7 (0.2)3.4 (0.3)3.3 (0.3)3.8 (0.2)0.58 (0.0)↑015/15
DD-CMA-ES-pycma Gharafi 1.6 (0.4)1.8 (0.1)2.4 (0.2)3.1 (0.2)3.2 (0.3)4.2 (0.3)0.65 (0.0)↑015/15
EXPLO2 Huntsman 10010/15

Δ fopt
f15, 40-D
1e1
1.9e5
1e0
7.9e5
1e-1
1.0e6
1e-2
1.1e6
1e-3
1.1e6
1e-5
1.1e6
1e-7
1.1e6
#succ
15/15
CMA-ES-Akimoto Gharafi 0.76 (0.2)↑00.56 (0.2)↑00.48 (0.1)↑00.48 (0.1)↑00.48 (0.1)↑00.49 (0.1)↑00.49 (0.1)↑015/15
CMA-ES-pycma Gharafi 0.85 (0.3)0.65 (0.3)0.61 (0.5)0.62 (0.5)0.62 (0.5)0.63 (0.5)0.64 (0.5)15/15
CMAES-APOP-KMA Nguyen 1.1 (0.1)0.26 (0.0)↑00.20 (0.0)↑00.20 (0.0)↑00.19 (0.0)↑00.19 (0.0)↑00.19 (0.0)↑015/15
CMAES-APOP-KP Nguyen 1.1 (0.2)0.30 (0.1)↑00.23 (0.0)↑00.23 (0.0)↑00.23 (0.0)↑00.22 (0.0)↑00.22 (0.0)↑015/15
CMAES-APOP-MA Nguyen 1.4 (0.2)0.34 (0.1)↑00.27 (0.0)↑00.27 (0.0)↑00.27 (0.0)↑00.26 (0.1)↑00.26 (0.0)↑015/15
DD-CMA-ES-Akimoto Gharafi 0.76 (0.2)↑00.60 (0.2)↑00.53 (0.3)↑00.54 (0.3)↑00.54 (0.3)↑00.54 (0.3)↑00.54 (0.3)↑015/15
DD-CMA-ES-pycma Gharafi 0.76 (0.2)↑10.56 (0.2)↑00.55 (0.2)↑00.55 (0.2)↑00.55 (0.2)↑00.56 (0.2)↑00.56 (0.2)↑015/15
EXPLO2 Huntsman 10010/15

Δ fopt
f16, 40-D
1e1
5244
1e0
72122
1e-1
3.2e5
1e-2
7.1e5
1e-3
1.4e6
1e-5
2.0e6
1e-7
2.0e6
#succ
15/15
CMA-ES-Akimoto Gharafi 0.98 (0.3)0.88 (0.3)0.55 (0.3)↑00.45 (0.3)↑00.48 (0.6)0.69 (1)0.70 (1.0)15/15
CMA-ES-pycma Gharafi 0.88 (0.3)0.75 (0.4)0.84 (0.9)0.74 (0.5)0.56 (0.4)↑10.44 (0.3)↑00.46 (0.3)↑015/15
CMAES-APOP-KMA Nguyen 0.57 (0.3)↑03.7 (2)0.91 (0.5)0.54 (0.3)↑10.38 (0.4)↑10.37 (0.3)↑10.36 (0.3)↑115/15
CMAES-APOP-KP Nguyen 0.92 (0.2)6.3 (4)1.5 (1.0)0.76 (0.5)0.47 (0.3)↑00.36 (0.2)↑00.40 (0.3)↑015/15
CMAES-APOP-MA Nguyen 0.57 (0.3)↑111 (7)2.6 (2)1.3 (1)0.68 (0.5)0.49 (0.4)↑10.49 (0.4)↑115/15
DD-CMA-ES-Akimoto Gharafi 0.84 (0.3)1.1 (0.7)0.54 (0.3)↑00.42 (0.3)↑00.48 (0.3)↑10.44 (0.4)↑10.45 (0.4)↑115/15
DD-CMA-ES-pycma Gharafi 0.74 (0.2)↑10.88 (0.6)0.83 (0.6)0.79 (0.6)0.65 (0.5)0.61 (0.5)0.65 (0.4)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f17, 40-D
1e1
399
1e0
4220
1e-1
14158
1e-2
34948
1e-3
51958
1e-5
1.3e5
1e-7
2.7e5
#succ
14/15
CMA-ES-Akimoto Gharafi 0.81 (0.4)0.56 (0.1)↑10.68 (1)1.4 (0.7)1.5 (0.5)1.1 (0.7)0.80 (0.2)15/15
CMA-ES-pycma Gharafi 0.59 (0.2)↑10.49 (0.1)↑00.92 (1.0)1.2 (0.5)1.2 (0.4)0.93 (0.3)0.73 (0.3)15/15
CMAES-APOP-KMA Nguyen 0.39 (0.2)↑00.36 (0.1)↑02.8 (3)2.1 (0.9)1.8 (0.5)0.97 (0.3)0.64 (0.3)15/15
CMAES-APOP-KP Nguyen 0.51 (0.3)↑00.49 (0.1)↑02.6 (3)2.7 (0.7)2.2 (0.4)1.2 (0.1)0.75 (0.1)↑115/15
CMAES-APOP-MA Nguyen 0.41 (0.2)↑02.8 (8)3.3 (4)3.0 (0.7)2.3 (0.5)1.3 (0.3)0.75 (0.2)↑115/15
DD-CMA-ES-Akimoto Gharafi 0.76 (0.3)1.1 (0.2)2.4 (1)1.7 (0.6)1.6 (0.9)1.2 (0.6)0.79 (0.2)15/15
DD-CMA-ES-pycma Gharafi 0.54 (0.3)↑00.51 (0.1)↑01.7 (1)1.5 (0.4)1.6 (0.5)1.1 (0.4)1.3 (0.4)15/15
EXPLO2 Huntsman 1.1 (0.9)10010/15

Δ fopt
f18, 40-D
1e1
1442
1e0
16998
1e-1
47068
1e-2
1.3e5
1e-3
1.9e5
1e-5
6.7e5
1e-7
9.5e5
#succ
15/15
CMA-ES-Akimoto Gharafi 1.1 (0.3)0.81 (1)1.8 (1.0)1.1 (0.7)1.1 (0.3)0.60 (0.3)↑00.55 (0.2)↑115/15
CMA-ES-pycma Gharafi 0.81 (0.2)0.60 (0.9)1.0 (0.4)0.74 (0.3)0.71 (0.2)↑10.56 (0.3)↑00.46 (0.2)↑015/15
CMAES-APOP-KMA Nguyen 0.60 (0.2)↑11.2 (2)1.5 (0.3)0.70 (0.1)↑00.55 (0.1)↑00.37 (0.4)↑10.41 (0.3)↑115/15
CMAES-APOP-KP Nguyen 0.85 (0.3)1.0 (3)2.1 (0.6)0.95 (0.2)0.76 (0.2)↑10.43 (0.1)↑10.59 (0.1)↑115/15
CMAES-APOP-MA Nguyen 0.61 (0.2)↑12.2 (3)2.0 (0.4)0.87 (0.2)0.67 (0.1)↑00.35 (0.1)↑00.40 (0.3)↑115/15
DD-CMA-ES-Akimoto Gharafi 0.94 (0.5)1.6 (1.0)1.5 (0.5)1.1 (0.3)0.90 (0.3)0.51 (0.3)↑00.43 (0.1)↑015/15
DD-CMA-ES-pycma Gharafi 0.73 (0.2)↑00.89 (0.8)1.1 (0.4)0.99 (0.5)1.3 (0.8)1.5 (3)1.4 (2)15/15
EXPLO2 Huntsman 10010/15

Δ fopt
f19, 40-D
1e1
1
1e0
1
1e-1
1.4e6
1e-2
1.7e7
1e-3
2.6e7
1e-5
4.5e7
1e-7
4.5e7
#succ
8/15
CMA-ES-Akimoto Gharafi 1 (0)1 (0)0.90 (0.6)2.4 (2)2e70/15
CMA-ES-pycma Gharafi 1 (0)1 (0)1.2 (1.0)0.57 (0.4)↑10.72 (0.4)0.91 (1)0.91 (0.8)6/15
CMAES-APOP-KMA Nguyen 1 (0)1 (0)0.57 (0.2)↑02.1 (2)4.4 (5)8e60/15
CMAES-APOP-KP Nguyen 1 (0)1 (0)0.72 (0.2)↑03.4 (4)8e60/15
CMAES-APOP-MA Nguyen 1 (0)1 (0)0.70 (0.2)↑10.14 (0.2)↑00.53 (0.7)0.30 (0.4)0.30 (0.3)6/15
DD-CMA-ES-Akimoto Gharafi 1 (0)1 (0)1.1 (0.6)5.5 (9)2e70/15
DD-CMA-ES-pycma Gharafi 1 (0)1 (0)1.3 (1)0.96 (0.7)1.1 (1.0)1.4 (2)1.4 (1)4/15
EXPLO2 Huntsman 91 (36)10010/15

Δ fopt
f20, 40-D
1e1
222
1e0
1.3e5
1e-1
1.6e8
1e-2
1e-3
1e-5
1e-7
#succ
0/15
CMA-ES-Akimoto Gharafi 3.7 (0.5)1.9 (0.8)0.02 (0.0)↑01.8e7 (2e7) 1.8e7 (3e7) 1.8e7 (2e7) 1.8e7 (2e7) 10/15
CMA-ES-pycma Gharafi 3.3 (0.6)10 (5)0.87 (1)2e70/15
CMAES-APOP-KMA Nguyen 1.8 (0.3)42 (30)8e60/15
CMAES-APOP-KP Nguyen 3.2 (0.5)13 (13)8e60/15
CMAES-APOP-MA Nguyen 2.0 (0.4)20 (13)8e60/15
DD-CMA-ES-Akimoto Gharafi 3.0 (0.5)2.1 (0.7)0.02 (0.0)↑02.0e7 (2e7) 2.0e7 (2e7) 2.0e7 (2e7) 2.0e7 (2e7) 11/15
DD-CMA-ES-pycma Gharafi 2.4 (0.3)5.1 (2)2e70/15
EXPLO2 Huntsman 10010/15

Δ fopt
f21, 40-D
1e1
1044
1e0
21144
1e-1
1.0e5
1e-2
1.0e5
1e-3
1.0e5
1e-5
1.0e5
1e-7
1.0e5
#succ
26/30
CMA-ES-Akimoto Gharafi 2.9 (5)24 (37)36 (81)36 (80)36 (80)36 (80)36 (79)14/15
CMA-ES-pycma Gharafi 2.7 (4)154 (418)141 (154)141 (290)141 (285)140 (196)139 (287)10/15
CMAES-APOP-KMA Nguyen 0.54 (0.1)↑110 (10)14 (36)14 (36)14 (36)14 (36)14 (36)14/15
CMAES-APOP-KP Nguyen 3.7 (9)10 (20)15 (34)15 (34)15 (34)15 (34)15 (33)14/15
CMAES-APOP-MA Nguyen 7.5 (26)80 (129)55 (83)55 (47)54 (61)54 (81)54 (91)10/15
DD-CMA-ES-Akimoto Gharafi 2.0 (0.3)51 (81)59 (110)59 (133)59 (99)58 (101)58 (99)13/15
DD-CMA-ES-pycma Gharafi 1.2 (0.6)30 (67)65 (103)64 (106)64 (99)64 (113)64 (103)13/15
EXPLO2 Huntsman 10010/15

Δ fopt
f22, 40-D
1e1
3090
1e0
35442
1e-1
6.5e5
1e-2
6.5e5
1e-3
6.5e5
1e-5
6.5e5
1e-7
6.5e5
#succ
8/30
CMA-ES-Akimoto Gharafi 291 (188)171 (307)2e70/15
CMA-ES-pycma Gharafi 113 (415)250 (368)2e70/15
CMAES-APOP-KMA Nguyen 43 (78)85 (114)184 (157)184 (237)184 (289)183 (141)182 (187)1/15
CMAES-APOP-KP Nguyen 151 (49)80 (113)8e60/15
CMAES-APOP-MA Nguyen 205 (108)201 (339)8e60/15
DD-CMA-ES-Akimoto Gharafi 18 (58)219 (298)2e70/15
DD-CMA-ES-pycma Gharafi 410 (551)261 (323)2e70/15
EXPLO2 Huntsman 10010/15

Δ fopt
f23, 40-D
1e1
7.0
1e0
11925
1e-1
75453
1e-2
6.6e5
1e-3
1.3e6
1e-5
3.2e6
1e-7
3.4e6
#succ
15/15
CMA-ES-Akimoto Gharafi 1.3 (1)3.9 (3)3.5 (2)3.6 (7)21 (17)28 (49)27 (34)3/15
CMA-ES-pycma Gharafi 1.4 (2)6.8 (8)21 (4)22 (31)103 (120)87 (82)84 (97)1/15
CMAES-APOP-KMA Nguyen 2.3 (2)143 (89)26 (4)3.1 (0.5)2.0 (0.9)5.1 (6)4.9 (5)6/15
CMAES-APOP-KP Nguyen 1.3 (1)5.1 (4)22 (13)2.9 (0.3)2.9 (2)3.8 (3)3.7 (2)8/15
CMAES-APOP-MA Nguyen 1.3 (1)139 (87)25 (3)3.0 (0.3)2.1 (0.9)4.1 (5)4.0 (3)7/15
DD-CMA-ES-Akimoto Gharafi 1.9 (2)3.3 (2)3.8 (3)1.1 (0.8)25 (29)20 (23)19 (21)4/15
DD-CMA-ES-pycma Gharafi 1.4 (2)8.6 (7)44 (134)12 (16)102 (94)2e70/15
EXPLO2 Huntsman 0.82 (0.8)10010/15

Δ fopt
f24, 40-D
1e1
5.8e6
1e0
9.8e7
1e-1
3.0e8
1e-2
3.0e8
1e-3
3.0e8
1e-5
3.0e8
1e-7
3.0e8
#succ
1/15
CMA-ES-Akimoto Gharafi 0.63 (0.9)0.56 (0.4)2e70/15
CMA-ES-pycma Gharafi 8.4 (8)2e70/15
CMAES-APOP-KMA Nguyen 2.3 (2)0.28 (0.2)8e60/15
CMAES-APOP-KP Nguyen 1.4 (3)0.21 (0.3)8e60/15
CMAES-APOP-MA Nguyen 4.1 (5)0.35 (0.5)8e60/15
DD-CMA-ES-Akimoto Gharafi 0.61 (0.7)0.71 (0.6)2e70/15
DD-CMA-ES-pycma Gharafi 1.2 (2)0.34 (0.4)0.99 (1)0.99 (1)0.99 (1)0.99 (2)0.99 (1)1/15
EXPLO2 Huntsman 10010/15

Expected runtime (ERT in number of f-evaluations) divided by the respective best ERT measured during BBOB-2009 (when finite) in different dimensions. This ERT ratio and, in braces as dispersion measure, the half difference between 10 and 90%-tile of bootstrapped run lengths appear for each algorithm and target, the corresponding reference ERT  in the first row. The different target ∆f-values are shown in the top row. #succ is the number of trials that reached the (final) target fopt+ 10−8. The median number of conducted function evaluations is additionally given in italics, if the target in the last column was never reached. Entries, succeeded by a star, are statistically significantly better (according to the rank-sum test) when compared to all other algorithms of the table, with p = 0.05 or p = 10−k when the number k following the star is larger than 1, with Bonferroni correction by the number of functions (24). A ↑ signifies the number of trials that were worse than the ERT of the best algorithm from BBOB 2009 shown only when less than 10 percent were worse and the ERT was better.Best results are printed in bold.