Overview page

Table showing the aRT in number of function evaluations divided by the best aRT measured during BBOB-2009

Last dimension | Dimension = 2 | Next dimension

Δ HVref
f1, 2-D		 1e0
1		 1e-1
32		 1e-2
224		 1e-3
1567		 #succ
10/10
DEMO Tusar bbob-biobj 3.2 (2)2.0 (2)2.0 (0.4)1.7 (0.1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.1 (0.5)1.5 (0.8)1.0 (0.1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 295 (299)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 212 (283)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)1.0 (0.5)1.00 (0.4)1.1 (0.2)0/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)3.8 (2)2.4 (0.4)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)3.5 (2)4.4 (2)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 87 (206)17 (2)3.8 (0.7)3.3 (0.7)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1242 (1512)1212 (806)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.7 (2)1.4 (1)5.6 (4)26 (26)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (4)2.5 (1)9.2 (9)40 (30)0/10
RM-MEDA Auger bbob-biobj 376 (332)31 (7)7.2 (1)2.1 (0.2)0/10
SMS-EMOA-DE Auger bbob-biobj 280 (113)21 (2)4.8 (0.3)1.3 (0.1)10/10
SMS-EMOA-PM Auger bbob-biobj 174 (53)16 (3)7.8 (2)5.7 (1)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.9 (3)2.4 (2)4.6 (1)7.6 (0.6)10/10

Δ HVref
f2, 2-D		 1e0
5.0		 1e-1
38		 1e-2
200		 1e-3
780		 #succ
10/10
DEMO Tusar bbob-biobj 3.9 (3)3.3 (3)2.5 (1)1.8 (0.9)3/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.8)1.0 (0.6)1.00 (0.8)1 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 137 (188)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.1 (2)3.9 (0.5)1.1 (0.3)2.5 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 11 (14)6.0 (3)4.0 (2)8.1 (11)0/10
MO-DIRECT-Rank Wong bbob-biobj 7.7 (9)4.2 (2)5.3 (4)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 44 (25)15 (4)4.4 (0.8)3.5 (4)3/10
RANDOMSEARCH-100 Auger bbob-biobj 950 (385)1542 (3430)9666 (2e4)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (0.8)4.9 (9)15 (22)91 (81)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.6 (6)7.5 (15)20 (34)169 (113)0/10
RM-MEDA Auger bbob-biobj 128 (51)31 (9)10 (0.5)3.9 (0.9)5/10
SMS-EMOA-DE Auger bbob-biobj 99 (40)23 (4)7.3 (1)601 (449)0/10
SMS-EMOA-PM Auger bbob-biobj 57 (32)20 (6)8.3 (5)12 (3)1/10
UP-MO-CMA-ES Krause bbob-biobj 1.6 (3)3.1 (2)6.4 (4)9.4 (5)10/10

Δ HVref
f3, 2-D		 1e0
5.0		 1e-1
62		 1e-2
317		 1e-3
2690		 #succ
10/10
DEMO Tusar bbob-biobj 5.7 (12)2.8 (0.7)1.9 (1.0)1.9 (0.8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (2)1.00 (0.3)1.00 (0.4)1.00 (0.7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 277 (645)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 86 (57)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.3 (10)2.2 (3)2.6 (7)1.6 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 10 (10)4.0 (4)6.0 (5)7.3 (4)0/10
MO-DIRECT-Rank Wong bbob-biobj 7.7 (8)2.9 (1)5.5 (8)7.3 (13)0/10
NSGA-II-MATLAB Auger bbob-biobj 25 (43)10 (3)3.7 (1)5.7 (6)0/10
RANDOMSEARCH-100 Auger bbob-biobj 4580 (2e4)1143 (2372)6033 (9779)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 12 (58)140 (346)78 (315)53 (39)0/10
RANDOMSEARCH-5 Auger bbob-biobj 67 (165)88 (1)603 (2976)127 (27)0/10
RM-MEDA Auger bbob-biobj 124 (153)20 (6)5.7 (2)1.3 (0.3)1/10
SMS-EMOA-DE Auger bbob-biobj 90 (63)14 (2)4.1 (1)33 (56)0/10
SMS-EMOA-PM Auger bbob-biobj 61 (17)13 (3)14 (16)34 (56)0/10
UP-MO-CMA-ES Krause bbob-biobj 4.5 (11)6.5 (2)6.3 (3)5.1 (2)10/10

Δ HVref
f4, 2-D		 1e0
10		 1e-1
52		 1e-2
209		 1e-3
892		 #succ
10/10
DEMO Tusar bbob-biobj 3.4 (7)2.4 (4)2.4 (2)1.9 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.95 (2)0.99 (1)1 (0.6)1.00 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 23 (27)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.5 (3)1.6 (2)2.2 (2)1.4 (0.9)0/10
MO-DIRECT-ND Wong bbob-biobj 4.8 (15)4.1 (6)3.3 (2)5.3 (7)0/10
MO-DIRECT-Rank Wong bbob-biobj 3.8 (10)2.9 (3)3.0 (1)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 21 (20)11 (2)4.1 (1)3.9 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 569 (858)1411 (2995)4576 (5340)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.3 (0.8)16 (1)90 (411)65 (92)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.6 (4)15 (5)136 (328)301 (20)0/10
RM-MEDA Auger bbob-biobj 53 (36)25 (12)10 (2)3.4 (1)2/10
SMS-EMOA-DE Auger bbob-biobj 40 (41)17 (5)6.2 (2)152 (0.6)0/10
SMS-EMOA-PM Auger bbob-biobj 36 (13)17 (21)9.4 (14)81 (81)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.2 (1)3.7 (14)6.3 (7)8.1 (6)10/10

Δ HVref
f5, 2-D		 1e0
2.0		 1e-1
49		 1e-2
603		 1e-3
4709		 #succ
10/10
DEMO Tusar bbob-biobj 2.6 (2)2.6 (2)1.4 (0.9)3.2 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.80 (0.5)1.0 (0.9)1.1 (0.4)1.7 (4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 348 (316)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 136 (210)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.4 (4)1.6 (0.9)1.0 (0.8)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 4.3 (8)3.9 (1)2.3 (1)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 3.5 (6)3.0 (2)2.6 (3)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 54 (96)12 (2)2.5 (0.5)7.4 (9)0/10
RANDOMSEARCH-100 Auger bbob-biobj 581 (705)819 (832)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 0.90 (0.8)2.2 (5)12 (22)30 (19)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.1 (2)3.2 (2)12 (4)98 (9)0/10
RM-MEDA Auger bbob-biobj 220 (106)25 (3)3.1 (0.3)1.0 (0.1)0/10
SMS-EMOA-DE Auger bbob-biobj 158 (50)16 (5)2.4 (0.7)4.0 (7)0/10
SMS-EMOA-PM Auger bbob-biobj 139 (70)16 (10)8.3 (9)14 (10)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.8 (2)2.3 (2)3.8 (5)3.7 (0.7)10/10

Δ HVref
f6, 2-D		 1e0
3.0		 1e-1
38		 1e-2
221		 1e-3
2306		 #succ
10/10
DEMO Tusar bbob-biobj 3.4 (10)3.7 (5)2.4 (2)1.8 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (2)1.1 (0.9)1.6 (2)1.0 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 207 (263)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 114 (132)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.7 (4)0.99 (0.2)1.0 (0.5)1.4 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 9.3 (6)5.1 (6)3.4 (0.7)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 5.0 (8)3.4 (2)4.2 (3)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 48 (90)15 (6)4.3 (1)3.7 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 3983 (2e4)2544 (4958)4382 (3520)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.9 (15)6.6 (2)49 (106)43 (61)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.2 (4)17 (67)66 (139)147 (44)0/10
RM-MEDA Auger bbob-biobj 154 (110)28 (10)7.5 (2)1.4 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 138 (28)20 (3)5.5 (1)2.0 (0.4)0/10
SMS-EMOA-PM Auger bbob-biobj 87 (80)21 (20)11 (4)4.2 (2)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.4 (23)5.1 (9)6.0 (7)4.8 (2)10/10

Δ HVref
f7, 2-D		 1e0
6.0		 1e-1
102		 1e-2
1953		 1e-3
10949		 #succ
10/10
DEMO Tusar bbob-biobj 2.6 (4)2.6 (2)1.0 (0.6)1.8 (0.9)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.00 (0.8)3.4 (4)3.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 229 (486)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 118 (150)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.4 (4)3.5 (4)1.6 (1)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 8.9 (15)4.0 (4)1.2 (2)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 6.2 (10)3.8 (3)2.2 (1)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 35 (35)7.1 (3)1.4 (0.2)2.2 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 943 (604)3209 (1991)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.8 (4)8.7 (14)12 (13)53 (23)0/10
RANDOMSEARCH-5 Auger bbob-biobj 0.97 (0.8)12 (17)22 (22)78 (170)0/10
RM-MEDA Auger bbob-biobj 99 (56)14 (2)1.4 (0.5)1.0 (0.9)0/10
SMS-EMOA-DE Auger bbob-biobj 65 (29)10 (3)1.5 (2)1.1 (0.6)0/10
SMS-EMOA-PM Auger bbob-biobj 44 (14)11 (8)5.1 (4)3.7 (3)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.2 (8)4.3 (7)2.1 (1.0)1.7 (0.5)10/10

Δ HVref
f8, 2-D		 1e0
9.0		 1e-1
146		 1e-2
747		 1e-3
3019		 #succ
10/10
DEMO Tusar bbob-biobj 3.5 (10)2.0 (2)1.5 (0.5)1.9 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.99 (1)1.9 (4)1.00 (1)1.0 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 150 (295)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 72 (111)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.7 (1)1.00 (2)2.0 (2)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 5.8 (7)2.1 (2)2.1 (1)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 4.4 (4)2.3 (1)2.7 (2)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 30 (32)4.8 (2)1.9 (0.6)2.4 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 989 (2131)1776 (2454)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (0.2)6.9 (22)38 (26)114 (66)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (6)19 (61)71 (139)254 (214)0/10
RM-MEDA Auger bbob-biobj 71 (32)10 (4)3.1 (0.7)1.5 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 54 (15)6.0 (1)2.3 (0.5)1.7 (0.7)4/10
SMS-EMOA-PM Auger bbob-biobj 36 (22)6.8 (7)8.6 (4)13 (7)2/10
UP-MO-CMA-ES Krause bbob-biobj 1.0 (0.7)3.9 (5)4.5 (4)4.7 (2)10/10

Δ HVref
f9, 2-D		 1e0
7.0		 1e-1
42		 1e-2
161		 1e-3
963		 #succ
10/10
DEMO Tusar bbob-biobj 2.8 (2)5.0 (4)3.6 (1)2.3 (0.7)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)1 (0.4)1.3 (0.7)1 (0.7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 195 (257)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 97 (80)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.1 (2)0.99 (0.4)1.0 (0.5)1.2 (0.5)0/10
MO-DIRECT-ND Wong bbob-biobj 7.9 (9)4.6 (3)3.8 (2)4.5 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 6.5 (7)4.0 (3)4.3 (2)10 (8)0/10
NSGA-II-MATLAB Auger bbob-biobj 35 (44)15 (3)5.7 (0.9)6.4 (12)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1185 (1722)3070 (1896)1.2e4 (7453)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.9 (5)4.9 (6)36 (43)253 (262)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.0 (4)8.3 (8)54 (52)330 (420)0/10
RM-MEDA Auger bbob-biobj 94 (58)28 (8)11 (3)2.9 (0.7)2/10
SMS-EMOA-DE Auger bbob-biobj 72 (38)21 (8)8.1 (2)142 (105)1/10
SMS-EMOA-PM Auger bbob-biobj 49 (18)18 (15)11 (7)102 (91)1/10
UP-MO-CMA-ES Krause bbob-biobj 2.5 (4)8.1 (5)13 (9)9.0 (5)10/10

Δ HVref
f10, 2-D		 1e0
5.0		 1e-1
187		 1e-2
726		 1e-3
6110		 #succ
10/10
DEMO Tusar bbob-biobj 2.0 (3)1.7 (3)2.1 (3)1.0 (0.4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.98 (0.5)14 (34)14 (29)4.2 (4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 134 (150)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 125 (83)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (2)1.00 (1)1.00 (0.7)1 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 9.4 (13)1.2 (0.7)1.4 (0.9)3.2 (4)0/10
MO-DIRECT-Rank Wong bbob-biobj 8.8 (13)1.3 (0.8)1.3 (0.5)3.2 (3)0/10
NSGA-II-MATLAB Auger bbob-biobj 48 (40)3.5 (1.0)1.9 (0.4)1.3 (0.7)0/10
RANDOMSEARCH-100 Auger bbob-biobj 628 (948)685 (414)2623 (1791)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (2)2.2 (3)52 (30)149 (188)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (0.7)2.1 (2)18 (11)53 (38)0/10
RM-MEDA Auger bbob-biobj 116 (106)7.1 (0.9)33 (69)4.2 (16)1/10
SMS-EMOA-DE Auger bbob-biobj 75 (30)4.8 (2)2.3 (1)3.7 (1)2/10
SMS-EMOA-PM Auger bbob-biobj 59 (40)7.5 (14)6.3 (3)2.2 (2)1/10
UP-MO-CMA-ES Krause bbob-biobj 1.6 (2)2.0 (2)4.3 (4)2.1 (0.8)10/10

Δ HVref
f11, 2-D		 1e0
3.0		 1e-1
30		 1e-2
200		 1e-3
1038		 #succ
10/10
DEMO Tusar bbob-biobj 10 (24)2.4 (0.6)1.3 (1)1.1 (0.4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (0.8)1.0 (0.4)1.00 (0.5)1.00 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 13 (7)24 (20)∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 12 (19)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.2 (3)2.0 (2)3.1 (3)∞20030/10
MO-DIRECT-ND Wong bbob-biobj 11 (39)6.8 (8)15 (7)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 20 (83)14 (20)31 (30)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 26 (59)11 (7)2.7 (2)1.6 (0.4)5/10
RANDOMSEARCH-100 Auger bbob-biobj 198 (14)759 (3339)141 (259)181 (226)0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.97 (0.8)2.1 (4)13 (29)23 (97)4/10
RANDOMSEARCH-5 Auger bbob-biobj 3.3 (7)5.1 (23)21 (98)90 (435)9/10
RM-MEDA Auger bbob-biobj 26 (6)14 (13)4.0 (0.4)2.1 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 16 (14)12 (15)3.2 (0.6)1737 (1734)0/10
SMS-EMOA-PM Auger bbob-biobj 21 (59)10 (21)5.4 (7)9.3 (11)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.6 (4)4.3 (9)2.1 (0.3)2.5 (6)10/10

Δ HVref
f12, 2-D		 1e0
4.0		 1e-1
53		 1e-2
256		 1e-3
986		 #succ
1/10
DEMO Tusar bbob-biobj 1.6 (2)2.4 (3)1.9 (2)1.4 (1)4/10
HMO-CMA-ES Loshchilov bbob-biobj 1.3 (2)1.0 (1)1.00 (0.7)1.00 (0.5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 33 (34)28 (26)∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 94 (202)12 (14)∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.7 (2)5.1 (10)3.0 (1)2.5 (5)0/10
MO-DIRECT-ND Wong bbob-biobj 3.6 (6)3.0 (5)5.1 (6)5.3 (6)1/10
MO-DIRECT-Rank Wong bbob-biobj 3.9 (9)2.3 (3)5.8 (5)5.2 (4)0/10
NSGA-II-MATLAB Auger bbob-biobj 33 (67)9.1 (6)4.0 (4)10 (19)3/10
RANDOMSEARCH-100 Auger bbob-biobj 259 (416)530 (368)818 (1186)479 (456)0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.9 (0.6)431 (963)530 (586)474 (457)1/10
RANDOMSEARCH-5 Auger bbob-biobj 0.93 (2)4196 (2e4)3362 (3925)968 (2059)2/10
RM-MEDA Auger bbob-biobj 61 (45)15 (11)5.4 (4)2.4 (2)6/10
SMS-EMOA-DE Auger bbob-biobj 51 (56)13 (11)4.2 (3)136 (254)3/10
SMS-EMOA-PM Auger bbob-biobj 30 (33)11 (5)52 (4)221 (275)3/10
UP-MO-CMA-ES Krause bbob-biobj 2.9 (0.8)4.1 (3)6.1 (11)7.5 (5)10/10

Δ HVref
f13, 2-D		 1e0
4.0		 1e-1
37		 1e-2
315		 1e-3
756		 #succ
1/10
DEMO Tusar bbob-biobj 4.5 (8)5.0 (4)1.4 (2)2.7 (2)8/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (2)1.1 (4)1.00 (3)2.2 (4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 356 (289)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.0 (4)0.99 (1)1.4 (1)1.0 (3)1/10
MO-DIRECT-ND Wong bbob-biobj 11 (10)6.4 (6)2.0 (1)1.5 (1)0/10
MO-DIRECT-Rank Wong bbob-biobj 8.0 (13)2.8 (3)1.5 (0.3)1.2 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 51 (85)14 (6)25 (115)11 (26)8/10
RANDOMSEARCH-100 Auger bbob-biobj 885 (962)1372 (2830)348 (798)365 (314)0/10
RANDOMSEARCH-4 Auger bbob-biobj 12 (39)7.3 (17)17 (41)41 (133)6/10
RANDOMSEARCH-5 Auger bbob-biobj 13 (28)16 (3)21 (2)210 (522)8/10
RM-MEDA Auger bbob-biobj 137 (166)26 (20)4.1 (3)2.6 (1)9/10
SMS-EMOA-DE Auger bbob-biobj 141 (73)24 (9)161 (476)68 (198)2/10
SMS-EMOA-PM Auger bbob-biobj 79 (38)14 (2)62 (123)67 (0.4)5/10
UP-MO-CMA-ES Krause bbob-biobj 4.8 (14)8.2 (5)4.2 (2)5.0 (11)10/10

Δ HVref
f14, 2-D		 1e0
9.0		 1e-1
110		 1e-2
566		 1e-3
3600		 #succ
10/10
DEMO Tusar bbob-biobj 4.0 (0.3)2.1 (2)1.6 (0.5)2.2 (3)2/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)1.00 (1)1.0 (0.5)1.0 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 48 (34)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 160 (116)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.2 (1)1.2 (2)2.8 (2)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 4.4 (8)2.7 (2)2.7 (2)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 2.9 (2)2.1 (0.9)2.9 (3)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 20 (20)5.2 (3)2.4 (0.9)7.7 (7)0/10
RANDOMSEARCH-100 Auger bbob-biobj 748 (1777)1098 (1700)3515 (2915)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (2)7.9 (13)175 (177)240 (181)0/10
RANDOMSEARCH-5 Auger bbob-biobj 6.9 (32)3.6 (5)476 (953)948 (1415)0/10
RM-MEDA Auger bbob-biobj 59 (51)11 (3)3.6 (0.9)1.0 (0.3)6/10
SMS-EMOA-DE Auger bbob-biobj 47 (24)10 (3)43 (177)141 (125)0/10
SMS-EMOA-PM Auger bbob-biobj 30 (20)22 (56)128 (244)234 (446)0/10
UP-MO-CMA-ES Krause bbob-biobj 6.5 (26)2.1 (3)5.4 (4)3.9 (2)10/10

Δ HVref
f15, 2-D		 1e0
5.0		 1e-1
24		 1e-2
497		 1e-3
2385		 #succ
10/10
DEMO Tusar bbob-biobj 1.1 (0.9)5.8 (6)1 (0.7)1.00 (1)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0.5)0.99 (1.0)1.9 (0.5)1.3 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 132 (98)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 39 (182)9.0 (0.7)1.4 (2)1.3 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 27 (18)10 (13)2.0 (2)2.5 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 14 (28)6.2 (7)2.6 (2)7.8 (9)0/10
NSGA-II-MATLAB Auger bbob-biobj 35 (40)22 (7)1.7 (0.6)1.5 (1)3/10
RANDOMSEARCH-100 Auger bbob-biobj 2259 (5191)3785 (9181)1034 (1234)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (5)4.1 (2)54 (115)67 (92)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.5 (4)11 (17)459 (1023)175 (418)1/10
RM-MEDA Auger bbob-biobj 97 (83)40 (24)3.5 (1)1.2 (0.4)5/10
SMS-EMOA-DE Auger bbob-biobj 83 (37)32 (15)47 (302)196 (377)0/10
SMS-EMOA-PM Auger bbob-biobj 61 (14)36 (29)54 (204)67 (78)1/10
UP-MO-CMA-ES Krause bbob-biobj 2.5 (2)6.6 (7)2.7 (2)3.2 (2)10/10

Δ HVref
f16, 2-D		 1e0
9.0		 1e-1
181		 1e-2
1690		 1e-3
6995		 #succ
10/10
DEMO Tusar bbob-biobj 16 (34)124 (1106)14 (31)4.7 (9)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (3)4.8 (11)4.1 (7)4.2 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 74 (77)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 11 (0.8)1.0 (2)1.0 (0.7)2.8 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 14 (38)2.0 (2)1.9 (2)2.7 (4)0/10
MO-DIRECT-Rank Wong bbob-biobj 9.0 (12)1.6 (1.0)1.0 (0.9)2.8 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 24 (17)10 (18)1.6 (2)1.6 (3)3/10
RANDOMSEARCH-100 Auger bbob-biobj 3459 (7641)912 (859)1070 (917)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 26 (2)28 (8)40 (92)119 (180)0/10
RANDOMSEARCH-5 Auger bbob-biobj 37 (170)91 (6)137 (4)140 (139)0/10
RM-MEDA Auger bbob-biobj 61 (43)7.3 (4)1.7 (1)1.0 (0.5)6/10
SMS-EMOA-DE Auger bbob-biobj 55 (88)6.6 (4)31 (60)44 (93)0/10
SMS-EMOA-PM Auger bbob-biobj 39 (50)4.9 (4)12 (26)53 (84)0/10
UP-MO-CMA-ES Krause bbob-biobj 10 (24)3.1 (5)3.0 (4)34 (1)8/10

Δ HVref
f17, 2-D		 1e0
13		 1e-1
200		 1e-2
509		 1e-3
1980		 #succ
10/10
DEMO Tusar bbob-biobj 2.1 (0.9)1.3 (1)1.8 (1)1.8 (2)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (0.3)1.0 (2)1.00 (0.9)1.00 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 31 (25)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.4 (1.0)2.3 (5)2.4 (3)3.0 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 5.6 (5)1.8 (1)3.5 (4)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 3.6 (3)1.1 (1.0)2.5 (1)10 (18)0/10
NSGA-II-MATLAB Auger bbob-biobj 30 (22)3.7 (1)2.8 (2)2.5 (2)4/10
RANDOMSEARCH-100 Auger bbob-biobj 927 (2368)1053 (1067)3847 (3143)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.3 (0.6)7.1 (14)36 (31)97 (240)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.8 (3)3.6 (5)39 (42)263 (386)0/10
RM-MEDA Auger bbob-biobj 51 (18)6.7 (3)4.7 (2)2.2 (0.4)8/10
SMS-EMOA-DE Auger bbob-biobj 38 (23)5.1 (3)3.8 (0.9)13 (26)1/10
SMS-EMOA-PM Auger bbob-biobj 21 (10)4.4 (5)53 (119)89 (76)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.4 (4)2.0 (2)5.5 (3)5.9 (4)10/10

Δ HVref
f18, 2-D		 1e0
6.0		 1e-1
45		 1e-2
474		 1e-3
1086		 #succ
10/10
DEMO Tusar bbob-biobj 2.9 (0.6)2.7 (11)1.1 (3)1.00 (0.9)3/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)1 (2)8.8 (43)4.7 (11)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 246 (175)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 248 (106)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.3 (2)8.0 (22)1.00 (1)1.5 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 12 (5)4.6 (6)1.4 (2)3.3 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 8.3 (15)3.8 (2)1.4 (0.4)4.6 (9)0/10
NSGA-II-MATLAB Auger bbob-biobj 35 (32)12 (6)1.7 (1)1.6 (2)5/10
RANDOMSEARCH-100 Auger bbob-biobj 1950 (5184)1490 (2028)317 (374)538 (581)0/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (4)9.3 (42)89 (92)192 (509)2/10
RANDOMSEARCH-5 Auger bbob-biobj 2.4 (5)41 (193)70 (3)803 (932)3/10
RM-MEDA Auger bbob-biobj 93 (74)21 (9)3.2 (2)2.2 (0.6)8/10
SMS-EMOA-DE Auger bbob-biobj 68 (47)18 (4)49 (0.6)80 (323)2/10
SMS-EMOA-PM Auger bbob-biobj 46 (36)15 (8)2.6 (0.6)41 (42)3/10
UP-MO-CMA-ES Krause bbob-biobj 1.7 (2)4.1 (9)2.1 (4)4.2 (9)10/10

Δ HVref
f19, 2-D		 1e0
9.0		 1e-1
211		 1e-2
926		 1e-3
2276		 #succ
1/10
DEMO Tusar bbob-biobj 4.8 (11)1.1 (0.6)1.1 (0.3)1.0 (0.5)3/10
HMO-CMA-ES Loshchilov bbob-biobj 0.98 (0.5)3.2 (6)15 (7)10 (11)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 161 (229)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.0 (1)2.0 (1)1.7 (0.4)1.9 (3)1/10
MO-DIRECT-ND Wong bbob-biobj 6.3 (9)1.3 (1)1.4 (0.2)4.0 (6)0/10
MO-DIRECT-Rank Wong bbob-biobj 5.3 (4)1.00 (0.4)1.00 (0.5)2.6 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 30 (19)6.0 (7)7.4 (6)3.5 (4)4/10
RANDOMSEARCH-100 Auger bbob-biobj 869 (417)1345 (2057)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 0.96 (1)2.3 (6)29 (5)47 (64)0/10
RANDOMSEARCH-5 Auger bbob-biobj 8.2 (19)2.3 (4)4.3 (4)29 (10)1/10
RM-MEDA Auger bbob-biobj 60 (33)6.5 (3)56 (108)23 (44)5/10
SMS-EMOA-DE Auger bbob-biobj 50 (37)5.1 (1)56 (55)89 (308)2/10
SMS-EMOA-PM Auger bbob-biobj 35 (11)4.4 (2)37 (108)73 (84)2/10
UP-MO-CMA-ES Krause bbob-biobj 1.9 (0.5)1.3 (1)2.0 (1)4.5 (4)10/10

Δ HVref
f20, 2-D		 1e0
4.0		 1e-1
147		 1e-2
391		 1e-3
2044		 #succ
10/10
DEMO Tusar bbob-biobj 3.2 (6)1.9 (3)1.6 (1)1.5 (0.9)3/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0.3)1.00 (2)1.0 (0.4)1.00 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 95 (134)10 (13)∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 100 (114)9.3 (10)∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.1 (3)1.5 (4)1.4 (2)2.0 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 21 (49)2.1 (2)2.7 (3)4.4 (4)0/10
MO-DIRECT-Rank Wong bbob-biobj 17 (31)1.2 (1)2.5 (2)4.1 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 70 (107)4.1 (2)2.7 (2)5.8 (6)3/10
RANDOMSEARCH-100 Auger bbob-biobj 3799 (1e4)534 (737)1208 (2558)419 (563)0/10
RANDOMSEARCH-4 Auger bbob-biobj 2.7 (4)160 (357)234 (266)157 (298)0/10
RANDOMSEARCH-5 Auger bbob-biobj 18 (36)1519 (6805)735 (17)368 (432)0/10
RM-MEDA Auger bbob-biobj 121 (134)6.8 (6)4.3 (3)1.5 (0.9)3/10
SMS-EMOA-DE Auger bbob-biobj 97 (95)5.0 (2)3.3 (0.9)43 (75)1/10
SMS-EMOA-PM Auger bbob-biobj 76 (117)5.2 (9)8.6 (9)72 (101)0/10
UP-MO-CMA-ES Krause bbob-biobj 8.0 (10)3.1 (6)5.3 (7)4.6 (4)10/10

Δ HVref
f21, 2-D		 1e0
8.0		 1e-1
72		 1e-2
237		 1e-3
878		 #succ
10/10
DEMO Tusar bbob-biobj 6.8 (11)2.8 (4)3.4 (2)2.7 (3)3/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.6)1.0 (2)1.0 (0.8)1 (0.7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 55 (80)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 20 (46)2.7 (5)1.9 (3)2.9 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 10 (13)3.5 (4)3.2 (2)4.6 (6)0/10
MO-DIRECT-Rank Wong bbob-biobj 9.1 (10)2.5 (2)3.2 (3)6.2 (4)0/10
NSGA-II-MATLAB Auger bbob-biobj 46 (58)43 (87)16 (4)12 (7)3/10
RANDOMSEARCH-100 Auger bbob-biobj 1832 (5253)1463 (2527)2116 (2555)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.4 (3)12 (29)147 (263)201 (321)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.9 (3)6.8 (10)105 (216)1139 (1804)2/10
RM-MEDA Auger bbob-biobj 93 (122)18 (13)9.1 (3)7.7 (22)5/10
SMS-EMOA-DE Auger bbob-biobj 61 (44)13 (8)7.6 (9)60 (2)0/10
SMS-EMOA-PM Auger bbob-biobj 56 (72)18 (22)30 (95)73 (174)0/10
UP-MO-CMA-ES Krause bbob-biobj 7.3 (8)5.5 (6)7.6 (10)10 (10)10/10

Δ HVref
f22, 2-D		 1e0
4.0		 1e-1
68		 1e-2
620		 1e-3
3965		 #succ
10/10
DEMO Tusar bbob-biobj 3.1 (5)3.2 (3)1.5 (0.7)3.5 (8)1/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (2)1.0 (0.5)1.0 (0.7)1.4 (0.7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 340 (494)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.5 (5)3.2 (3)2.2 (2)4.6 (5)0/10
MO-DIRECT-ND Wong bbob-biobj 6.6 (5)4.7 (3)3.4 (3)4.9 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 5.3 (7)3.9 (2)4.5 (3)5.0 (6)0/10
NSGA-II-MATLAB Auger bbob-biobj 38 (60)10 (2)3.0 (2)8.3 (10)0/10
RANDOMSEARCH-100 Auger bbob-biobj 923 (2358)2105 (2423)3148 (4435)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.6 (1)4.9 (8)30 (26)223 (297)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (3)8.1 (10)37 (41)553 (405)0/10
RM-MEDA Auger bbob-biobj 83 (117)17 (6)3.0 (1.0)1.0 (0.4)2/10
SMS-EMOA-DE Auger bbob-biobj 87 (46)13 (4)38 (0.9)212 (265)0/10
SMS-EMOA-PM Auger bbob-biobj 73 (49)15 (5)58 (76)458 (555)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.5 (2)4.0 (5)5.4 (4)4.4 (1)10/10

Δ HVref
f23, 2-D		 1e0
1		 1e-1
19		 1e-2
183		 1e-3
1196		 #succ
10/10
DEMO Tusar bbob-biobj 3.4 (3)1.8 (1)2.1 (1)2.4 (4)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (0.5)0.98 (0.4)1.00 (0.8)1.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 122 (142)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 160 (100)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (1)3.1 (0.9)1.8 (1.0)1.4 (0.8)0/10
MO-DIRECT-ND Wong bbob-biobj 5.4 (1)3.1 (2)2.5 (2)2.8 (4)0/10
MO-DIRECT-Rank Wong bbob-biobj 5.3 (2)4.1 (3)2.9 (3)5.0 (5)0/10
NSGA-II-MATLAB Auger bbob-biobj 26 (122)21 (12)4.6 (1)4.1 (2)1/10
RANDOMSEARCH-100 Auger bbob-biobj 770 (1605)856 (255)756 (879)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (1)2.0 (3)736 (2186)296 (270)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.4 (2)2.4 (3)4692 (1e4)1165 (1733)0/10
RM-MEDA Auger bbob-biobj 135 (186)24 (19)6.3 (2)2.0 (0.6)3/10
SMS-EMOA-DE Auger bbob-biobj 146 (230)27 (18)6.1 (2)194 (197)0/10
SMS-EMOA-PM Auger bbob-biobj 78 (74)25 (14)14 (20)73 (108)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.1 (2)1.6 (2)4.5 (7)5.5 (3)10/10

Δ HVref
f24, 2-D		 1e0
3.0		 1e-1
343		 1e-2
3334		 1e-3
19378		 #succ
10/10
DEMO Tusar bbob-biobj 3.9 (1)1.2 (0.8)1.0 (0.5)1.4 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1 (2)3.6 (7)2.6 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 454 (625)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 29 (131)1.2 (1)1.8 (1)1.0 (0.9)0/10
MO-DIRECT-ND Wong bbob-biobj 15 (25)1.5 (1)2.8 (2)∞20030/10
MO-DIRECT-Rank Wong bbob-biobj 12 (9)1.4 (0.6)1.8 (3)1.0 (1)0/10
NSGA-II-MATLAB Auger bbob-biobj 64 (79)2.6 (0.7)1.8 (2)1.5 (2)1/10
RANDOMSEARCH-100 Auger bbob-biobj 1010 (1384)1177 (1292)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.4 (6)7.2 (12)33 (79)46 (40)0/10
RANDOMSEARCH-5 Auger bbob-biobj 4.6 (4)7.2 (15)20 (31)85 (134)0/10
RM-MEDA Auger bbob-biobj 207 (122)4.1 (1)1.0 (0.3)1.8 (3)2/10
SMS-EMOA-DE Auger bbob-biobj 151 (30)3.3 (0.7)16 (46)19 (20)1/10
SMS-EMOA-PM Auger bbob-biobj 132 (105)6.8 (2)13 (16)44 (41)1/10
UP-MO-CMA-ES Krause bbob-biobj 3.1 (4)1.8 (2)2.5 (2)1.0 (0.2)8/10

Δ HVref
f25, 2-D		 1e0
4.0		 1e-1
166		 1e-2
975		 1e-3
4432		 #succ
10/10
DEMO Tusar bbob-biobj 1.3 (2)1.0 (2)1.0 (0.5)1.1 (0.5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (0.4)3.1 (7)3.5 (6)2.3 (5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.5 (8)1.4 (2)2.3 (1)4.5 (7)0/10
MO-DIRECT-ND Wong bbob-biobj 8.4 (14)2.0 (1)4.2 (2)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 8.2 (12)1.5 (1)2.2 (0.8)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 42 (55)3.9 (1)1.3 (0.4)1.0 (0.9)1/10
RANDOMSEARCH-100 Auger bbob-biobj 2648 (418)467 (1084)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.3 (2)2.4 (8)31 (56)31 (34)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.9 (4)2.0 (3)17 (31)83 (199)0/10
RM-MEDA Auger bbob-biobj 119 (58)7.1 (2)2.1 (0.3)1.0 (0.2)6/10
SMS-EMOA-DE Auger bbob-biobj 112 (41)5.6 (2)1.7 (0.6)5.9 (12)1/10
SMS-EMOA-PM Auger bbob-biobj 76 (23)4.5 (1)3.6 (3)13 (19)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.4 (12)1.8 (0.5)3.1 (2)3.1 (1)8/10

Δ HVref
f26, 2-D		 1e0
4.0		 1e-1
213		 1e-2
805		 1e-3
1889		 #succ
1/10
DEMO Tusar bbob-biobj 3.4 (2)1.0 (2)29 (63)27 (27)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.6)1.00 (2)14 (35)12 (17)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 356 (403)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.5 (2)2.2 (2)1.1 (1)1.0 (0.7)1/10
MO-DIRECT-ND Wong bbob-biobj 12 (13)1.7 (3)1.2 (1)2.8 (5)1/10
MO-DIRECT-Rank Wong bbob-biobj 11 (5)1.2 (1)1.00 (0.9)2.9 (2)1/10
NSGA-II-MATLAB Auger bbob-biobj 82 (62)3.4 (1)6.7 (9)6.8 (7)1/10
RANDOMSEARCH-100 Auger bbob-biobj 3661 (5920)469 (689)665 (1542)956 (1350)0/10
RANDOMSEARCH-4 Auger bbob-biobj 3.9 (11)1.5 (2)9.1 (6)94 (117)1/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (3)1.4 (2)16 (48)161 (177)1/10
RM-MEDA Auger bbob-biobj 164 (49)5.3 (3)2.3 (1)13 (0.7)4/10
SMS-EMOA-DE Auger bbob-biobj 111 (60)3.7 (2)1.9 (1)28 (27)2/10
SMS-EMOA-PM Auger bbob-biobj 81 (40)3.6 (3)5.0 (0.8)15 (35)1/10
UP-MO-CMA-ES Krause bbob-biobj 4.3 (5)2.0 (4)3.8 (6)5.3 (5)10/10

Δ HVref
f27, 2-D		 1e0
22		 1e-1
293		 1e-2
874		 1e-3
4818		 #succ
10/10
DEMO Tusar bbob-biobj 2.4 (11)1.0 (1)1.1 (0.9)1.0 (0.5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)7.1 (15)16 (16)4.9 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 33 (33)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.7 (12)1.5 (2)2.3 (3)1.2 (0.8)0/10
MO-DIRECT-ND Wong bbob-biobj 3.5 (8)1.7 (0.5)2.0 (1)∞20030/10
MO-DIRECT-Rank Wong bbob-biobj 2.7 (0.9)1.1 (0.6)1.0 (0.1)4.0 (6)0/10
NSGA-II-MATLAB Auger bbob-biobj 10 (10)3.0 (2)4.8 (8)1.7 (0.4)2/10
RANDOMSEARCH-100 Auger bbob-biobj 1076 (4578)465 (740)2203 (2346)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 6.2 (0.3)14 (0.9)159 (229)76 (53)0/10
RANDOMSEARCH-5 Auger bbob-biobj 0.99 (0.9)14 (68)85 (5)63 (14)0/10
RM-MEDA Auger bbob-biobj 26 (29)4.8 (2)2.4 (0.5)11 (21)2/10
SMS-EMOA-DE Auger bbob-biobj 21 (16)3.5 (2)2.3 (0.6)5.4 (0.1)0/10
SMS-EMOA-PM Auger bbob-biobj 33 (112)9.0 (17)8.0 (4)11 (18)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.4 (15)1.2 (3)2.4 (2)2.1 (1)10/10

Δ HVref
f28, 2-D		 1e0
3.0		 1e-1
26		 1e-2
269		 1e-3
1200		 #succ
1/10
DEMO Tusar bbob-biobj 2.6 (1)2.8 (2)1.6 (2)1.9 (2)6/10
HMO-CMA-ES Loshchilov bbob-biobj 2.1 (2)1.1 (2)1.2 (2)2.1 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 456 (350)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 231 (217)55 (26)∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.5 (5)0.98 (0.6)2.4 (6)1.6 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 4.8 (8)2.4 (4)1.4 (2)1.5 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 4.7 (11)2.3 (5)1.0 (1.0)1.00 (1)1/10
NSGA-II-MATLAB Auger bbob-biobj 65 (68)20 (7)6.6 (2)5.4 (6)6/10
RANDOMSEARCH-100 Auger bbob-biobj 470 (501)545 (445)619 (938)705 (531)0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (2)3.5 (7)86 (377)57 (100)3/10
RANDOMSEARCH-5 Auger bbob-biobj 1.6 (3)3.1 (2)829 (5579)434 (1680)4/10
RM-MEDA Auger bbob-biobj 142 (105)27 (15)5.6 (4)2.2 (1)7/10
SMS-EMOA-DE Auger bbob-biobj 124 (54)25 (12)4.3 (1)73 (125)1/10
SMS-EMOA-PM Auger bbob-biobj 112 (56)24 (13)7.7 (8)10 (12)2/10
UP-MO-CMA-ES Krause bbob-biobj 2.2 (2)3.9 (3)2.6 (5)3.3 (4)10/10

Δ HVref
f29, 2-D		 1e0
4.0		 1e-1
149		 1e-2
754		 1e-3
2848		 #succ
10/10
DEMO Tusar bbob-biobj 4.9 (7)1.4 (2)1.2 (0.3)2.7 (0.8)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (0.4)1.00 (1.0)1.0 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 352 (188)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 354 (262)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.1 (2)2.5 (5)1.6 (2)6.9 (5)0/10
MO-DIRECT-ND Wong bbob-biobj 7.8 (14)2.2 (3)1.8 (2)6.9 (14)0/10
MO-DIRECT-Rank Wong bbob-biobj 4.2 (7)1.4 (0.9)1.4 (1)6.8 (6)0/10
NSGA-II-MATLAB Auger bbob-biobj 59 (64)4.5 (3)2.3 (4)6.6 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 753 (270)607 (409)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.6 (10)1.2 (2)25 (55)319 (378)0/10
RANDOMSEARCH-5 Auger bbob-biobj 5.2 (11)4.0 (9)52 (189)427 (668)0/10
RM-MEDA Auger bbob-biobj 157 (51)10 (4)3.1 (0.5)1.3 (0.7)5/10
SMS-EMOA-DE Auger bbob-biobj 117 (60)7.1 (2)2.3 (0.6)21 (71)1/10
SMS-EMOA-PM Auger bbob-biobj 126 (143)12 (17)37 (70)62 (41)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.4 (10)2.6 (1)3.7 (4)4.3 (2)10/10

Δ HVref
f30, 2-D		 1e0
10		 1e-1
67		 1e-2
224		 1e-3
1596		 #succ
10/10
DEMO Tusar bbob-biobj 2.5 (9)2.9 (0.8)2.4 (3)1.9 (1)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (6)1.00 (0.1)1.00 (3)1.00 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 142 (206)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (1)2.0 (4)1.9 (3)1.1 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 3.4 (10)4.3 (12)3.5 (4)2.5 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 2.4 (5)2.4 (0.2)3.5 (6)2.6 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 20 (30)8.3 (6)4.7 (6)3.5 (3)2/10
RANDOMSEARCH-100 Auger bbob-biobj 1009 (4623)967 (1590)1695 (1370)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 18 (0.2)34 (4)119 (446)50 (40)0/10
RANDOMSEARCH-5 Auger bbob-biobj 29 (144)155 (386)682 (1694)263 (1159)0/10
RM-MEDA Auger bbob-biobj 66 (63)16 (17)8.5 (2)2.0 (1)3/10
SMS-EMOA-DE Auger bbob-biobj 44 (12)12 (8)5.9 (2)91 (95)0/10
SMS-EMOA-PM Auger bbob-biobj 24 (24)11 (6)23 (9)54 (45)0/10
UP-MO-CMA-ES Krause bbob-biobj 11 (0.6)8.2 (31)7.8 (17)4.6 (2)10/10

Δ HVref
f31, 2-D		 1e0
4.0		 1e-1
173		 1e-2
1349		 1e-3
7267		 #succ
10/10
DEMO Tusar bbob-biobj 2.8 (6)2.0 (2)18 (38)9.0 (28)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.00 (2)2.3 (1)4.1 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 339 (478)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 343 (225)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (3)2.0 (4)1.00 (0.7)1.3 (0.4)0/10
MO-DIRECT-ND Wong bbob-biobj 8.0 (9)1.9 (2)2.4 (2)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 5.5 (9)1.3 (0.8)1.1 (1.0)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 52 (65)4.2 (2)6.7 (9)2.3 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 671 (1077)1467 (2632)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (0.8)2.4 (3)43 (80)55 (48)0/10
RANDOMSEARCH-5 Auger bbob-biobj 0.88 (0.5)5.3 (5)42 (40)83 (146)0/10
RM-MEDA Auger bbob-biobj 129 (56)8.3 (4)2.3 (0.6)1.00 (0.2)2/10
SMS-EMOA-DE Auger bbob-biobj 84 (62)5.7 (3)2.1 (1)4.5 (1.0)5/10
SMS-EMOA-PM Auger bbob-biobj 82 (21)7.0 (5)21 (41)25 (51)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.5 (8)2.6 (5)3.6 (3)2.2 (0.7)9/10

Δ HVref
f32, 2-D		 1e0
8.0		 1e-1
176		 1e-2
1116		 1e-3
4723		 #succ
10/10
DEMO Tusar bbob-biobj 2.7 (4)2.1 (2)1.2 (0.5)1.3 (0.8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (1)1.9 (0.6)1.00 (0.3)1 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 173 (230)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.0 (2)1.0 (0.4)1.7 (4)2.0 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 13 (11)2.6 (1)2.9 (6)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 7.2 (7)1.7 (1)1.7 (1)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 43 (30)4.8 (1)2.5 (4)1.8 (1.0)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1640 (5036)2074 (2979)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.7 (3)10 (2)33 (30)121 (190)0/10
RANDOMSEARCH-5 Auger bbob-biobj 5.2 (13)18 (20)64 (27)261 (198)0/10
RM-MEDA Auger bbob-biobj 90 (102)11 (5)2.7 (0.6)1.3 (0.2)5/10
SMS-EMOA-DE Auger bbob-biobj 63 (27)6.5 (2)1.9 (0.4)5.8 (0.4)4/10
SMS-EMOA-PM Auger bbob-biobj 52 (14)7.0 (3)13 (3)11 (13)0/10
UP-MO-CMA-ES Krause bbob-biobj 8.2 (2)4.3 (4)4.3 (4)3.3 (1)10/10

Δ HVref
f33, 2-D		 1e0
9.0		 1e-1
294		 1e-2
822		 1e-3
1842		 #succ
1/10
DEMO Tusar bbob-biobj 3.6 (7)2.4 (8)1.8 (1)1.6 (1)5/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (0.4)15 (4)19 (31)8.7 (14)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 164 (300)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 155 (125)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.8 (1)1.2 (3)1.4 (2)1.0 (2)1/10
MO-DIRECT-ND Wong bbob-biobj 7.2 (8)2.1 (3)1.4 (2)1.2 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 4.6 (4)1.0 (0.9)1.0 (2)1.1 (0.5)1/10
NSGA-II-MATLAB Auger bbob-biobj 29 (29)6.0 (20)2.8 (5)7.0 (12)5/10
RANDOMSEARCH-100 Auger bbob-biobj 1544 (2601)418 (459)306 (170)463 (1004)0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (2)6.1 (8)37 (75)54 (140)2/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (2)3.5 (8)68 (157)385 (388)3/10
RM-MEDA Auger bbob-biobj 78 (94)5.0 (3)29 (123)14 (2)5/10
SMS-EMOA-DE Auger bbob-biobj 58 (28)3.4 (2)29 (1)13 (110)2/10
SMS-EMOA-PM Auger bbob-biobj 43 (26)3.0 (3)20 (7)32 (54)2/10
UP-MO-CMA-ES Krause bbob-biobj 1.1 (1)2.2 (0.8)3.4 (6)4.2 (5)10/10

Δ HVref
f34, 2-D		 1e0
5.0		 1e-1
168		 1e-2
494		 1e-3
1885		 #succ
10/10
DEMO Tusar bbob-biobj 1.3 (1)1.0 (1.0)1.3 (0.4)1.5 (1)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (3)1.8 (4)22 (40)14 (13)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 60 (59)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.6 (2)1.8 (0.5)1.4 (0.9)1.0 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 3.0 (4)1.3 (0.9)1.5 (0.9)1.7 (0.8)0/10
MO-DIRECT-Rank Wong bbob-biobj 2.9 (5)1.00 (0.6)1.0 (0.8)1.0 (1)0/10
NSGA-II-MATLAB Auger bbob-biobj 43 (46)4.5 (1)11 (22)12 (3)2/10
RANDOMSEARCH-100 Auger bbob-biobj 417 (360)547 (792)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (2)133 (1)110 (204)57 (85)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.0 (3)1.5 (1)4.4 (3)19 (12)0/10
RM-MEDA Auger bbob-biobj 111 (43)10 (5)50 (203)14 (53)5/10
SMS-EMOA-DE Auger bbob-biobj 98 (19)6.0 (2)3.6 (1)1.9 (0.6)5/10
SMS-EMOA-PM Auger bbob-biobj 52 (22)6.7 (7)8.1 (7)6.8 (9)2/10
UP-MO-CMA-ES Krause bbob-biobj 2.5 (4)1.1 (2)2.9 (2)4.1 (3)9/10

Δ HVref
f35, 2-D		 1e0
2.0		 1e-1
89		 1e-2
789		 1e-3
4587		 #succ
10/10
DEMO Tusar bbob-biobj 6.8 (3)2.1 (3)1.3 (1)4.1 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.85 (0)1.00 (2)1.0 (0.7)2.8 (3)9/10
MAT-DIRECT Al-Dujaili bbob-biobj 122 (184)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 146 (190)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.4 (6)1.0 (2)1.1 (1)∞20030/10
MO-DIRECT-ND Wong bbob-biobj 7.8 (18)2.5 (2)1.9 (1)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 6.5 (15)2.2 (1)2.9 (3)∞20030/10
NSGA-II-MATLAB Auger bbob-biobj 59 (153)7.0 (3)2.6 (2)13 (13)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2411 (5679)1426 (2414)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 11 (2)3.0 (1)66 (190)79 (76)0/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (45)13 (0.9)57 (93)497 (218)0/10
RM-MEDA Auger bbob-biobj 212 (260)13 (4)2.5 (0.6)1.0 (0.5)3/10
SMS-EMOA-DE Auger bbob-biobj 184 (104)10 (3)2.1 (0.7)25 (24)1/10
SMS-EMOA-PM Auger bbob-biobj 136 (88)12 (11)10 (15)64 (43)0/10
UP-MO-CMA-ES Krause bbob-biobj 19 (46)4.6 (6)4.1 (3)4.0 (1)10/10

Δ HVref
f36, 2-D		 1e0
4.0		 1e-1
25		 1e-2
1137		 1e-3
6309		 #succ
10/10
DEMO Tusar bbob-biobj 2.2 (2)6.1 (8)1.2 (0.9)2.4 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.88 (1)0.98 (0.8)1.2 (0.3)1.4 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 167 (209)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (4)1.7 (0.9)1.1 (1)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 10 (14)7.5 (8)1.00 (0.3)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 7.8 (22)6.0 (3)1.4 (1)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 24 (66)22 (13)1.1 (0.5)3.6 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 622 (1453)4604 (4315)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.9 (7)20 (67)11 (18)35 (19)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.9 (2)18 (37)17 (35)119 (174)0/10
RM-MEDA Auger bbob-biobj 97 (74)41 (18)1.5 (0.4)1.0 (1.0)0/10
SMS-EMOA-DE Auger bbob-biobj 83 (35)31 (9)1.7 (0.4)4.8 (7)1/10
SMS-EMOA-PM Auger bbob-biobj 70 (30)32 (19)5.0 (7)14 (6)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.9 (2)13 (27)1.8 (2)2.4 (0.7)10/10

Δ HVref
f37, 2-D		 1e0
7.0		 1e-1
384		 1e-2
2779		 1e-3
21596		 #succ
10/10
DEMO Tusar bbob-biobj 2.3 (3)1.2 (0.7)1.00 (0.3)2.2 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (3)4.7 (8)2.2 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 196 (177)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 95 (93)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.8 (2)1.7 (2)1.0 (0.9)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 7.4 (8)1.8 (3)2.2 (3)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 5.4 (9)1.3 (0.8)1.7 (1)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 29 (48)2.1 (0.6)1.2 (0.4)1.5 (1)0/10
RANDOMSEARCH-100 Auger bbob-biobj 253 (428)1284 (2349)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.2 (7)6.4 (19)32 (56)89 (93)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.6 (7)11 (12)41 (11)183 (117)0/10
RM-MEDA Auger bbob-biobj 81 (38)3.9 (2)1.2 (0.5)1.6 (7)0/10
SMS-EMOA-DE Auger bbob-biobj 56 (21)3.2 (0.6)1.4 (0.5)3.2 (3)0/10
SMS-EMOA-PM Auger bbob-biobj 38 (42)3.6 (2)8.0 (9)26 (38)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.5 (3)2.1 (3)3.0 (2)1.00 (0.1)10/10

Δ HVref
f38, 2-D		 1e0
2.0		 1e-1
180		 1e-2
1395		 1e-3
6364		 #succ
10/10
DEMO Tusar bbob-biobj 2.0 (1)1.0 (0.7)1.00 (0.3)2.5 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.7 (3)4.7 (8)2.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 674 (712)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 134 (132)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (5)1.3 (0.8)1.6 (0.6)∞20030/10
MO-DIRECT-ND Wong bbob-biobj 12 (19)1.5 (0.5)1.6 (0.4)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 7.0 (14)1.5 (0.8)1.4 (1)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 74 (79)3.7 (0.7)1.1 (0.2)2.9 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 664 (579)583 (1189)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.6 (2)1.5 (2)4.2 (3)28 (19)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.9 (11)2.5 (3)6.7 (8)47 (73)0/10
RM-MEDA Auger bbob-biobj 194 (116)6.8 (1)1.7 (0.4)1 (0.7)0/10
SMS-EMOA-DE Auger bbob-biobj 143 (59)5.1 (1)1.5 (0.2)2.3 (1)0/10
SMS-EMOA-PM Auger bbob-biobj 110 (46)4.9 (3)3.6 (3)5.6 (4)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.1 (0.5)1.6 (2)2.4 (0.9)2.6 (0.7)9/10

Δ HVref
f39, 2-D		 1e0
4.0		 1e-1
239		 1e-2
1103		 1e-3
3676		 #succ
10/10
DEMO Tusar bbob-biobj 3.8 (4)1.4 (1)1.0 (1.0)2.2 (3)2/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.8)18 (42)10 (26)4.3 (8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 368 (346)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.8 (3)1 (1)1.0 (1)2.5 (4)0/10
MO-DIRECT-ND Wong bbob-biobj 7.7 (8)1.3 (1)1.3 (2)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 6.2 (9)1.3 (2)1.2 (2)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 50 (67)2.9 (0.9)2.3 (6)4.6 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1117 (1431)998 (1396)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.5 (4)3.7 (2)32 (54)145 (177)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (3)10 (2)54 (120)638 (1090)0/10
RM-MEDA Auger bbob-biobj 138 (76)5.5 (2)2.0 (0.5)1.00 (0.7)3/10
SMS-EMOA-DE Auger bbob-biobj 120 (51)4.0 (1)1.6 (0.5)16 (27)1/10
SMS-EMOA-PM Auger bbob-biobj 77 (29)10 (4)22 (22)44 (53)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.4 (1)2.5 (3)3.5 (4)3.8 (2)10/10

Δ HVref
f40, 2-D		 1e0
2.0		 1e-1
300		 1e-2
1003		 1e-3
7955		 #succ
10/10
DEMO Tusar bbob-biobj 5.2 (5)1.3 (0.7)1.9 (0.9)1.5 (2)1/10
HMO-CMA-ES Loshchilov bbob-biobj 5.2 (14)5.2 (14)30 (20)5.7 (1)9/10
MAT-DIRECT Al-Dujaili bbob-biobj 738 (712)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 676 (878)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.7 (6)2.0 (2)2.2 (1)2.5 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 16 (17)1.2 (0.7)2.2 (1)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 18 (21)1.00 (0.4)1 (0.3)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 93 (131)2.7 (0.4)8.8 (11)5.2 (1)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1181 (2864)510 (711)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.2 (4)1.3 (1)37 (62)57 (57)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (0.6)1.8 (1.0)22 (23)81 (290)0/10
RM-MEDA Auger bbob-biobj 256 (190)4.5 (1)2.3 (0.2)1.0 (0.2)2/10
SMS-EMOA-DE Auger bbob-biobj 193 (113)3.8 (2)24 (100)4.5 (8)1/10
SMS-EMOA-PM Auger bbob-biobj 120 (67)3.8 (4)5.8 (2)6.2 (2)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.6 (5)1.3 (1.0)3.5 (0.9)2.0 (0.5)10/10

Δ HVref
f41, 2-D		 1e0
2.0		 1e-1
33		 1e-2
292		 1e-3
2801		 #succ
10/10
DEMO Tusar bbob-biobj 3.1 (4)4.5 (4)2.5 (2)2.6 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.8)1.0 (0.8)1.0 (0.6)1.0 (0.6)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 710 (619)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 314 (188)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 13 (42)2.0 (3)1.3 (0.7)1.5 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 13 (21)7.3 (6)4.4 (1)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 8.7 (11)3.9 (2)4.8 (3)∞20030/10
NSGA-II-MATLAB Auger bbob-biobj 109 (96)17 (3)3.6 (0.8)3.1 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1260 (1209)3064 (6171)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.8 (2)2.9 (4)19 (32)63 (76)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.6 (1)6.5 (6)28 (37)100 (177)0/10
RM-MEDA Auger bbob-biobj 242 (154)34 (8)6.2 (1)1.5 (0.8)0/10
SMS-EMOA-DE Auger bbob-biobj 204 (61)24 (6)4.7 (0.8)2.9 (6)0/10
SMS-EMOA-PM Auger bbob-biobj 178 (74)25 (7)10 (6)4.8 (3)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.6 (3)5.9 (3)7.0 (4)5.2 (1)10/10

Δ HVref
f42, 2-D		 1e0
5.0		 1e-1
69		 1e-2
1277		 1e-3
9996		 #succ
10/10
DEMO Tusar bbob-biobj 2.6 (1)3.1 (4)1.0 (0.4)2.1 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (0.2)1.00 (2)1.7 (2)3.3 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 95 (33)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 141 (195)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.3 (2)1.6 (3)1 (0.9)2.0 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 4.8 (0)4.1 (5)1.5 (1.0)∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 3.3 (8)3.3 (3)1.4 (1)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 26 (1)9.4 (1)1.3 (0.6)1.9 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 370 (1176)1630 (2181)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.2 (9)26 (18)35 (55)24 (12)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (5)46 (7)22 (56)73 (36)0/10
RM-MEDA Auger bbob-biobj 71 (51)17 (8)1.9 (0.6)1.00 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 86 (87)13 (8)1.9 (0.5)1.3 (1.0)0/10
SMS-EMOA-PM Auger bbob-biobj 47 (41)14 (9)9.1 (3)10 (8)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.8 (0.8)7.3 (15)2.6 (2)1.8 (0.7)8/10

Δ HVref
f43, 2-D		 1e0
2.0		 1e-1
161		 1e-2
867		 1e-3
5539		 #succ
10/10
DEMO Tusar bbob-biobj 5.0 (8)1.7 (1)1.4 (0.4)1.6 (0.5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.9)1.1 (2)1.00 (0.8)1.8 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 703 (788)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.8 (11)1.00 (0.8)1.9 (2)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 19 (28)2.0 (0.9)2.0 (1)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 12 (4)1.8 (1)1.7 (0.6)∞20020/10
NSGA-II-MATLAB Auger bbob-biobj 83 (83)4.3 (0.8)1.6 (0.5)1.3 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1013 (836)1350 (1363)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (2)2.8 (2)10 (17)52 (51)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.6 (2)2.6 (3)19 (16)54 (47)0/10
RM-MEDA Auger bbob-biobj 289 (159)8.5 (1)2.5 (0.3)1.0 (0.2)0/10
SMS-EMOA-DE Auger bbob-biobj 243 (60)6.4 (0.9)2.3 (0.4)1.5 (0.5)0/10
SMS-EMOA-PM Auger bbob-biobj 147 (57)6.3 (2)5.4 (2)3.4 (1)0/10
UP-MO-CMA-ES Krause bbob-biobj 4.5 (6)1.6 (1)3.6 (2)2.6 (0.8)10/10

Δ HVref
f44, 2-D		 1e0
4.0		 1e-1
26		 1e-2
431		 1e-3
1848		 #succ
10/10
DEMO Tusar bbob-biobj 9.0 (35)8.2 (14)1.3 (1)2.3 (4)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (0.9)1.0 (2)2.5 (11)3.1 (7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 177 (346)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 98 (100)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.3 (4)5.9 (8)1.00 (1)1.00 (0.7)0/10
MO-DIRECT-ND Wong bbob-biobj 18 (13)11 (14)1.9 (3)1.7 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 10 (10)5.4 (8)1.5 (1)2.5 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 70 (96)24 (12)2.4 (1)2.0 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2754 (1e4)3000 (7872)700 (620)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.5 (14)65 (136)98 (179)81 (243)0/10
RANDOMSEARCH-5 Auger bbob-biobj 17 (39)41 (22)207 (503)376 (371)0/10
RM-MEDA Auger bbob-biobj 145 (112)42 (21)4.0 (2)1.5 (0.7)3/10
SMS-EMOA-DE Auger bbob-biobj 125 (57)31 (16)3.0 (1)10 (20)0/10
SMS-EMOA-PM Auger bbob-biobj 90 (78)33 (41)5.8 (6)11 (5)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.1 (7)22 (23)4.8 (5)4.0 (4)10/10

Δ HVref
f45, 2-D		 1e0
3.0		 1e-1
232		 1e-2
1131		 1e-3
8714		 #succ
10/10
DEMO Tusar bbob-biobj 1.8 (1)1.1 (0.7)1.1 (0.5)1.3 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0.8)6.5 (25)12 (9)2.1 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 212 (288)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 228 (240)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.6 (4)1.6 (2)1.4 (1)1.0 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 8.4 (11)1.3 (0.6)1.8 (1)∞20030/10
MO-DIRECT-Rank Wong bbob-biobj 6.4 (11)1.0 (0.9)1.00 (0.7)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 66 (76)2.8 (0.5)3.7 (4)3.6 (7)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1470 (5605)634 (617)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.3 (1)3.7 (6)85 (95)212 (367)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.7 (2)1.5 (2)9.3 (4)35 (25)0/10
RM-MEDA Auger bbob-biobj 197 (146)5.5 (1)1.9 (0.4)3.1 (6)1/10
SMS-EMOA-DE Auger bbob-biobj 125 (37)3.8 (0.8)1.7 (0.4)1.0 (0.8)0/10
SMS-EMOA-PM Auger bbob-biobj 88 (48)5.1 (9)6.8 (6)2.1 (0.8)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.4 (0.5)1.4 (3)2.8 (1)1.8 (0.6)10/10

Δ HVref
f46, 2-D		 1e0
4.0		 1e-1
684		 1e-2
4108		 1e-3
17611		 #succ
10/10
DEMO Tusar bbob-biobj 3.6 (6)1.0 (1)1.0 (0.2)1.1 (0.4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.5)1.5 (2)7.8 (5)4.4 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 354 (372)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.9 (2)1.5 (2)4.8 (4)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 17 (21)1.2 (0.5)∞∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 11 (8)1.0 (0.4)∞∞20030/10
NSGA-II-MATLAB Auger bbob-biobj 66 (31)1.5 (0.5)1.3 (0.6)2.4 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1502 (2550)2721 (5117)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.7)7.9 (4)37 (27)∞2e50/10
RANDOMSEARCH-5 Auger bbob-biobj 6.0 (5)7.9 (7)51 (66)177 (148)0/10
RM-MEDA Auger bbob-biobj 139 (123)2.6 (1.0)1.3 (0.2)2.1 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 117 (44)2.2 (0.7)1.2 (0.2)1.00 (0.2)0/10
SMS-EMOA-PM Auger bbob-biobj 65 (28)3.8 (2)10 (12)9.2 (6)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.6 (3)2.5 (2)2.6 (1)30 (0.1)6/10

Δ HVref
f47, 2-D		 1e0
7.0		 1e-1
370		 1e-2
3988		 1e-3
15233		 #succ
10/10
DEMO Tusar bbob-biobj 7.2 (16)1.4 (1)1.0 (0.8)1.5 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.99 (1)1.00 (1)4.0 (4)2.9 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 201 (272)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.4 (3)1.5 (2)1.6 (2)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 15 (56)1.9 (0.8)∞∞20020/10
MO-DIRECT-Rank Wong bbob-biobj 8.2 (28)1.5 (0.8)1.6 (2)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 26 (46)2.3 (0.6)1.1 (2)3.6 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2817 (269)1492 (1803)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (1)35 (83)33 (44)∞2e50/10
RANDOMSEARCH-5 Auger bbob-biobj 27 (67)16 (22)72 (15)145 (328)0/10
RM-MEDA Auger bbob-biobj 77 (67)4.5 (0.9)1.1 (0.4)1.3 (0.7)1/10
SMS-EMOA-DE Auger bbob-biobj 73 (54)3.1 (1)1.0 (0.6)1.0 (0.6)2/10
SMS-EMOA-PM Auger bbob-biobj 51 (15)5.4 (3)8.3 (10)8.9 (5)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.2 (0.8)2.8 (3)2.3 (0.9)1.8 (2)7/10

Δ HVref
f48, 2-D		 1e0
17		 1e-1
319		 1e-2
1440		 1e-3
6136		 #succ
10/10
DEMO Tusar bbob-biobj 1.8 (0.6)1.5 (3)1.00 (0.8)1.3 (1)3/10
HMO-CMA-ES Loshchilov bbob-biobj 15 (74)19 (46)8.8 (20)4.4 (5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 41 (40)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 81 (143)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 0.98 (2)1.3 (1.0)1.3 (1)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 3.4 (7)1.5 (2)1.2 (1)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 2.9 (5)1.0 (2)1.2 (1)1.6 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 17 (15)11 (0.5)3.1 (11)2.1 (3)5/10
RANDOMSEARCH-100 Auger bbob-biobj 1481 (260)245 (247)1351 (1007)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 6.0 (15)20 (33)64 (39)34 (37)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.0 (7)32 (79)133 (320)172 (444)0/10
RM-MEDA Auger bbob-biobj 27 (21)4.2 (3)1.8 (0.9)1.0 (0.2)5/10
SMS-EMOA-DE Auger bbob-biobj 25 (12)3.6 (2)1.4 (0.8)15 (25)1/10
SMS-EMOA-PM Auger bbob-biobj 20 (13)11 (37)6.1 (17)18 (13)4/10
UP-MO-CMA-ES Krause bbob-biobj 1.4 (3)2.8 (3)2.6 (2)2.2 (1)7/10

Δ HVref
f49, 2-D		 1e0
6.0		 1e-1
454		 1e-2
1525		 1e-3
20183		 #succ
10/10
DEMO Tusar bbob-biobj 2.4 (4)1.3 (0.9)16 (33)3.1 (10)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1)4.0 (2)5.2 (7)1.3 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 109 (131)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.2 (3)1.1 (1)1.1 (1)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 15 (19)1.2 (0.7)1.7 (0.7)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 12 (13)1.0 (0.6)1 (0.6)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 44 (37)2.2 (2)5.0 (7)2.7 (9)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1755 (3831)4117 (6167)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.6 (9)5.4 (13)34 (60)97 (94)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.5 (6)8.6 (9)88 (102)109 (92)0/10
RM-MEDA Auger bbob-biobj 138 (46)4.0 (0.8)2.2 (0.5)1.8 (0.1)1/10
SMS-EMOA-DE Auger bbob-biobj 78 (46)3.0 (0.7)2.0 (0.8)1.1 (2)5/10
SMS-EMOA-PM Auger bbob-biobj 59 (23)5.2 (3)33 (38)7.9 (8)1/10
UP-MO-CMA-ES Krause bbob-biobj 4.1 (4)2.9 (3)4.2 (2)1.0 (0.1)9/10

Δ HVref
f50, 2-D		 1e0
3.0		 1e-1
322		 1e-2
1692		 1e-3
7560		 #succ
10/10
DEMO Tusar bbob-biobj 4.1 (2)1.0 (0.7)1.4 (1)1.7 (0.8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.87 (0.9)2.3 (3)6.5 (13)3.9 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 451 (512)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 237 (383)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.7 (3)1.0 (1)2.0 (2)∞20010/10
MO-DIRECT-ND Wong bbob-biobj 13 (34)1.1 (0.6)2.0 (1)∞20030/10
MO-DIRECT-Rank Wong bbob-biobj 8.1 (14)1.2 (0.3)2.7 (2)∞20010/10
NSGA-II-MATLAB Auger bbob-biobj 79 (90)2.3 (0.5)1.0 (0.1)1.0 (0.9)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1310 (2456)733 (1320)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (1)1.4 (0.4)27 (32)250 (278)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (3)3.8 (5)26 (11)110 (60)0/10
RM-MEDA Auger bbob-biobj 109 (75)4.5 (0.7)1.9 (0.5)1.3 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 130 (46)3.2 (0.4)1.7 (0.3)1.3 (0.3)10/10
SMS-EMOA-PM Auger bbob-biobj 89 (12)4.0 (2)6.1 (7)8.6 (17)6/10
UP-MO-CMA-ES Krause bbob-biobj 4.5 (10)1.5 (2)3.4 (2)2.4 (0.5)9/10

Δ HVref
f51, 2-D		 1e0
3.0		 1e-1
55		 1e-2
738		 1e-3
3757		 #succ
10/10
DEMO Tusar bbob-biobj 1.6 (2)2.3 (3)1.0 (0.5)6.9 (13)1/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (0.7)1.3 (0.7)8.3 (27)2.4 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 451 (638)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 221 (182)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.8 (2)1.0 (1)1.3 (0.7)2.6 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 5.4 (5)2.9 (2)1.4 (0.7)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 4.4 (4)2.2 (2)1.4 (0.5)5.3 (7)0/10
NSGA-II-MATLAB Auger bbob-biobj 61 (61)10 (4)1.5 (0.7)1.3 (0.2)3/10
RANDOMSEARCH-100 Auger bbob-biobj 668 (1101)1183 (1280)1199 (1558)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (2)2.9 (4)14 (33)41 (32)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.0 (3)3.2 (1)10 (15)54 (83)0/10
RM-MEDA Auger bbob-biobj 127 (115)19 (6)2.7 (0.7)1.00 (0.4)7/10
SMS-EMOA-DE Auger bbob-biobj 139 (55)15 (4)2.1 (1.0)7.0 (0.5)3/10
SMS-EMOA-PM Auger bbob-biobj 74 (24)10 (5)4.5 (6)4.5 (1)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.1 (2)4.1 (7)3.0 (3)2.9 (1)9/10

Δ HVref
f52, 2-D		 1e0
4.0		 1e-1
376		 1e-2
1157		 1e-3
9626		 #succ
10/10
DEMO Tusar bbob-biobj 1.1 (1)1.1 (0.8)1.6 (1)1.00 (0.4)1/10
HMO-CMA-ES Loshchilov bbob-biobj 3.1 (1)20 (3)22 (25)3.7 (4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 178 (272)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.4 (5)1.7 (2)1.4 (1)1.0 (1.0)0/10
MO-DIRECT-ND Wong bbob-biobj 7.7 (9)1.0 (0.9)1.6 (2)∞20010/10
MO-DIRECT-Rank Wong bbob-biobj 8.4 (12)1.0 (0.6)1.0 (0.2)2.1 (1)0/10
NSGA-II-MATLAB Auger bbob-biobj 74 (56)7.4 (2)7.4 (11)1.7 (0.9)0/10
RANDOMSEARCH-100 Auger bbob-biobj 988 (1759)610 (550)1723 (1340)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (4)8.4 (33)62 (103)57 (33)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.4 (3)1.3 (1)15 (19)30 (35)0/10
RM-MEDA Auger bbob-biobj 131 (68)3.9 (0.8)21 (44)2.9 (5)4/10
SMS-EMOA-DE Auger bbob-biobj 92 (29)2.8 (2)22 (44)3.4 (11)7/10
SMS-EMOA-PM Auger bbob-biobj 74 (33)4.2 (4)19 (28)4.7 (7)6/10
UP-MO-CMA-ES Krause bbob-biobj 3.0 (6)1.4 (2)3.2 (2)1.4 (0.7)9/10

Δ HVref
f53, 2-D		 1e0
2.0		 1e-1
230		 1e-2
529		 1e-3
661		 #succ
10/10
DEMO Tusar bbob-biobj 2.4 (2)1.0 (2)1.3 (3)6.3 (14)8/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)7.4 (32)13 (34)17 (41)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 314 (188)6.4 (11)∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 208 (405)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.0 (4)2.2 (7)1.0 (4)1.0 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 3.5 (8)2.3 (4)1.1 (3)1.3 (4)0/10
MO-DIRECT-Rank Wong bbob-biobj 3.0 (0)1.6 (2)1.1 (1.0)8.1 (8)0/10
NSGA-II-MATLAB Auger bbob-biobj 34 (84)3.2 (5)13 (24)16 (46)8/10
RANDOMSEARCH-100 Auger bbob-biobj 544 (772)235 (230)148 (202)∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (5)4.7 (6)95 (378)77 (151)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.0 (7)17 (0.0)188 (505)758 (1514)8/10
RM-MEDA Auger bbob-biobj 156 (46)3.8 (3)2.8 (3)4.8 (7)8/10
SMS-EMOA-DE Auger bbob-biobj 180 (33)3.6 (2)2.5 (3)4.9 (8)0/10
SMS-EMOA-PM Auger bbob-biobj 136 (45)4.8 (2)19 (44)24 (78)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.2 (0.8)2.6 (4)4.1 (10)7.5 (8)10/10

Δ HVref
f54, 2-D		 1e0
4.0		 1e-1
240		 1e-2
817		 1e-3
2789		 #succ
10/10
DEMO Tusar bbob-biobj 2.1 (3)1.5 (2)21 (2)7.5 (15)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (1)11 (23)19 (26)8.3 (9)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (2)1.1 (1)1.2 (1)1.0 (0.6)0/10
MO-DIRECT-ND Wong bbob-biobj 8.6 (7)1.7 (3)1.4 (2)6.8 (9)0/10
MO-DIRECT-Rank Wong bbob-biobj 7.2 (7)1.2 (1)1.0 (0.8)2.3 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 59 (54)2.7 (2)12 (22)6.9 (12)0/10
RANDOMSEARCH-100 Auger bbob-biobj 963 (1029)690 (1300)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.6)1.00 (1)35 (122)67 (97)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.7 (2)2.4 (4)10 (16)58 (53)0/10
RM-MEDA Auger bbob-biobj 143 (67)4.8 (2)30 (0.4)9.3 (36)2/10
SMS-EMOA-DE Auger bbob-biobj 86 (13)4.0 (1.0)2.9 (4)1.7 (2)2/10
SMS-EMOA-PM Auger bbob-biobj 65 (33)6.7 (11)6.0 (6)5.7 (6)2/10
UP-MO-CMA-ES Krause bbob-biobj 3.6 (5)1.8 (2)4.9 (3)4.1 (2)10/10

Δ HVref
f55, 2-D		 1e0
4.0		 1e-1
340		 1e-2
1693		 1e-3
4427		 #succ
10/10
DEMO Tusar bbob-biobj 3.8 (14)1.4 (1)14 (30)8.9 (5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (1)5.4 (9)12 (17)14 (6)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 157 (172)∞∞∞1500/10
MAT-SMS Al-Dujaili bbob-biobj 101 (56)∞∞∞1500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.7 (3)1.7 (0.6)1.0 (1)1.00 (0.5)0/10
MO-DIRECT-ND Wong bbob-biobj 5.4 (6)1.0 (0.5)1.0 (1)2.1 (1)0/10
MO-DIRECT-Rank Wong bbob-biobj 5.8 (12)1.0 (0.6)1 (0.7)4.4 (3)0/10
NSGA-II-MATLAB Auger bbob-biobj 83 (43)2.9 (1)13 (19)10 (7)0/10
RANDOMSEARCH-100 Auger bbob-biobj 651 (904)608 (612)∞∞2e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.9 (2)2.2 (2)87 (35)195 (248)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (1)2.3 (2)7.3 (11)77 (111)0/10
RM-MEDA Auger bbob-biobj 126 (94)4.3 (1)1.6 (0.5)6.4 (0.9)1/10
SMS-EMOA-DE Auger bbob-biobj 112 (25)3.7 (2)15 (59)6.9 (13)1/10
SMS-EMOA-PM Auger bbob-biobj 59 (23)3.4 (3)12 (30)9.2 (12)1/10
UP-MO-CMA-ES Krause bbob-biobj 1.8 (0.9)1.5 (2)2.2 (2)3.1 (2)10/10

Previous dimension | Dimension = 3 | Next dimension

Δ HVref
f1, 3-D		 1e0
1		 1e-1
42		 1e-2
416		 1e-3
2270		 #succ
10/10
DEMO Tusar bbob-biobj 5.9 (4)7.0 (3)2.8 (0.4)2.5 (0.2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (2)1.0 (0.5)1.00 (0.2)1.00 (0.1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 682 (618)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.8 (9)1.8 (1)1.4 (0.6)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 3.8 (7)8.5 (3)5.2 (5)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 3.4 (12)8.6 (4)16 (11)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 66 (163)17 (2)3.0 (0.6)6.8 (5)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.2e4 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (1)5.6 (6)36 (32)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.3 (0.5)6.5 (5)78 (67)1963 (1544)0/10
RM-MEDA Auger bbob-biobj 898 (410)41 (3)5.8 (0.5)2.0 (0.1)0/10
SMS-EMOA-DE Auger bbob-biobj 614 (214)30 (4)4.4 (0.3)1.5 (0.1)10/10
SMS-EMOA-PM Auger bbob-biobj 372 (152)21 (2)8.3 (3)9.3 (3)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.6 (2)6.7 (6)10 (2)14 (0.1)10/10

Δ HVref
f2, 3-D		 1e0
8.0		 1e-1
73		 1e-2
343		 1e-3
1417		 #succ
10/10
DEMO Tusar bbob-biobj 2.3 (4)5.4 (3)3.9 (1)2.3 (0.5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (1)1.00 (1)1 (0.5)1 (0.6)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (3)2.2 (3)2.6 (1)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 12 (19)10 (5)12 (12)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 7.8 (12)5.4 (6)16 (16)∞30020/10
NSGA-II-MATLAB Auger bbob-biobj 37 (39)11 (1)3.4 (0.6)5.2 (5)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.2e4 (8881)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (2)4.6 (9)114 (204)2089 (1641)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (11)13 (5)180 (513)2394 (2983)0/10
RM-MEDA Auger bbob-biobj 136 (69)33 (3)11 (2)3.7 (0.5)0/10
SMS-EMOA-DE Auger bbob-biobj 98 (24)21 (5)7.0 (1)1908 (3387)0/10
SMS-EMOA-PM Auger bbob-biobj 65 (20)14 (7)17 (18)103 (112)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.6 (6)6.4 (6)10 (8)13 (7)10/10

Δ HVref
f3, 3-D		 1e0
3.0		 1e-1
78		 1e-2
392		 1e-3
2435		 #succ
10/10
DEMO Tusar bbob-biobj 4.6 (6)4.4 (4)3.4 (0.3)2.9 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.0 (1)1.00 (0.2)1.00 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 705 (1219)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 33 (3)3.1 (2)2.2 (3)2.7 (4)0/10
MO-DIRECT-ND Wong bbob-biobj 20 (56)8.3 (5)6.5 (5)∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 23 (54)5.5 (3)6.3 (4)∞30020/10
NSGA-II-MATLAB Auger bbob-biobj 61 (82)55 (223)14 (1)17 (30)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.0e4 (6e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (5)16 (4)157 (112)1210 (1602)0/10
RANDOMSEARCH-5 Auger bbob-biobj 6.8 (13)14 (5)183 (38)1109 (967)0/10
RM-MEDA Auger bbob-biobj 303 (90)21 (4)6.4 (1)2.0 (0.6)0/10
SMS-EMOA-DE Auger bbob-biobj 226 (86)16 (3)4.8 (1)300 (439)0/10
SMS-EMOA-PM Auger bbob-biobj 173 (153)15 (8)122 (383)523 (1151)0/10
UP-MO-CMA-ES Krause bbob-biobj 6.8 (24)5.5 (7)11 (9)10 (3)10/10

Δ HVref
f4, 3-D		 1e0
4.0		 1e-1
91		 1e-2
346		 1e-3
2072		 #succ
10/10
DEMO Tusar bbob-biobj 2.5 (2)3.9 (2)3.6 (0.8)2.1 (0.7)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)1 (1)1.0 (0.5)1.0 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 252 (278)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (4)1.7 (2)2.2 (2)2.5 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 11 (9)5.8 (3)6.0 (2)∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 7.3 (15)3.7 (2)6.0 (3)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 74 (66)8.2 (1)3.3 (0.5)4.3 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.4e4 (2e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.6 (2)4.7 (7)52 (33)1422 (1737)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.6 (4)7.1 (9)108 (141)1613 (2792)0/10
RM-MEDA Auger bbob-biobj 253 (55)22 (3)8.0 (1)1.9 (0.2)0/10
SMS-EMOA-DE Auger bbob-biobj 180 (29)15 (3)5.5 (0.5)248 (253)0/10
SMS-EMOA-PM Auger bbob-biobj 128 (20)11 (3)57 (224)635 (964)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.8 (2)2.5 (1)9.3 (4)9.2 (4)10/10

Δ HVref
f5, 3-D		 1e0
2.0		 1e-1
81		 1e-2
697		 1e-3
9434		 #succ
10/10
DEMO Tusar bbob-biobj 2.2 (2)5.2 (2)7.0 (19)8.2 (10)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (2)1.00 (0.8)1.0 (0.3)1.0 (0.4)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 1078 (478)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (5)1.2 (0.9)2.1 (1)∞30010/10
MO-DIRECT-ND Wong bbob-biobj 7.0 (13)6.7 (5)14 (8)∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 6.0 (9)6.3 (2)43 (50)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 81 (135)10 (2)5.4 (1)25 (26)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.4e4 (3204)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (5)7.4 (4)118 (103)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.6 (1)32 (6)596 (126)1515 (1106)0/10
RM-MEDA Auger bbob-biobj 443 (247)23 (4)3.9 (0.6)1.6 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 319 (145)17 (5)3.6 (0.7)38 (46)0/10
SMS-EMOA-PM Auger bbob-biobj 219 (92)32 (4)57 (10)101 (48)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.2 (18)7.5 (17)12 (8)3.5 (0.1)10/10

Δ HVref
f6, 3-D		 1e0
2.0		 1e-1
48		 1e-2
383		 1e-3
2902		 #succ
10/10
DEMO Tusar bbob-biobj 5.2 (3)7.2 (6)3.3 (0.9)3.3 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.95 (1)1.00 (0.4)1.00 (0.4)1.0 (0.6)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 504 (204)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.1 (2)1.5 (0.3)1.9 (1)5.0 (4)0/10
MO-DIRECT-ND Wong bbob-biobj 27 (8)11 (9)7.7 (5)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 10 (19)6.4 (4)11 (7)∞30040/10
NSGA-II-MATLAB Auger bbob-biobj 151 (157)17 (4)4.2 (2)11 (11)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.9e4 (2e4)6.0e4 (3e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.4 (10)8.3 (16)103 (175)475 (491)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.2 (2)18 (28)194 (326)1147 (2228)0/10
RM-MEDA Auger bbob-biobj 460 (73)35 (5)6.1 (0.9)1.5 (0.3)0/10
SMS-EMOA-DE Auger bbob-biobj 333 (164)29 (6)5.3 (0.6)1.9 (0.5)0/10
SMS-EMOA-PM Auger bbob-biobj 222 (111)21 (8)10 (3)7.8 (2)0/10
UP-MO-CMA-ES Krause bbob-biobj 4.0 (13)6.8 (4)11 (5)10 (2)10/10

Δ HVref
f7, 3-D		 1e0
7.0		 1e-1
349		 1e-2
21556		 1e-3
2.1e5		 #succ
9/10
DEMO Tusar bbob-biobj 6.2 (13)2.8 (3)1.7 (6)1.9 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (2)1.9 (4)1.4 (1.0)1.2 (2)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.2 (2)1.00 (1)1.3 (0.8)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 13 (3)3.2 (3)1.4 (1)∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 9.4 (16)2.8 (3)1.4 (2)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 27 (44)3.1 (3)1.00 (2)1.7 (1)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.4e4 (2e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (1)40 (52)39 (36)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.9 (6)42 (39)118 (214)∞3e60/10
RM-MEDA Auger bbob-biobj 134 (59)6.1 (2)2.1 (4)4.2 (4)0/10
SMS-EMOA-DE Auger bbob-biobj 105 (25)4.9 (2)1.7 (1)1.0 (1)0/10
SMS-EMOA-PM Auger bbob-biobj 76 (26)4.9 (2)7.8 (9)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.6 (4)5.3 (6)36 (70)15 (18)1/10

Δ HVref
f8, 3-D		 1e0
10		 1e-1
549		 1e-2
14045		 1e-3
1.7e5		 #succ
2/10
DEMO Tusar bbob-biobj 3.0 (6)2.0 (0.7)2.2 (3)4.3 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (4)1 (0.5)2.2 (2)1.0 (0.3)2/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 216 (360)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.3 (1)1.3 (2)∞∞30030/10
MO-DIRECT-ND Wong bbob-biobj 5.2 (7)2.1 (0.7)∞∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 3.9 (3)2.7 (3)∞∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 31 (31)2.0 (0.6)4.7 (6)4.6 (5)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.1e4 (2e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (3)20 (24)92 (71)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (2)30 (33)345 (498)167 (112)0/10
RM-MEDA Auger bbob-biobj 122 (24)4.2 (0.7)1.00 (0.4)1.6 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 68 (24)3.3 (0.7)1.3 (1)1.8 (2)0/10
SMS-EMOA-PM Auger bbob-biobj 49 (16)6.1 (9)8.7 (8)4.5 (3)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.5 (0.9)3.3 (2)31 (54)29 (50)0/10

Δ HVref
f9, 3-D		 1e0
6.0		 1e-1
68		 1e-2
290		 1e-3
1918		 #succ
10/10
DEMO Tusar bbob-biobj 6.1 (8)6.3 (5)4.4 (1)1.9 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 2.2 (3)1.0 (0.7)1.0 (0.4)1 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 340 (338)∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.9 (3)1.7 (0.4)1.4 (1)2.3 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 15 (15)5.8 (3)5.0 (3)7.8 (9)0/10
MO-DIRECT-Rank Wong bbob-biobj 11 (8)4.3 (2)5.7 (4)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 65 (53)12 (3)4.1 (1)4.0 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.6e4 (1e5)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.3)5.9 (5)132 (198)722 (1078)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (2)9.4 (14)218 (287)2268 (2993)0/10
RM-MEDA Auger bbob-biobj 205 (84)26 (4)8.2 (2)2.0 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 126 (44)19 (3)6.4 (0.8)107 (78)0/10
SMS-EMOA-PM Auger bbob-biobj 92 (27)17 (9)83 (41)113 (391)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.3 (8)7.4 (5)12 (7)8.4 (3)9/10

Δ HVref
f10, 3-D		 1e0
7.0		 1e-1
348		 1e-2
2284		 1e-3
6628		 #succ
10/10
DEMO Tusar bbob-biobj 2.1 (0.6)1.9 (1.0)1.5 (1)3.5 (6)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (2)2.7 (6)11 (16)7.2 (7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 297 (416)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.5 (1)1.0 (0.8)1.0 (0.9)4.5 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 9.1 (0.2)1.5 (0.3)1.8 (1)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 12 (14)1.6 (0.5)1.9 (2)∞30020/10
NSGA-II-MATLAB Auger bbob-biobj 50 (41)3.0 (2)13 (26)15 (8)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.3e4 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (2)3.0 (3)69 (80)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 0.99 (0.5)2.3 (2)43 (53)1023 (576)0/10
RM-MEDA Auger bbob-biobj 159 (31)5.6 (1)1.3 (0.2)1.0 (0.6)2/10
SMS-EMOA-DE Auger bbob-biobj 96 (26)4.2 (0.8)16 (34)13 (12)1/10
SMS-EMOA-PM Auger bbob-biobj 63 (22)3.6 (1)34 (47)26 (19)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.1 (2)1.6 (0.4)3.2 (1)4.2 (1.0)10/10

Δ HVref
f11, 3-D		 1e0
6.0		 1e-1
50		 1e-2
320		 1e-3
3248		 #succ
10/10
DEMO Tusar bbob-biobj 5.2 (12)4.1 (9)1.9 (2)1.5 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0.4)1.0 (0.4)1 (0.8)1.00 (0.1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 37 (68)∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 16 (29)44 (45)∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.4 (6)13 (54)15 (13)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 17 (10)14 (8)91 (183)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 24 (20)24 (29)∞∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 31 (35)46 (96)8.5 (1)2.1 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2679 (20)1805 (4436)1528 (1859)∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (2)18 (42)114 (255)28 (30)0/10
RANDOMSEARCH-5 Auger bbob-biobj 4.5 (9)23 (55)354 (0.9)189 (695)1/10
RM-MEDA Auger bbob-biobj 94 (175)31 (32)12 (9)13 (38)0/10
SMS-EMOA-DE Auger bbob-biobj 49 (55)15 (9)3.9 (1)∞3e50/10
SMS-EMOA-PM Auger bbob-biobj 46 (8)16 (8)7.3 (6)29 (70)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.2 (12)5.6 (2)4.6 (3)3.3 (3)10/10

Δ HVref
f12, 3-D		 1e0
3.0		 1e-1
32		 1e-2
238		 1e-3
1581		 #succ
10/10
DEMO Tusar bbob-biobj 2.0 (2)3.3 (5)2.3 (3)1.3 (2)6/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.6)0.98 (0.9)1.2 (2)1.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 105 (87)30 (41)∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 141 (125)66 (82)∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.9 (7)1.5 (0.9)1.00 (1)2.3 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 13 (0.1)8.5 (10)7.3 (15)2.7 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 10 (16)8.7 (11)7.4 (4)2.7 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 41 (58)16 (11)5.8 (1)11 (13)4/10
RANDOMSEARCH-100 Auger bbob-biobj 1246 (2576)5950 (9180)2239 (2521)1718 (1281)0/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (1)1.5 (2)155 (325)208 (257)2/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (7)3.4 (5)1432 (68)563 (459)4/10
RM-MEDA Auger bbob-biobj 156 (178)38 (21)9.3 (2)2.4 (1)6/10
SMS-EMOA-DE Auger bbob-biobj 152 (151)29 (18)6.4 (4)83 (95)1/10
SMS-EMOA-PM Auger bbob-biobj 81 (48)19 (11)167 (635)138 (252)1/10
UP-MO-CMA-ES Krause bbob-biobj 2.6 (4)2.5 (4)5.7 (2)6.1 (8)10/10

Δ HVref
f13, 3-D		 1e0
4.0		 1e-1
17		 1e-2
163		 1e-3
621		 #succ
10/10
DEMO Tusar bbob-biobj 1.8 (2)5.6 (5)2.9 (2)2.2 (0.8)9/10
HMO-CMA-ES Loshchilov bbob-biobj 1.3 (0.7)0.98 (1)1.0 (2)1.00 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 516 (547)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.3 (7)11 (43)2.7 (5)4.7 (9)0/10
MO-DIRECT-ND Wong bbob-biobj 34 (51)28 (25)8.0 (7)8.2 (14)0/10
MO-DIRECT-Rank Wong bbob-biobj 26 (21)16 (23)5.9 (8)7.3 (7)0/10
NSGA-II-MATLAB Auger bbob-biobj 87 (89)38 (11)4.9 (1)2.0 (1)7/10
RANDOMSEARCH-100 Auger bbob-biobj 9536 (7624)3.6e4 (2e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (2)6.2 (17)7.1 (6)71 (169)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.5 (1)11 (6)8.9 (3)98 (230)2/10
RM-MEDA Auger bbob-biobj 219 (36)74 (32)15 (3)6.7 (6)4/10
SMS-EMOA-DE Auger bbob-biobj 197 (32)71 (12)10 (2)57 (363)0/10
SMS-EMOA-PM Auger bbob-biobj 101 (34)36 (19)6.0 (2)7.8 (14)1/10
UP-MO-CMA-ES Krause bbob-biobj 2.2 (2)3.5 (3)4.5 (9)5.8 (9)10/10

Δ HVref
f14, 3-D		 1e0
5.0		 1e-1
139		 1e-2
1774		 1e-3
10133		 #succ
10/10
DEMO Tusar bbob-biobj 3.2 (2)5.0 (4)4.5 (2)10 (8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.00 (2)1.0 (2)1.6 (3)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.0 (4)8.8 (12)17 (21)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 20 (22)8.7 (8)∞∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 11 (17)6.0 (4)∞∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 61 (68)10 (0.8)8.2 (2)21 (11)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.3e4 (3e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (6)48 (90)326 (423)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.3 (8)26 (15)918 (1314)∞3e60/10
RM-MEDA Auger bbob-biobj 230 (126)20 (5)2.6 (0.4)1.00 (0.6)1/10
SMS-EMOA-DE Auger bbob-biobj 172 (64)253 (1082)1524 (2029)∞3e50/10
SMS-EMOA-PM Auger bbob-biobj 97 (56)12 (16)715 (592)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.5 (2)5.8 (8)5.1 (0.8)2.9 (0.6)10/10

Δ HVref
f15, 3-D		 1e0
3.0		 1e-1
42		 1e-2
352		 1e-3
1838		 #succ
10/10
DEMO Tusar bbob-biobj 2.7 (2)7.8 (7)4.2 (2)4.7 (5)1/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (1)1 (1)1.0 (1.0)1.00 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.6 (6)1.6 (0.6)3.0 (1)16 (9)0/10
MO-DIRECT-ND Wong bbob-biobj 17 (38)13 (13)14 (7)16 (18)0/10
MO-DIRECT-Rank Wong bbob-biobj 8.1 (12)7.1 (5)13 (27)16 (10)0/10
NSGA-II-MATLAB Auger bbob-biobj 57 (111)20 (6)7.2 (7)17 (55)0/10
RANDOMSEARCH-100 Auger bbob-biobj 9828 (2e4)2.1e4 (1e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.3 (2)11 (33)125 (119)709 (964)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.5 (0.7)16 (17)249 (420)2637 (4897)0/10
RM-MEDA Auger bbob-biobj 302 (97)47 (22)9.3 (3)2.8 (0.9)2/10
SMS-EMOA-DE Auger bbob-biobj 267 (190)40 (15)372 (1066)1470 (1714)0/10
SMS-EMOA-PM Auger bbob-biobj 151 (71)31 (24)784 (924)696 (1203)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.0 (5)11 (11)12 (10)11 (4)10/10

Δ HVref
f16, 3-D		 1e0
4.0		 1e-1
670		 1e-2
5703		 1e-3
2.0e5		 #succ
7/10
DEMO Tusar bbob-biobj 13 (55)1.2 (2)10 (20)2.2 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.95 (1)1.6 (0.3)6.6 (6)1.0 (1.0)7/10
MAT-DIRECT Al-Dujaili bbob-biobj 558 (661)∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 541 (731)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 12 (4)1.4 (2)1.0 (1)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 34 (76)1.5 (2)2.4 (4)∞30040/10
MO-DIRECT-Rank Wong bbob-biobj 11 (21)1.00 (1)2.4 (1)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 67 (76)1.7 (2)4.0 (7)1.1 (0.8)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.4e4 (8e4)4125 (4366)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 7.0 (12)17 (37)63 (78)14 (13)0/10
RANDOMSEARCH-5 Auger bbob-biobj 8.9 (18)100 (248)275 (313)139 (176)0/10
RM-MEDA Auger bbob-biobj 211 (83)4.3 (3)8.3 (15)2.1 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 188 (90)2.7 (1)24 (27)∞3e50/10
SMS-EMOA-PM Auger bbob-biobj 98 (55)2.1 (3)126 (107)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 7.7 (14)2.3 (4)61 (265)36 (69)0/10

Δ HVref
f17, 3-D		 1e0
22		 1e-1
1125		 1e-2
25640		 1e-3
1.7e5		 #succ
7/10
DEMO Tusar bbob-biobj 2.1 (4)1.0 (0.9)1.6 (3)3.3 (6)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (5)1.3 (0.7)4.0 (2)4.9 (8)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.2 (1)2.0 (5)1.2 (1)∞30010/10
MO-DIRECT-ND Wong bbob-biobj 11 (10)1.8 (2)∞∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 6.4 (13)1.3 (2)∞∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 20 (19)1.2 (0.5)2.0 (6)4.3 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 6544 (1e4)2487 (2267)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (3)26 (58)34 (49)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.0 (2)39 (66)187 (439)166 (364)0/10
RM-MEDA Auger bbob-biobj 56 (32)3.4 (2)1.00 (0.7)1.00 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 38 (13)2.0 (1.0)5.4 (9)∞3e50/10
SMS-EMOA-PM Auger bbob-biobj 24 (11)31 (1)13 (12)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.4 (3)2.6 (3)18 (36)42 (39)0/10

Δ HVref
f18, 3-D		 1e0
14		 1e-1
78		 1e-2
636		 1e-3
2621		 #succ
10/10
DEMO Tusar bbob-biobj 6.8 (16)2.5 (0.9)1.3 (0.6)1.0 (0.3)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)1.00 (3)1.00 (3)1.4 (5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 157 (265)∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 146 (177)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.0 (0.4)4.9 (29)1.8 (3)4.7 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 24 (54)7.2 (11)6.8 (8)5.3 (9)0/10
MO-DIRECT-Rank Wong bbob-biobj 18 (2)7.5 (11)9.3 (17)∞30020/10
NSGA-II-MATLAB Auger bbob-biobj 15 (30)8.9 (3)1.7 (1)2.9 (3)2/10
RANDOMSEARCH-100 Auger bbob-biobj 6178 (1e4)6984 (7729)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (9)9.3 (0.7)63 (27)290 (272)0/10
RANDOMSEARCH-5 Auger bbob-biobj 26 (64)151 (377)238 (589)420 (459)0/10
RM-MEDA Auger bbob-biobj 64 (24)24 (10)4.7 (1)2.0 (0.7)1/10
SMS-EMOA-DE Auger bbob-biobj 57 (11)16 (6)2.8 (0.9)116 (172)0/10
SMS-EMOA-PM Auger bbob-biobj 37 (34)11 (4)2.7 (2)121 (200)0/10
UP-MO-CMA-ES Krause bbob-biobj 16 (75)5.0 (0.6)2.4 (1)5.0 (4)10/10

Δ HVref
f19, 3-D		 1e0
4.0		 1e-1
222		 1e-2
5928		 1e-3
29477		 #succ
10/10
DEMO Tusar bbob-biobj 1.4 (1)1.9 (3)1.00 (1)3.2 (10)1/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (0.6)2.0 (8)9.0 (7)4.4 (5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.2 (3)1.0 (2)1.5 (2)1 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 9.1 (7)2.8 (3)2.4 (2)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 8.2 (6)1.9 (2)1.0 (1)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 64 (63)8.6 (25)4.9 (5)3.0 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 7654 (7509)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (1)1.9 (2)4.9 (4)97 (140)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (0.5)1.5 (0.6)2.9 (2)34 (34)0/10
RM-MEDA Auger bbob-biobj 277 (101)12 (4)6.5 (0.3)10 (20)4/10
SMS-EMOA-DE Auger bbob-biobj 191 (59)8.1 (2)24 (25)41 (31)0/10
SMS-EMOA-PM Auger bbob-biobj 100 (37)4.5 (1)25 (42)43 (34)0/10
UP-MO-CMA-ES Krause bbob-biobj 1 (1)1.0 (0.9)1.2 (0.9)12 (0.4)9/10

Δ HVref
f20, 3-D		 1e0
19		 1e-1
102		 1e-2
389		 1e-3
2887		 #succ
10/10
DEMO Tusar bbob-biobj 9.0 (22)5.6 (12)4.1 (2)2.6 (2)3/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1)1.0 (0.8)1.0 (0.6)1.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 49 (57)20 (26)∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 31 (17)10 (14)∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 19 (80)4.6 (23)3.1 (0.6)1.8 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 22 (6)12 (9)10 (8)4.4 (5)0/10
MO-DIRECT-Rank Wong bbob-biobj 22 (79)8.6 (6)8.2 (22)10 (16)0/10
NSGA-II-MATLAB Auger bbob-biobj 127 (23)93 (76)137 (291)32 (110)1/10
RANDOMSEARCH-100 Auger bbob-biobj 4179 (4613)3763 (7062)1869 (1535)∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 12 (60)117 (11)512 (774)432 (432)0/10
RANDOMSEARCH-5 Auger bbob-biobj 92 (2)453 (1115)1244 (3858)1208 (943)0/10
RM-MEDA Auger bbob-biobj 969 (4694)219 (522)67 (155)11 (42)3/10
SMS-EMOA-DE Auger bbob-biobj 46 (25)14 (9)22 (45)70 (104)0/10
SMS-EMOA-PM Auger bbob-biobj 46 (16)27 (54)109 (620)158 (183)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.6 (6)10 (29)15 (18)6.6 (5)10/10

Δ HVref
f21, 3-D		 1e0
9.0		 1e-1
78		 1e-2
536		 1e-3
3164		 #succ
10/10
DEMO Tusar bbob-biobj 2.0 (2)5.5 (8)2.5 (1)1.7 (3)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)1.0 (2)1.0 (1)1.0 (1.0)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.6 (3)5.6 (19)1.5 (2)1.2 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 13 (10)14 (24)6.7 (11)9.0 (12)0/10
MO-DIRECT-Rank Wong bbob-biobj 13 (10)8.0 (7)4.1 (8)2.1 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 40 (34)11 (4)4.9 (7)15 (30)1/10
RANDOMSEARCH-100 Auger bbob-biobj 1.8e4 (8e4)3.6e4 (6e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (2)10 (10)108 (326)223 (284)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.4 (0.7)30 (65)706 (228)1132 (1707)0/10
RM-MEDA Auger bbob-biobj 104 (59)21 (12)4.6 (2)1.8 (1)3/10
SMS-EMOA-DE Auger bbob-biobj 94 (32)19 (8)4.0 (2)68 (101)0/10
SMS-EMOA-PM Auger bbob-biobj 58 (27)15 (11)67 (141)97 (167)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.3 (5)10 (17)8.2 (6)5.0 (3)10/10

Δ HVref
f22, 3-D		 1e0
1		 1e-1
94		 1e-2
677		 1e-3
9994		 #succ
10/10
DEMO Tusar bbob-biobj 17 (38)5.1 (9)4.0 (3)4.2 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (1)1.00 (1.0)1.0 (0.3)1.00 (0.9)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 2117 (4444)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.1 (10)5.5 (7)4.9 (5)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 6.3 (13)7.3 (7)43 (40)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 4.5 (18)6.3 (3)44 (39)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 73 (0)14 (12)15 (30)38 (35)0/10
RANDOMSEARCH-100 Auger bbob-biobj 3.8e4 (8e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.2 (4)31 (2)340 (484)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.9 (2)96 (4)739 (287)2911 (3077)0/10
RM-MEDA Auger bbob-biobj 664 (686)17 (4)4.0 (1)1.4 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 562 (417)15 (4)448 (444)271 (150)0/10
SMS-EMOA-PM Auger bbob-biobj 517 (74)18 (23)479 (364)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 25 (2)7.7 (23)15 (5)3.2 (0.4)10/10

Δ HVref
f23, 3-D		 1e0
2.0		 1e-1
28		 1e-2
369		 1e-3
3845		 #succ
10/10
DEMO Tusar bbob-biobj 3.3 (4)6.0 (8)3.3 (1)2.5 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.75 (1)1.00 (0.3)1.00 (0.5)1 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 1067 (900)∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.4 (0)5.4 (8)3.0 (2)1.8 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 1.7 (3)11 (5)7.0 (4)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 2.2 (8)12 (11)7.5 (4)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 37 (46)26 (11)4.1 (1)14 (15)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.3e4 (1e4)5.0e4 (2e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.6)3.3 (3)113 (36)357 (161)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (0.6)6.2 (6)48 (25)814 (789)0/10
RM-MEDA Auger bbob-biobj 355 (198)50 (11)9.1 (9)1.5 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 364 (121)52 (10)6.3 (1)710 (917)0/10
SMS-EMOA-PM Auger bbob-biobj 211 (164)44 (20)243 (40)720 (1034)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.2 (1)4.4 (5)8.5 (4)7.0 (0.9)10/10

Δ HVref
f24, 3-D		 1e0
8.0		 1e-1
926		 1e-2
8211		 1e-3
2.1e5		 #succ
7/10
DEMO Tusar bbob-biobj 4.7 (1)1 (1.0)1.8 (2)2.4 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.3 (2)3.8 (8)6.0 (9)1.0 (1)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.3 (4)1.8 (1)1.00 (0.8)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 15 (28)2.0 (2)1.8 (2)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 10 (16)1.1 (0.8)3.5 (3)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 42 (64)6.9 (28)2.5 (0.5)1.7 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2705 (2135)1444 (1482)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (1)16 (17)62 (62)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.5 (3)24 (16)222 (481)∞3e60/10
RM-MEDA Auger bbob-biobj 113 (71)2.2 (1)1.3 (1)2.3 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 95 (48)2.4 (3)17 (10)13 (18)0/10
SMS-EMOA-PM Auger bbob-biobj 58 (32)11 (25)39 (72)13 (19)0/10
UP-MO-CMA-ES Krause bbob-biobj 1 (0.8)2.1 (3)93 (276)15 (29)0/10

Δ HVref
f25, 3-D		 1e0
4.0		 1e-1
520		 1e-2
6596		 1e-3
1.6e5		 #succ
4/10
DEMO Tusar bbob-biobj 10 (7)1.1 (2)2.2 (3)3.4 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (0.6)1.00 (3)3.4 (2)1.00 (2)4/10
MAT-DIRECT Al-Dujaili bbob-biobj 517 (309)∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 550 (380)4.2 (3)∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.7 (9)1.0 (4)∞∞30020/10
MO-DIRECT-ND Wong bbob-biobj 28 (42)1.8 (2)4.4 (6)∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 22 (29)1.2 (1.0)4.3 (4)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 74 (92)1.9 (0.5)2.4 (0.9)2.0 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.0e4 (2e4)1541 (3723)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0.8)11 (26)111 (153)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.5 (19)23 (52)292 (228)∞3e60/10
RM-MEDA Auger bbob-biobj 229 (98)3.4 (0.8)1.00 (0.3)1.0 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 211 (65)2.8 (0.6)1.4 (1)3.7 (3)0/10
SMS-EMOA-PM Auger bbob-biobj 118 (56)3.0 (2)17 (23)17 (14)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.3 (12)1.9 (1)53 (228)19 (23)0/10

Δ HVref
f26, 3-D		 1e0
8.0		 1e-1
836		 1e-2
2117		 1e-3
3012		 #succ
10/10
DEMO Tusar bbob-biobj 5.0 (6)1.00 (2)1.9 (3)27 (56)2/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (0.7)11 (42)14 (24)44 (29)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.5 (4)1.1 (2)1.0 (1)1.0 (0.8)0/10
MO-DIRECT-ND Wong bbob-biobj 15 (20)1.9 (3)1.4 (2)2.7 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 18 (17)1.6 (2)1.2 (1)4.2 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 49 (40)2.3 (4)20 (71)30 (50)2/10
RANDOMSEARCH-100 Auger bbob-biobj 1.3e4 (2e4)958 (942)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (3)82 (95)68 (45)175 (150)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.0 (0.6)62 (147)501 (1351)556 (1382)0/10
RM-MEDA Auger bbob-biobj 149 (47)2.4 (1)17 (36)44 (100)2/10
SMS-EMOA-DE Auger bbob-biobj 114 (45)2.1 (1)17 (0.8)44 (101)0/10
SMS-EMOA-PM Auger bbob-biobj 80 (40)1.8 (2)5.0 (13)58 (41)0/10
UP-MO-CMA-ES Krause bbob-biobj 6.6 (21)2.8 (5)4.1 (10)122 (502)7/10

Δ HVref
f27, 3-D		 1e0
4.0		 1e-1
588		 1e-2
3297		 1e-3
23418		 #succ
10/10
DEMO Tusar bbob-biobj 1.8 (2)1.0 (1)1.0 (0.5)2.6 (5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.8)1.3 (3)17 (8)2.9 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.7 (5)1.6 (2)1.9 (2)1.2 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 17 (36)1.3 (0.4)2.8 (3)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 19 (18)1.1 (0.3)1.2 (2)1.3 (1)0/10
NSGA-II-MATLAB Auger bbob-biobj 68 (62)1.7 (1)6.8 (11)2.8 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 8882 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (2)6.6 (12)32 (13)117 (157)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.5 (1)1.2 (2)14 (37)73 (103)0/10
RM-MEDA Auger bbob-biobj 230 (100)2.9 (0.3)40 (68)8.8 (10)1/10
SMS-EMOA-DE Auger bbob-biobj 166 (75)2.4 (0.6)11 (23)5.9 (6)0/10
SMS-EMOA-PM Auger bbob-biobj 116 (38)4.0 (0.4)37 (46)14 (26)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.3 (1)1.0 (2)1.7 (2)1.00 (0.3)10/10

Δ HVref
f28, 3-D		 1e0
3.0		 1e-1
55		 1e-2
215		 1e-3
1830		 #succ
10/10
DEMO Tusar bbob-biobj 4.8 (5)3.6 (2)4.0 (2)5.5 (12)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (2)1.0 (0.5)1.00 (0.9)1.9 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.7 (5)2.1 (3)1.6 (2)1.0 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 16 (17)4.3 (3)3.4 (2)2.7 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 14 (20)3.6 (3)4.4 (3)7.5 (10)0/10
NSGA-II-MATLAB Auger bbob-biobj 79 (62)12 (4)4.8 (0.8)5.9 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.7e4 (5e4)1.4e4 (2e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0.6)2.5 (0.8)25 (38)226 (240)0/10
RANDOMSEARCH-5 Auger bbob-biobj 7.4 (6)4.2 (3)51 (76)441 (954)0/10
RM-MEDA Auger bbob-biobj 286 (179)29 (7)11 (2)2.5 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 244 (85)22 (9)8.8 (2)165 (410)0/10
SMS-EMOA-PM Auger bbob-biobj 207 (63)26 (15)14 (6)225 (164)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.8 (6)5.0 (5)10 (12)6.0 (2)10/10

Δ HVref
f29, 3-D		 1e0
1		 1e-1
74		 1e-2
2499		 1e-3
7053		 #succ
10/10
DEMO Tusar bbob-biobj 6.4 (2)7.2 (4)16 (32)18 (22)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (0)1.00 (0.8)1.00 (4)3.0 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 976 (1095)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.2 (10)7.6 (7)1.8 (2)∞30040/10
MO-DIRECT-ND Wong bbob-biobj 13 (30)11 (5)∞∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 11 (34)8.9 (6)11 (10)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 144 (274)11 (3)1.9 (1)20 (29)0/10
RANDOMSEARCH-100 Auger bbob-biobj 3.9e4 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.6 (1)17 (46)121 (303)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 8.7 (30)30 (39)521 (1126)∞3e60/10
RM-MEDA Auger bbob-biobj 765 (285)26 (8)1.2 (0.2)1 (0.4)1/10
SMS-EMOA-DE Auger bbob-biobj 721 (176)19 (4)281 (270)384 (734)0/10
SMS-EMOA-PM Auger bbob-biobj 409 (138)86 (329)161 (181)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 4.6 (6)7.8 (5)3.4 (2)4.3 (0.6)10/10

Δ HVref
f30, 3-D		 1e0
4.0		 1e-1
46		 1e-2
339		 1e-3
2110		 #succ
10/10
DEMO Tusar bbob-biobj 10 (22)6.2 (10)4.0 (2)3.8 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (1)0.99 (2)1.00 (0.9)1.0 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 530 (422)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.3 (2)2.4 (2)1.3 (1)2.9 (4)0/10
MO-DIRECT-ND Wong bbob-biobj 14 (44)12 (16)10 (10)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 7.5 (14)6.4 (5)6.5 (8)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 21 (52)18 (4)10 (18)16 (11)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.7e4 (3e4)3.0e4 (3e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (1)24 (2)149 (40)600 (1357)0/10
RANDOMSEARCH-5 Auger bbob-biobj 7.5 (2)40 (8)876 (3973)1853 (1908)0/10
RM-MEDA Auger bbob-biobj 217 (81)35 (20)7.3 (1)2.0 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 204 (104)32 (9)227 (0.9)335 (603)0/10
SMS-EMOA-PM Auger bbob-biobj 164 (43)30 (17)121 (243)597 (1399)0/10
UP-MO-CMA-ES Krause bbob-biobj 10 (22)7.8 (8)12 (10)9.4 (6)10/10

Δ HVref
f31, 3-D		 1e0
2.0		 1e-1
660		 1e-2
8169		 1e-3
1.8e5		 #succ
9/10
DEMO Tusar bbob-biobj 4.6 (8)1.4 (1)2.1 (3)1.2 (0.9)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (0)2.4 (3)7.5 (3)1.2 (1)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 1109 (1125)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.6 (5)1 (2)3.3 (3)∞30020/10
MO-DIRECT-ND Wong bbob-biobj 10 (0)1.5 (1)1.6 (3)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 7.0 (0)1.1 (1.0)1.00 (0.4)∞30020/10
NSGA-II-MATLAB Auger bbob-biobj 76 (176)3.5 (0.9)1.8 (2)1.0 (0.7)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.4e4 (3e4)4331 (7614)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 5.0 (2)10 (7)50 (49)16 (19)0/10
RANDOMSEARCH-5 Auger bbob-biobj 5.4 (9)14 (23)283 (647)76 (42)0/10
RM-MEDA Auger bbob-biobj 477 (276)4.3 (3)6.4 (21)1.4 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 392 (70)2.5 (0.9)17 (18)4.1 (5)1/10
SMS-EMOA-PM Auger bbob-biobj 245 (109)3.7 (2)59 (64)16 (11)0/10
UP-MO-CMA-ES Krause bbob-biobj 8.8 (14)2.3 (3)43 (92)11 (17)0/10

Δ HVref
f32, 3-D		 1e0
12		 1e-1
946		 1e-2
8968		 1e-3
1.1e5		 #succ
3/10
DEMO Tusar bbob-biobj 10 (14)1.2 (0.8)2.5 (3)7.1 (9)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)1.3 (4)2.7 (4)1 (0.5)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 25 (60)2.3 (2)1.00 (1)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 13 (23)1.9 (0.3)3.2 (5)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 6.4 (6)1.00 (1)1.6 (0.3)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 38 (27)1.1 (0.3)2.1 (2)2.6 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 8732 (6535)3033 (1823)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 8.5 (13)44 (89)301 (335)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.7 (12)47 (104)448 (378)267 (135)0/10
RM-MEDA Auger bbob-biobj 92 (30)2.6 (1.0)1.1 (0.1)1.7 (0.8)0/10
SMS-EMOA-DE Auger bbob-biobj 78 (33)2.0 (0.6)1.6 (1)7.1 (6)0/10
SMS-EMOA-PM Auger bbob-biobj 57 (23)2.2 (2)19 (40)27 (38)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.1 (3)2.5 (3)86 (1)44 (107)0/10

Δ HVref
f33, 3-D		 1e0
11		 1e-1
553		 1e-2
2134		 1e-3
3462		 #succ
10/10
DEMO Tusar bbob-biobj 4.6 (7)1.7 (3)18 (4)38 (65)4/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.6)28 (32)14 (10)58 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (3)1.8 (2)1.2 (0.7)1.0 (0.5)0/10
MO-DIRECT-ND Wong bbob-biobj 13 (19)2.9 (7)1.8 (2)1.7 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 6.3 (8)1.00 (0.2)1.0 (1)1.6 (2)0/10
NSGA-II-MATLAB Auger bbob-biobj 31 (31)2.9 (7)19 (1)27 (87)2/10
RANDOMSEARCH-100 Auger bbob-biobj 3.1e4 (3e4)1211 (1510)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (2)77 (407)63 (109)96 (281)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.5 (4)135 (340)619 (703)534 (690)0/10
RM-MEDA Auger bbob-biobj 107 (43)4.4 (3)37 (106)23 (87)3/10
SMS-EMOA-DE Auger bbob-biobj 88 (19)3.1 (1)17 (0.8)59 (108)0/10
SMS-EMOA-PM Auger bbob-biobj 52 (22)6.4 (22)37 (35)63 (72)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.5 (3)3.9 (0.4)354 (358)375 (1517)5/10

Δ HVref
f34, 3-D		 1e0
22		 1e-1
674		 1e-2
3168		 1e-3
14422		 #succ
10/10
DEMO Tusar bbob-biobj 2.8 (5)1.1 (2)3.7 (7)14 (31)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (0.2)4.1 (8)11 (7)34 (17)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 0.98 (1)2.3 (1)1.2 (2)1.00 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 2.8 (1)1.1 (0.9)1.1 (1)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 2.9 (7)1.00 (1)1.00 (1)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 14 (12)1.4 (0.4)10 (13)9.4 (11)0/10
RANDOMSEARCH-100 Auger bbob-biobj 6325 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.6 (10)4.8 (6)32 (55)100 (78)0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.0 (2)3.9 (5)101 (60)201 (138)0/10
RM-MEDA Auger bbob-biobj 41 (37)3.5 (1)20 (47)21 (31)0/10
SMS-EMOA-DE Auger bbob-biobj 37 (21)2.9 (0.5)13 (26)49 (94)0/10
SMS-EMOA-PM Auger bbob-biobj 29 (15)4.7 (5)9.3 (6)40 (38)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.9 (4)1.1 (2)1.7 (1)1.6 (0.5)10/10

Δ HVref
f35, 3-D		 1e0
1		 1e-1
69		 1e-2
1370		 1e-3
14850		 #succ
10/10
DEMO Tusar bbob-biobj 4.2 (5)8.0 (4)3.7 (4)7.4 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.3 (0.8)1.0 (0.4)1.0 (0.8)1.9 (3)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 974 (1033)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.3 (6)3.2 (5)2.8 (2)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 7.3 (0)9.3 (4)11 (8)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 7.5 (0)7.4 (4)∞∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 135 (264)13 (2)4.0 (2)22 (25)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.7e4 (5e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.8 (2)265 (629)286 (296)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.7 (3)34 (46)625 (988)∞3e60/10
RM-MEDA Auger bbob-biobj 740 (381)26 (2)2.1 (0.2)1.0 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 645 (226)21 (2)61 (165)52 (34)0/10
SMS-EMOA-PM Auger bbob-biobj 410 (109)19 (3)75 (84)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.9 (4)13 (12)8.5 (3)2.3 (0.1)10/10

Δ HVref
f36, 3-D		 1e0
2.0		 1e-1
63		 1e-2
690		 1e-3
7454		 #succ
10/10
DEMO Tusar bbob-biobj 25 (112)8.9 (10)10 (18)12 (14)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.95 (0.2)1 (0.7)1.00 (0.4)1.00 (0.4)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 1044 (757)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.3 (14)6.7 (24)3.0 (2)∞30010/10
MO-DIRECT-ND Wong bbob-biobj 26 (57)13 (6)14 (9)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 17 (62)10 (6)∞∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 79 (200)14 (3)5.9 (1)25 (12)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.6e4 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 10 (22)62 (23)305 (595)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.7 (12)86 (183)750 (2363)3987 (5433)0/10
RM-MEDA Auger bbob-biobj 403 (179)28 (6)4.0 (0.5)1.9 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 321 (224)24 (5)4.8 (2)77 (69)0/10
SMS-EMOA-PM Auger bbob-biobj 275 (130)27 (17)15 (15)117 (101)0/10
UP-MO-CMA-ES Krause bbob-biobj 7.9 (16)13 (23)12 (5)4.4 (0.3)10/10

Δ HVref
f37, 3-D		 1e0
2.0		 1e-1
730		 1e-2
12388		 1e-3
1.8e5		 #succ
4/10
DEMO Tusar bbob-biobj 5.0 (5)1.3 (0.7)1.4 (0.9)2.8 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.85 (0.9)1.7 (4)3.4 (3)1.00 (1)4/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (0)1.00 (2)2.4 (4)∞30010/10
MO-DIRECT-ND Wong bbob-biobj 6.6 (16)1.7 (1)∞∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 4.9 (14)1.8 (1)∞∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 87 (115)1.8 (0.7)1.6 (2)2.1 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 7249 (5928)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.7 (2)12 (15)224 (194)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.4 (2)20 (41)408 (419)∞3e60/10
RM-MEDA Auger bbob-biobj 414 (354)3.1 (1)1.0 (0.4)1.8 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 319 (111)2.5 (0.9)1.4 (1)2.3 (2)0/10
SMS-EMOA-PM Auger bbob-biobj 214 (102)3.4 (1)22 (8)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.9 (6)2.4 (2)1.7 (0.6)40 (55)1/10

Δ HVref
f38, 3-D		 1e0
2.0		 1e-1
639		 1e-2
15065		 1e-3
4.5e5		 #succ
0/10
DEMO Tusar bbob-biobj 3.5 (3)1.6 (2)3.1 (4)3.1 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.75 (0)2.2 (3)4.4 (8)1.6 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.6 (6)1.00 (2)∞∞30020/10
MO-DIRECT-ND Wong bbob-biobj 3.3 (0)2.1 (3)∞∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 2.7 (0)2.2 (2)∞∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 64 (120)2.1 (0.8)3.3 (1)1.4 (1)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.8e4 (9623)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (1)11 (4)181 (264)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.9 (3)38 (86)167 (83)∞3e60/10
RM-MEDA Auger bbob-biobj 439 (197)3.5 (2)1.00 (0.3)1.3 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 300 (99)3.1 (1)1.7 (1)1.00 (1)0/10
SMS-EMOA-PM Auger bbob-biobj 238 (76)4.7 (5)13 (6)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.9 (6)2.5 (3)25 (100)4.7 (7)0/10

Δ HVref
f39, 3-D		 1e0
3.0		 1e-1
271		 1e-2
1604		 1e-3
32411		 #succ
9/10
DEMO Tusar bbob-biobj 4.9 (3)2.3 (2)2.0 (0.6)2.2 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.3 (2)24 (59)7.9 (38)1.3 (3)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.1 (9)1.00 (0.7)1.0 (0.6)∞30020/10
MO-DIRECT-ND Wong bbob-biobj 20 (32)3.2 (4)2.7 (4)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 15 (26)2.8 (6)6.1 (4)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 73 (83)13 (47)3.0 (8)3.3 (5)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.0e4 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (1)174 (385)214 (94)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 4.7 (5)99 (240)608 (1023)907 (1226)0/10
RM-MEDA Auger bbob-biobj 348 (54)7.5 (4)2.1 (0.9)1.3 (0.1)0/10
SMS-EMOA-DE Auger bbob-biobj 226 (84)5.9 (3)82 (94)40 (32)0/10
SMS-EMOA-PM Auger bbob-biobj 153 (44)5.3 (3)67 (63)86 (35)0/10
UP-MO-CMA-ES Krause bbob-biobj 4.4 (8)3.6 (5)5.4 (4)1.0 (0.3)9/10

Δ HVref
f40, 3-D		 1e0
6.0		 1e-1
899		 1e-2
8116		 1e-3
31550		 #succ
10/10
DEMO Tusar bbob-biobj 2.6 (2)1.2 (0.5)1.00 (0.8)10 (15)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (2)6.1 (18)7.7 (8)3.0 (2)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 362 (615)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.9 (3)2.0 (3)1.8 (2)∞30010/10
MO-DIRECT-ND Wong bbob-biobj 8.6 (8)1.4 (0.9)1.8 (2)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 11 (9)1.0 (0.2)∞∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 49 (45)1.4 (0.5)2.3 (3)3.8 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 8642 (8486)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (7)4.6 (6)77 (118)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.2 (2)3.5 (3)86 (113)∞3e60/10
RM-MEDA Auger bbob-biobj 175 (16)2.4 (0.2)4.6 (9)3.6 (4)0/10
SMS-EMOA-DE Auger bbob-biobj 121 (31)2.2 (0.3)1.7 (2)5.2 (3)0/10
SMS-EMOA-PM Auger bbob-biobj 78 (32)3.5 (2)7.3 (4)17 (31)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.1 (2)1.5 (1)1.4 (0.6)1 (0.1)10/10

Δ HVref
f41, 3-D		 1e0
4.0		 1e-1
55		 1e-2
456		 1e-3
4400		 #succ
10/10
DEMO Tusar bbob-biobj 4.0 (9)6.3 (7)3.7 (0.9)3.7 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (1)1.00 (1)1.0 (0.5)1.00 (0.5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (3)4.5 (6)1.8 (1.0)3.3 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 12 (12)10 (4)7.7 (6)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 6.2 (8)6.0 (2)9.3 (6)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 49 (70)15 (2)3.6 (0.7)11 (8)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.2e4 (2e4)5.3e4 (6e4)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.3 (0.5)22 (45)157 (226)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.5 (6)25 (83)578 (332)1086 (482)0/10
RM-MEDA Auger bbob-biobj 183 (95)30 (7)5.4 (1)1.2 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 215 (91)28 (7)5.1 (1.0)2.9 (2)0/10
SMS-EMOA-PM Auger bbob-biobj 127 (68)24 (10)13 (6)7.7 (3)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.1 (6)10 (13)12 (7)6.7 (1)10/10

Δ HVref
f42, 3-D		 1e0
2.0		 1e-1
981		 1e-2
9715		 1e-3
2.0e5		 #succ
7/10
DEMO Tusar bbob-biobj 4.7 (4)1 (1)2.2 (3)1.9 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.75 (0)7.1 (30)5.7 (4)1.3 (0.4)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.0 (0)1.1 (1)1.4 (3)∞30020/10
MO-DIRECT-ND Wong bbob-biobj 28 (137)1.4 (2)∞∞30040/10
MO-DIRECT-Rank Wong bbob-biobj 13 (31)1.1 (1)∞∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 102 (54)2.2 (3)2.4 (2)1.3 (0.6)0/10
RANDOMSEARCH-100 Auger bbob-biobj 3.8e4 (6e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 5.6 (8)20 (42)59 (60)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.3 (5)37 (82)228 (198)∞3e60/10
RM-MEDA Auger bbob-biobj 449 (95)2.3 (1)1.00 (0.6)4.6 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 412 (158)1.8 (0.5)1.4 (1)1.00 (0.9)0/10
SMS-EMOA-PM Auger bbob-biobj 273 (95)2.9 (5)27 (55)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 12 (24)1.7 (2)36 (0.4)22 (30)0/10

Δ HVref
f43, 3-D		 1e0
11		 1e-1
971		 1e-2
8225		 1e-3
2.6e5		 #succ
3/10
DEMO Tusar bbob-biobj 1.7 (2)1.00 (0.5)2.2 (2)5.0 (8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.5 (0.3)5.7 (3)1.0 (0.3)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.1 (1)1.0 (1)1.8 (2)∞30020/10
MO-DIRECT-ND Wong bbob-biobj 2.9 (6)1.2 (0.6)∞∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 2.9 (8)1.1 (0.8)∞∞30020/10
NSGA-II-MATLAB Auger bbob-biobj 21 (26)1.2 (0.8)1.2 (0.8)4.9 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 5687 (2507)3082 (6565)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 0.96 (0.4)7.0 (14)66 (37)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (1)7.5 (9)281 (327)∞3e60/10
RM-MEDA Auger bbob-biobj 89 (30)2.1 (0.5)1 (0.6)2.2 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 78 (15)1.8 (0.5)1.4 (1)1.7 (3)0/10
SMS-EMOA-PM Auger bbob-biobj 41 (12)2.3 (2)11 (16)11 (14)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.3 (2)1.8 (2)2.7 (1)28 (11)0/10

Δ HVref
f44, 3-D		 1e0
9.0		 1e-1
519		 1e-2
1675		 1e-3
5358		 #succ
10/10
DEMO Tusar bbob-biobj 13 (32)1.5 (2)1.6 (3)1.9 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)3.4 (16)10 (26)4.4 (11)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 105 (155)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 39 (84)2.6 (9)1.00 (1)1.2 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 15 (21)1.8 (3)1.4 (1)5.5 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 7.3 (9)1.0 (1)1.5 (2)5.5 (5)0/10
NSGA-II-MATLAB Auger bbob-biobj 37 (40)1.8 (1)3.1 (6)5.9 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.7e4 (3e4)2417 (2263)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.2 (6)60 (179)126 (90)226 (281)0/10
RANDOMSEARCH-5 Auger bbob-biobj 8.6 (17)286 (359)668 (800)962 (1293)0/10
RM-MEDA Auger bbob-biobj 115 (42)3.6 (1)1.7 (0.9)1 (0.3)0/10
SMS-EMOA-DE Auger bbob-biobj 83 (48)3.1 (2)1.7 (2)40 (128)0/10
SMS-EMOA-PM Auger bbob-biobj 78 (9)5.0 (1)65 (90)70 (54)0/10
UP-MO-CMA-ES Krause bbob-biobj 6.7 (2)2.7 (5)4.5 (5)65 (143)9/10

Δ HVref
f45, 3-D		 1e0
3.0		 1e-1
280		 1e-2
1904		 1e-3
21523		 #succ
10/10
DEMO Tusar bbob-biobj 1.9 (1)1.5 (1)1.9 (0.4)1.4 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 2.6 (5)1.0 (0.3)20 (19)2.9 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.7 (10)1.5 (3)1 (2)∞30010/10
MO-DIRECT-ND Wong bbob-biobj 11 (16)2.0 (1)2.7 (3)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 12 (15)1.5 (0.5)1.3 (0.3)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 52 (81)2.9 (0.6)4.1 (3)3.8 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 6601 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.3 (2)1.5 (1)55 (16)132 (77)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.2 (2)1.4 (1)36 (60)148 (125)0/10
RM-MEDA Auger bbob-biobj 251 (156)6.1 (0.8)1.5 (0.2)3.8 (7)0/10
SMS-EMOA-DE Auger bbob-biobj 225 (49)5.0 (0.9)1.7 (0.7)1.00 (1)0/10
SMS-EMOA-PM Auger bbob-biobj 164 (64)4.4 (2)5.5 (6)2.7 (3)0/10
UP-MO-CMA-ES Krause bbob-biobj 1 (0.3)1.7 (1)3.4 (2)17 (35)9/10

Δ HVref
f46, 3-D		 1e0
3.0		 1e-1
1937		 1e-2
26783		 1e-3
3.2e5		 #succ
8/10
DEMO Tusar bbob-biobj 2.2 (2)1.00 (0.7)1.3 (0.4)1.5 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0.8)3.6 (4)3.1 (1)1.00 (0.5)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.5 (6)2.4 (2)∞∞30030/10
MO-DIRECT-ND Wong bbob-biobj 21 (32)1.1 (0.7)∞∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 15 (23)1.3 (0.3)∞∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 116 (105)1.9 (2)2.1 (0.5)2.9 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.1e4 (2e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 0.90 (0.7)11 (18)∞∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.7 (2)22 (31)337 (442)∞3e60/10
RM-MEDA Auger bbob-biobj 304 (51)1.7 (0.6)1.0 (0.5)4.6 (6)0/10
SMS-EMOA-DE Auger bbob-biobj 247 (102)1.4 (0.6)2.4 (3)2.9 (3)0/10
SMS-EMOA-PM Auger bbob-biobj 185 (34)3.6 (2)50 (49)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.1 (3)1.9 (2)29 (56)∞3e60/10

Δ HVref
f47, 3-D		 1e0
3.0		 1e-1
1404		 1e-2
32368		 1e-3
4.9e5		 #succ
0/10
DEMO Tusar bbob-biobj 2.8 (0.7)1.00 (1)1.7 (2)5.9 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.5 (0.4)8.6 (23)2.7 (1)1.00 (0.2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.0 (5)1.4 (2)∞∞30030/10
MO-DIRECT-ND Wong bbob-biobj 18 (8)1.2 (0.3)∞∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 11 (19)1.2 (0.8)∞∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 54 (46)1.7 (2)2.0 (5)∞3e50/10
RANDOMSEARCH-100 Auger bbob-biobj 1.2e4 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (6)31 (12)42 (32)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 0.90 (0.7)39 (86)232 (211)∞3e60/10
RM-MEDA Auger bbob-biobj 303 (166)1.8 (1.0)1.1 (0.9)2.9 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 241 (61)1.6 (0.9)1.0 (0.4)∞3e50/10
SMS-EMOA-PM Auger bbob-biobj 144 (34)3.7 (6)11 (12)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 5.2 (2)2.1 (3)15 (7)59 (47)0/10

Δ HVref
f48, 3-D		 1e0
5.0		 1e-1
483		 1e-2
24290		 1e-3
1.7e5		 #succ
9/10
DEMO Tusar bbob-biobj 3.0 (2)2.4 (3)1.0 (0.5)2.0 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0.7)6.0 (4)2.5 (3)1.3 (2)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.1 (4)1.0 (2)1.2 (2)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 16 (19)3.0 (5)∞∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 8.7 (15)1.7 (1)∞∞30040/10
NSGA-II-MATLAB Auger bbob-biobj 45 (73)3.1 (5)2.1 (0.5)1.00 (0.3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.6e4 (2e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.9 (3)63 (252)116 (148)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.5 (0.9)165 (653)95 (154)∞3e60/10
RM-MEDA Auger bbob-biobj 194 (48)4.9 (3)3.4 (0.2)3.0 (4)0/10
SMS-EMOA-DE Auger bbob-biobj 157 (59)4.0 (1)3.5 (6)16 (17)0/10
SMS-EMOA-PM Auger bbob-biobj 88 (21)5.8 (7)31 (59)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.5 (1)3.6 (4)15 (62)18 (26)1/10

Δ HVref
f49, 3-D		 1e0
11		 1e-1
1489		 1e-2
12838		 1e-3
1.3e5		 #succ
9/10
DEMO Tusar bbob-biobj 3.5 (3)1.3 (0.9)3.6 (1.0)10 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.98 (1)1.9 (1)5.5 (4)1.00 (0.4)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.4 (2)1.5 (2)∞∞30020/10
MO-DIRECT-ND Wong bbob-biobj 7.2 (12)1.0 (0.4)2.3 (2)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 6.8 (3)1.3 (1.0)2.3 (5)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 32 (31)1.0 (0.5)2.3 (3)1.6 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 7810 (6273)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 5.7 (0.9)15 (21)230 (298)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.6 (3)45 (42)290 (467)∞3e60/10
RM-MEDA Auger bbob-biobj 109 (32)1.9 (0.7)3.3 (12)1.7 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 78 (33)1.5 (0.5)1.00 (0.9)2.9 (5)0/10
SMS-EMOA-PM Auger bbob-biobj 50 (14)2.1 (3)22 (74)21 (17)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.9 (0.8)1.7 (1)28 (0.7)90 (151)0/10

Δ HVref
f50, 3-D		 1e0
5.0		 1e-1
1434		 1e-2
28762		 1e-3
5.5e5		 #succ
1/10
DEMO Tusar bbob-biobj 2.5 (3)1.5 (2)1.4 (0.8)2.5 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (1)3.1 (4)2.2 (2)1 (1)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.0 (2)1.7 (2)1 (1)∞30010/10
MO-DIRECT-ND Wong bbob-biobj 14 (23)1.0 (0.8)1.0 (1)∞30010/10
MO-DIRECT-Rank Wong bbob-biobj 8.8 (15)1.1 (0.9)∞∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 73 (33)1.1 (0.7)1.7 (4)2.5 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.1e4 (6233)1916 (3190)∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.7 (2)22 (25)43 (42)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.5 (5)26 (20)192 (307)52 (92)0/10
RM-MEDA Auger bbob-biobj 217 (75)1.8 (1)1.1 (2)2.5 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 155 (61)1.6 (1)1.7 (1)1.1 (1)0/10
SMS-EMOA-PM Auger bbob-biobj 97 (29)1.9 (2)6.7 (6)5.4 (3)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.2 (2)2.3 (2)46 (78)22 (11)0/10

Δ HVref
f51, 3-D		 1e0
4.0		 1e-1
922		 1e-2
8936		 1e-3
1.4e5		 #succ
3/10
DEMO Tusar bbob-biobj 6.8 (8)1.2 (3)4.6 (1)3.0 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (0.3)9.0 (39)3.5 (6)1.0 (0.3)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.8 (4)1.3 (2)1.0 (0.7)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 11 (8)1.2 (2)1.0 (0.8)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 6.7 (16)1.0 (0.9)1.0 (0.5)∞30020/10
NSGA-II-MATLAB Auger bbob-biobj 66 (88)2.8 (9)2.3 (5)2.5 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 3.1e4 (4e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 6.4 (13)39 (164)83 (42)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 7.6 (16)88 (16)421 (367)208 (142)0/10
RM-MEDA Auger bbob-biobj 265 (83)2.4 (0.7)4.3 (0.1)1.5 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 191 (77)2.0 (1)1.8 (1)3.6 (7)0/10
SMS-EMOA-PM Auger bbob-biobj 118 (24)3.0 (9)12 (11)∞3e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.9 (3)1.9 (0.6)39 (168)15 (27)0/10

Δ HVref
f52, 3-D		 1e0
3.0		 1e-1
845		 1e-2
14105		 1e-3
1.4e5		 #succ
3/10
DEMO Tusar bbob-biobj 2.6 (2)1.3 (0.8)1 (0.4)4.1 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)8.2 (9)4.9 (0.8)1 (0.8)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 690 (750)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.1 (5)1.5 (1)2.1 (0.4)∞30030/10
MO-DIRECT-ND Wong bbob-biobj 15 (21)1.1 (0.6)2.1 (3)∞30020/10
MO-DIRECT-Rank Wong bbob-biobj 16 (16)1.00 (0.6)2.1 (4)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 90 (133)3.3 (5)4.7 (3)4.0 (6)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.7e4 (2e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (1)42 (3)193 (303)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (4)7.8 (8)99 (122)∞3e60/10
RM-MEDA Auger bbob-biobj 329 (99)2.5 (0.3)10 (17)3.0 (5)0/10
SMS-EMOA-DE Auger bbob-biobj 227 (95)2.2 (0.7)10 (11)4.1 (3)0/10
SMS-EMOA-PM Auger bbob-biobj 167 (30)2.5 (0.7)4.6 (17)10 (9)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.4 (5)2.0 (2)1.9 (3)25 (53)0/10

Δ HVref
f53, 3-D		 1e0
2.0		 1e-1
6.0		 1e-2
32		 1e-3
240		 #succ
10/10
DEMO Tusar bbob-biobj 5.3 (8)6.9 (8)8.5 (6)4.3 (0.9)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (1)0.93 (0.5)1.0 (0.5)1 (0.7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 491 (672)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.1 (6)3.8 (0.2)1.7 (0.5)1.3 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 8.6 (0)10 (0.6)8.0 (7)3.9 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 7.7 (12)7.4 (0.6)11 (5)15 (27)0/10
NSGA-II-MATLAB Auger bbob-biobj 31 (154)91 (25)22 (2)4.5 (1)3/10
RANDOMSEARCH-100 Auger bbob-biobj 7464 (2e4)4.6e4 (6e4)9.3e4 (1e5)∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (2)2.8 (2)4.4 (2)18 (5)0/10
RANDOMSEARCH-5 Auger bbob-biobj 4.5 (5)6.2 (5)6.2 (2)28 (4)0/10
RM-MEDA Auger bbob-biobj 357 (126)205 (43)50 (6)9.1 (1)2/10
SMS-EMOA-DE Auger bbob-biobj 343 (104)166 (14)40 (7)7.7 (0.8)0/10
SMS-EMOA-PM Auger bbob-biobj 214 (88)115 (25)28 (6)7.7 (0.4)3/10
UP-MO-CMA-ES Krause bbob-biobj 3.1 (1)8.4 (10)8.1 (4)10 (3)8/10

Δ HVref
f54, 3-D		 1e0
4.0		 1e-1
193		 1e-2
2668		 1e-3
17832		 #succ
10/10
DEMO Tusar bbob-biobj 3.6 (4)2.2 (2)1.2 (0.4)4.5 (3)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (0.8)6.9 (8)11 (14)5.7 (8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj 517 (1181)∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.2 (3)2.6 (1)1.8 (2)1.6 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 13 (12)2.0 (1)1.4 (1)1.7 (2)0/10
MO-DIRECT-Rank Wong bbob-biobj 12 (5)2.2 (1)1.1 (0.5)∞30030/10
NSGA-II-MATLAB Auger bbob-biobj 41 (58)16 (1)6.5 (8)3.5 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.9e4 (1e4)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (1)1.0 (0.6)19 (34)34 (23)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.8 (8)1.7 (1.0)14 (18)73 (57)0/10
RM-MEDA Auger bbob-biobj 237 (39)8.0 (0.9)14 (0.2)4.5 (8)0/10
SMS-EMOA-DE Auger bbob-biobj 185 (56)6.5 (1)1.0 (0.3)17 (29)0/10
SMS-EMOA-PM Auger bbob-biobj 133 (35)5.3 (1)3.5 (3)9.4 (13)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.1 (5)2.3 (2)1.6 (1)1.0 (0.7)9/10

Δ HVref
f55, 3-D		 1e0
3.0		 1e-1
564		 1e-2
2563		 1e-3
94621		 #succ
10/10
DEMO Tusar bbob-biobj 16 (67)2.3 (1)1.8 (0.5)1.5 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0.8)21 (46)17 (14)1.0 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞2250/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.5 (10)1.7 (1)1.6 (1)∞30040/10
MO-DIRECT-ND Wong bbob-biobj 14 (4)1.00 (0.5)1.6 (1)∞30030/10
MO-DIRECT-Rank Wong bbob-biobj 15 (20)1.3 (0.4)1.0 (1)∞30010/10
NSGA-II-MATLAB Auger bbob-biobj 79 (53)4.1 (10)5.0 (6)1.3 (2)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.2e4 (5314)∞∞∞3e50/10
RANDOMSEARCH-4 Auger bbob-biobj 8.8 (19)3.5 (5)81 (92)∞3e50/10
RANDOMSEARCH-5 Auger bbob-biobj 9.1 (19)5.2 (5)72 (121)82 (104)0/10
RM-MEDA Auger bbob-biobj 362 (123)4.2 (1)1.5 (0.2)2.2 (6)0/10
SMS-EMOA-DE Auger bbob-biobj 245 (103)3.6 (1)15 (59)1.3 (2)0/10
SMS-EMOA-PM Auger bbob-biobj 177 (114)3.8 (2)27 (43)3.8 (6)0/10
UP-MO-CMA-ES Krause bbob-biobj 15 (9)1.8 (2)2.4 (0.7)3.8 (16)8/10

Previous dimension | Dimension = 5 | Next dimension

Δ HVref
f1, 5-D		 1e0
1		 1e-1
75		 1e-2
584		 1e-3
3660		 #succ
10/10
DEMO Tusar bbob-biobj 3.7 (2)15 (3)6.3 (0.6)5.2 (0.8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (1)0.99 (0.4)★1.0 (0.2)1.00 (0.1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj 1647 (2625)∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.8 (0)2.4 (0.9)3.1 (2)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 31 (151)19 (8)∞∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 38 (92)23 (5)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 39 (94)14 (2)11 (2)35 (9)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.4e6 (4e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.0 (6)28 (10)3940 (6132)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.7 (4)58 (41)6220 (4591)∞5e60/10
RM-MEDA Auger bbob-biobj 1042 (189)26 (1)4.7 (0.3)1.6 (0.1)0/10
SMS-EMOA-DE Auger bbob-biobj 1157 (234)28 (2)5.5 (0.4)1.7 (0.1)10/10
SMS-EMOA-PM Auger bbob-biobj 569 (199)20 (3)16 (4)56 (78)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.8 (3)15 (9)25 (2)15 (0.1)10/10

Δ HVref
f2, 5-D		 1e0
5.0		 1e-1
105		 1e-2
601		 1e-3
3715		 #succ
10/10
DEMO Tusar bbob-biobj 4.2 (8)12 (6)6.4 (1)3.7 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.92 (2)1.0 (0.3)1.0 (0.3)1.00 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.5 (11)3.6 (4)7.3 (6)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 65 (113)35 (12)∞∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 50 (120)22 (13)∞∞50010/10
NSGA-II-MATLAB Auger bbob-biobj 63 (84)11 (2)4.1 (0.7)36 (50)0/10
RANDOMSEARCH-100 Auger bbob-biobj 9.7e5 (4e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 5.2 (9)91 (17)937 (1456)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.8 (1)171 (300)6439 (8478)∞5e60/10
RM-MEDA Auger bbob-biobj 385 (173)35 (7)8.9 (3)18 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 310 (90)26 (3)561 (624)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 131 (45)16 (4)90 (72)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 5.3 (13)16 (15)19 (12)13 (2)10/10

Δ HVref
f3, 5-D		 1e0
3.0		 1e-1
115		 1e-2
665		 1e-3
5170		 #succ
10/10
DEMO Tusar bbob-biobj 6.1 (5)11 (6)6.3 (2)6.3 (8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (1)1.00 (1)1.00 (0.5)1.00 (0.9)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.9 (18)3.8 (1)3.7 (5)∞50030/10
MO-DIRECT-ND Wong bbob-biobj 13 (28)25 (16)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 23 (74)20 (4)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 89 (153)22 (5)14 (21)93 (249)0/10
RANDOMSEARCH-100 Auger bbob-biobj 7.1e5 (2e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 0.97 (0.9)47 (38)3497 (3971)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 4.0 (7)95 (190)5885 (5117)∞5e60/10
RM-MEDA Auger bbob-biobj 443 (183)20 (5)5.0 (0.5)1.5 (1.0)0/10
SMS-EMOA-DE Auger bbob-biobj 462 (100)21 (4)89 (189)874 (629)0/10
SMS-EMOA-PM Auger bbob-biobj 263 (55)15 (3)1782 (3379)948 (1136)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.1 (3)11 (9)20 (9)10 (1.0)10/10

Δ HVref
f4, 5-D		 1e0
2.0		 1e-1
109		 1e-2
571		 1e-3
2669		 #succ
10/10
DEMO Tusar bbob-biobj 10 (22)10 (4)6.6 (0.8)5.4 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.85 (0.9)1.00 (0.5)★1.0 (0.3)1.00 (0.5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.9 (32)5.3 (6)3.4 (2)∞50030/10
MO-DIRECT-ND Wong bbob-biobj 50 (123)23 (25)86 (129)∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 29 (71)14 (7)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 171 (194)10 (1)13 (0.8)46 (26)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.3e6 (3e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.1 (3)26 (45)1329 (1471)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 8.4 (5)64 (33)5873 (8925)∞5e60/10
RM-MEDA Auger bbob-biobj 604 (126)20 (3)5.4 (0.4)2.0 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 709 (105)22 (3)225 (219)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 357 (28)15 (2)1362 (1532)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 6.1 (12)11 (9)17 (5)19 (0.6)10/10

Δ HVref
f5, 5-D		 1e0
2.0		 1e-1
120		 1e-2
1277		 1e-3
21889		 #succ
10/10
DEMO Tusar bbob-biobj 1.8 (2)13 (2)8.5 (10)62 (66)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.80 (0)1.00 (0.3)★1.0 (0.4)1.0 (2)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.1 (0)3.4 (3)6.8 (3)∞50020/10
MO-DIRECT-ND Wong bbob-biobj 7.0 (0)19 (5)∞∞50010/10
MO-DIRECT-Rank Wong bbob-biobj 6.7 (0)17 (6)∞∞50020/10
NSGA-II-MATLAB Auger bbob-biobj 86 (150)11 (2)19 (15)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0.6)60 (46)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.8 (0.8)161 (328)∞∞5e60/10
RM-MEDA Auger bbob-biobj 526 (156)19 (1)2.9 (0.5)17 (29)0/10
SMS-EMOA-DE Auger bbob-biobj 564 (102)20 (0.7)48 (1)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 292 (73)23 (15)49 (22)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.8 (1)19 (11)20 (5)2.7 (0.2)10/10

Δ HVref
f6, 5-D		 1e0
3.0		 1e-1
73		 1e-2
688		 1e-3
3976		 #succ
10/10
DEMO Tusar bbob-biobj 11 (25)17 (9)6.2 (0.9)11 (5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (1)1.00 (1)1.0 (0.5)1.0 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.1 (8)2.5 (0.7)3.9 (2)∞50030/10
MO-DIRECT-ND Wong bbob-biobj 41 (147)37 (15)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 15 (31)19 (8)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 104 (142)18 (6)11 (6)114 (54)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.7 (8)177 (368)1469 (1438)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 4.8 (2)249 (694)1.2e4 (3e4)∞5e60/10
RM-MEDA Auger bbob-biobj 471 (84)32 (5)4.7 (0.4)1.7 (0.6)0/10
SMS-EMOA-DE Auger bbob-biobj 507 (158)35 (7)5.7 (0.5)3.2 (0.5)0/10
SMS-EMOA-PM Auger bbob-biobj 244 (63)27 (11)21 (16)50 (70)0/10
UP-MO-CMA-ES Krause bbob-biobj 6.9 (14)24 (17)21 (8)13 (0.5)10/10

Δ HVref
f7, 5-D		 1e0
2.0		 1e-1
2763		 1e-2
1.2e5		 1e-3
3.5e6		 #succ
0/10
DEMO Tusar bbob-biobj 2.5 (4)1.2 (1)3.0 (3)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 0.80 (0)1.0 (2)1.00 (0.6)1.00 (0.8)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.1 (4)1.1 (2)∞∞50070/10
MO-DIRECT-ND Wong bbob-biobj 6.5 (0)2.1 (0.8)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 10 (23)2.1 (5)∞∞50020/10
NSGA-II-MATLAB Auger bbob-biobj 80 (162)3.2 (10)4.2 (4)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (0.2)90 (272)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.9 (2)285 (146)∞∞5e60/10
RM-MEDA Auger bbob-biobj 593 (95)1.1 (0.4)2.7 (5)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 681 (143)1.3 (0.5)1.7 (3)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 318 (92)2.2 (2)19 (19)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.8 (4)2.5 (2)28 (30)∞5e60/10

Δ HVref
f8, 5-D		 1e0
3.0		 1e-1
2167		 1e-2
1.6e5		 1e-3
2.1e6		 #succ
0/10
DEMO Tusar bbob-biobj 3.4 (3)1.6 (0.9)1.8 (2)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.8)4.3 (3)1.0 (0.3)1.00 (0.9)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.0 (11)1.5 (2)∞∞50030/10
MO-DIRECT-ND Wong bbob-biobj 55 (94)3.1 (4)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 27 (42)3.2 (3)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 142 (110)1.0 (0.3)4.6 (4)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.6 (2)194 (280)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 14 (2)408 (480)∞∞5e60/10
RM-MEDA Auger bbob-biobj 440 (67)1.3 (0.4)1.2 (0.9)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 467 (74)1.6 (0.1)1.0 (2)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 265 (44)4.8 (8)14 (6)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 7.7 (2)3.4 (3)49 (63)∞5e60/10

Δ HVref
f9, 5-D		 1e0
4.0		 1e-1
96		 1e-2
521		 1e-3
1986		 #succ
10/10
DEMO Tusar bbob-biobj 2.5 (1)11 (7)6.5 (1)4.5 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)1.0 (0.6)1 (0.3)1.00 (0.2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.3 (3)3.7 (4)3.3 (2)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 14 (51)14 (8)91 (62)∞50020/10
MO-DIRECT-Rank Wong bbob-biobj 10 (32)12 (8)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 67 (88)25 (2)11 (19)30 (45)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 5.6 (12)29 (48)1460 (972)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.1 (3)65 (49)4672 (1456)∞5e60/10
RM-MEDA Auger bbob-biobj 288 (81)21 (2)5.5 (0.4)2.3 (0.2)0/10
SMS-EMOA-DE Auger bbob-biobj 332 (36)22 (3)6.0 (0.5)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 155 (56)15 (3)64 (144)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.8 (5)12 (13)18 (5)23 (6)7/10

Δ HVref
f10, 5-D		 1e0
4.0		 1e-1
323		 1e-2
9839		 1e-3
52107		 #succ
10/10
DEMO Tusar bbob-biobj 7.0 (15)6.7 (1)1.0 (0.3)7.8 (10)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.9)1.00 (2)4.0 (5)2.6 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.8 (12)2.3 (2)2.4 (2)∞50030/10
MO-DIRECT-ND Wong bbob-biobj 34 (24)7.0 (4)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 30 (19)6.4 (2)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 95 (110)10 (1)6.1 (9)10 (5)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.2e6 (2e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.5 (5)127 (36)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.2 (6)98 (145)1426 (762)∞5e60/10
RM-MEDA Auger bbob-biobj 285 (36)7.7 (1)13 (26)15 (22)0/10
SMS-EMOA-DE Auger bbob-biobj 331 (54)8.6 (1)15 (26)15 (14)0/10
SMS-EMOA-PM Auger bbob-biobj 187 (73)10 (7)18 (40)25 (34)0/10
UP-MO-CMA-ES Krause bbob-biobj 3.4 (9)10 (4)1.9 (0.3)1.0 (0.0)10/10

Δ HVref
f11, 5-D		 1e0
5.0		 1e-1
56		 1e-2
436		 1e-3
9189		 #succ
9/10
DEMO Tusar bbob-biobj 2.7 (1)4.9 (7)2.9 (5)3.0 (5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.3 (2)1.0 (0.8)1.0 (0.3)1 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 66 (113)∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj 60 (40)∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 31 (4)17 (19)∞∞50030/10
MO-DIRECT-ND Wong bbob-biobj 25 (31)32 (24)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 63 (40)247 (316)∞∞50060/10
NSGA-II-MATLAB Auger bbob-biobj 28 (42)11 (4)3.2 (4)5.7 (11)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.3e4 (3e4)5629 (4815)∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (2)16 (38)148 (612)29 (44)0/10
RANDOMSEARCH-5 Auger bbob-biobj 0.98 (2)24 (98)551 (1363)191 (640)0/10
RM-MEDA Auger bbob-biobj 189 (482)96 (141)38 (16)13 (20)0/10
SMS-EMOA-DE Auger bbob-biobj 89 (113)19 (7)4.7 (2)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 65 (54)16 (5)31 (6)54 (53)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.0 (4)4.6 (5)7.5 (10)3.9 (2)9/10

Δ HVref
f12, 5-D		 1e0
5.0		 1e-1
44		 1e-2
641		 1e-3
3991		 #succ
10/10
DEMO Tusar bbob-biobj 3.5 (5)17 (37)3.7 (3)3.6 (9)5/10
HMO-CMA-ES Loshchilov bbob-biobj 0.94 (0.7)1.00 (1)1.0 (2)1.00 (3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 332 (310)∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj 335 (188)∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 11 (13)18 (58)3.1 (11)1.8 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 18 (15)40 (29)6.8 (5)3.9 (6)0/10
MO-DIRECT-Rank Wong bbob-biobj 36 (65)36 (45)5.7 (12)2.7 (3)0/10
NSGA-II-MATLAB Auger bbob-biobj 80 (69)23 (15)11 (24)36 (63)5/10
RANDOMSEARCH-100 Auger bbob-biobj 3.0e4 (1e4)3.0e4 (2e4)∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (1)58 (235)549 (975)197 (223)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.3 (5)116 (32)4091 (7152)875 (965)0/10
RM-MEDA Auger bbob-biobj 271 (305)63 (90)8.4 (5)19 (63)2/10
SMS-EMOA-DE Auger bbob-biobj 147 (80)42 (21)91 (197)55 (0.1)1/10
SMS-EMOA-PM Auger bbob-biobj 76 (74)1281 (2857)90 (390)56 (35)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.4 (3)23 (44)8.7 (10)4.7 (7)10/10

Δ HVref
f13, 5-D		 1e0
7.0		 1e-1
60		 1e-2
560		 1e-3
5582		 #succ
10/10
DEMO Tusar bbob-biobj 6.4 (12)12 (14)4.4 (3)1.6 (2)3/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1)1.0 (2)1.0 (1)1.0 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.0 (9)1.7 (0.7)2.3 (4)2.5 (3)0/10
MO-DIRECT-ND Wong bbob-biobj 86 (103)37 (36)40 (26)∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 29 (17)13 (11)12 (10)∞50060/10
NSGA-II-MATLAB Auger bbob-biobj 65 (51)15 (7)9.4 (1)10 (37)3/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.5 (3)22 (64)67 (136)228 (291)0/10
RANDOMSEARCH-5 Auger bbob-biobj 5.6 (8)35 (60)529 (534)1175 (1260)0/10
RM-MEDA Auger bbob-biobj 269 (86)44 (12)6.9 (2)1.5 (1.0)3/10
SMS-EMOA-DE Auger bbob-biobj 248 (50)36 (5)5.5 (1)39 (135)0/10
SMS-EMOA-PM Auger bbob-biobj 96 (29)18 (5)3.2 (1)60 (112)0/10
UP-MO-CMA-ES Krause bbob-biobj 2.6 (5)10 (10)6.7 (6)3.4 (0.9)10/10

Δ HVref
f14, 5-D		 1e0
5.0		 1e-1
247		 1e-2
1713		 1e-3
12801		 #succ
10/10
DEMO Tusar bbob-biobj 2.2 (2)10 (5)60 (39)32 (32)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.94 (0.7)1.0 (1.0)1 (0.6)1.0 (0.4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.0 (6)7.0 (4)∞∞50030/10
MO-DIRECT-ND Wong bbob-biobj 16 (14)196 (96)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 11 (13)26 (13)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 66 (64)13 (17)45 (92)378 (566)0/10
RANDOMSEARCH-100 Auger bbob-biobj 4.2e5 (5e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.6)730 (830)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.3 (2)1190 (818)∞∞5e60/10
RM-MEDA Auger bbob-biobj 303 (64)18 (4)4.5 (1.0)12 (11)0/10
SMS-EMOA-DE Auger bbob-biobj 269 (82)882 (2028)∞∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 148 (66)1443 (2657)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.1 (0.6)22 (13)19 (8)4.2 (0.2)10/10

Δ HVref
f15, 5-D		 1e0
6.0		 1e-1
74		 1e-2
677		 1e-3
6559		 #succ
10/10
DEMO Tusar bbob-biobj 15 (33)19 (14)7.1 (2)4.5 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1)1.0 (1)1.00 (0.5)1.00 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.8 (10)12 (16)22 (22)∞50010/10
MO-DIRECT-ND Wong bbob-biobj 82 (113)65 (52)73 (67)∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 23 (29)38 (62)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 80 (95)32 (41)11 (17)28 (37)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.6e5 (2e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 14 (32)166 (231)1338 (2280)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 9.0 (16)332 (856)7207 (5858)∞5e60/10
RM-MEDA Auger bbob-biobj 344 (171)54 (17)9.5 (4)10 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 300 (101)795 (1689)1731 (2216)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 145 (92)28 (16)1732 (3692)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 18 (42)23 (22)17 (8)7.1 (1)10/10

Δ HVref
f16, 5-D		 1e0
3.0		 1e-1
2810		 1e-2
2.0e5		 1e-3
4.2e6		 #succ
3/10
DEMO Tusar bbob-biobj 3.2 (5)1.7 (3)10 (15)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.2 (1)1.0 (2)1.00 (0.6)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.5 (8)1 (1)∞∞50060/10
MO-DIRECT-ND Wong bbob-biobj 17 (24)5.0 (3)∞∞50060/10
MO-DIRECT-Rank Wong bbob-biobj 13 (22)2.4 (5)∞∞50020/10
NSGA-II-MATLAB Auger bbob-biobj 135 (134)4.0 (14)6.5 (7)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 6.9e5 (6e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (0.8)43 (102)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.9 (2)138 (241)231 (221)∞5e60/10
RM-MEDA Auger bbob-biobj 645 (193)2.1 (0.8)3.1 (6)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 541 (140)46 (89)∞∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 238 (101)26 (51)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.5 (2)2.6 (5)58 (123)∞5e60/10

Δ HVref
f17, 5-D		 1e0
29		 1e-1
2935		 1e-2
48624		 1e-3
2.4e6		 #succ
3/10
DEMO Tusar bbob-biobj 11 (1)1.6 (0.7)10 (17)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.00 (2)5.7 (10)3.0 (2)1.0 (1)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.2 (3)1.00 (1)1 (2)∞50040/10
MO-DIRECT-ND Wong bbob-biobj 24 (148)2.2 (2)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 21 (47)1.3 (1)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 16 (17)3.4 (4)10 (23)2.0 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.6e5 (3e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 51 (126)75 (87)94 (237)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 846 (4227)539 (869)933 (2185)∞5e60/10
RM-MEDA Auger bbob-biobj 79 (16)2.3 (0.8)2.1 (5)2.0 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 55 (8)1.5 (0.9)93 (147)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 25 (13)2.9 (0.5)96 (62)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 16 (76)3.2 (2)85 (144)∞5e60/10

Δ HVref
f18, 5-D		 1e0
2.0		 1e-1
56		 1e-2
557		 1e-3
6912		 #succ
8/10
DEMO Tusar bbob-biobj 6.8 (14)4.8 (8)2.9 (3)1.6 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 2.6 (2)1 (0.6)1.00 (1)1.0 (0.5)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj 1763 (936)∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (10)2.8 (1)3.8 (5)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 34 (37)12 (9)14 (8)∞50080/10
MO-DIRECT-Rank Wong bbob-biobj 25 (28)27 (26)∞∞50010/10
NSGA-II-MATLAB Auger bbob-biobj 158 (199)15 (7)2.4 (0.4)41 (31)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.5e6 (3e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.8)3.9 (1)89 (10)196 (333)0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (7)12 (44)384 (1842)953 (1287)0/10
RM-MEDA Auger bbob-biobj 848 (518)53 (14)8.7 (3)1.9 (0.8)0/10
SMS-EMOA-DE Auger bbob-biobj 654 (120)36 (4)104 (225)290 (326)0/10
SMS-EMOA-PM Auger bbob-biobj 304 (106)20 (4)13 (20)316 (465)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.5 (4)5.2 (8)5.0 (5)4.5 (3)8/10

Δ HVref
f19, 5-D		 1e0
9.0		 1e-1
1292		 1e-2
6164		 1e-3
88464		 #succ
9/10
DEMO Tusar bbob-biobj 2.7 (8)2.2 (3)11 (41)1 (0.3)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (0.9)4.6 (10)13 (15)1.6 (2)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.2 (3)1.0 (1)1.00 (0.8)∞50020/10
MO-DIRECT-ND Wong bbob-biobj 26 (56)2.0 (1)4.0 (5)∞50020/10
MO-DIRECT-Rank Wong bbob-biobj 20 (28)1.0 (0.4)4.0 (4)∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 67 (16)18 (43)11 (13)1.8 (1)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 5.7 (13)3.1 (2)155 (166)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.2 (3)10 (4)279 (80)∞5e60/10
RM-MEDA Auger bbob-biobj 221 (92)3.6 (0.6)14 (9)5.8 (18)0/10
SMS-EMOA-DE Auger bbob-biobj 154 (29)45 (0.5)731 (1420)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 72 (23)48 (98)325 (507)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.5 (2)1.3 (0.8)2.1 (2)6.7 (28)9/10

Δ HVref
f20, 5-D		 1e0
4.0		 1e-1
70		 1e-2
1162		 1e-3
5724		 #succ
10/10
DEMO Tusar bbob-biobj 6.1 (4)17 (11)3.7 (3)5.0 (4)2/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.3)1.0 (0.6)★1.00 (1)1.00 (0.9)10/10
MAT-DIRECT Al-Dujaili bbob-biobj 863 (1195)∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj 920 (1617)∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (16)11 (19)3.8 (1)∞50010/10
MO-DIRECT-ND Wong bbob-biobj 36 (63)40 (34)20 (21)∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 82 (161)50 (28)19 (22)∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 761 (92)63 (225)12 (5)153 (131)0/10
RANDOMSEARCH-100 Auger bbob-biobj 3.3e5 (5e5)6.8e4 (6e4)∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 22 (10)377 (1686)722 (1286)787 (1070)0/10
RANDOMSEARCH-5 Auger bbob-biobj 14 (12)321 (649)3138 (4477)3520 (2865)0/10
RM-MEDA Auger bbob-biobj 380 (138)43 (14)4.2 (3)3.5 (9)2/10
SMS-EMOA-DE Auger bbob-biobj 431 (151)41 (17)51 (109)205 (219)0/10
SMS-EMOA-PM Auger bbob-biobj 268 (173)34 (14)304 (216)358 (332)0/10
UP-MO-CMA-ES Krause bbob-biobj 20 (35)20 (19)11 (8)7.6 (2)10/10

Δ HVref
f21, 5-D		 1e0
5.0		 1e-1
86		 1e-2
3249		 1e-3
9924		 #succ
10/10
DEMO Tusar bbob-biobj 8.9 (1)12 (14)1.5 (0.9)2.8 (7)1/10
HMO-CMA-ES Loshchilov bbob-biobj 0.92 (0.7)1.00 (2)2.2 (0.1)1.8 (8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.0 (16)12 (11)1.0 (1)1.0 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 16 (37)30 (28)2.9 (5)∞50060/10
MO-DIRECT-Rank Wong bbob-biobj 38 (40)26 (21)2.7 (4)∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 56 (69)13 (5)19 (3)20 (45)1/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (2)59 (124)231 (385)221 (189)0/10
RANDOMSEARCH-5 Auger bbob-biobj 4.8 (14)94 (195)953 (1801)900 (1125)0/10
RM-MEDA Auger bbob-biobj 291 (74)28 (18)18 (0.7)6.3 (25)1/10
SMS-EMOA-DE Auger bbob-biobj 344 (85)29 (2)39 (0.2)76 (63)0/10
SMS-EMOA-PM Auger bbob-biobj 184 (76)22 (11)22 (46)60 (50)0/10
UP-MO-CMA-ES Krause bbob-biobj 4.1 (1)17 (27)2.8 (3)3.3 (2)10/10

Δ HVref
f22, 5-D		 1e0
3.0		 1e-1
97		 1e-2
1168		 1e-3
12608		 #succ
9/10
DEMO Tusar bbob-biobj 10 (42)19 (19)13 (14)40 (34)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)1.00 (0.2)★1.0 (0.3)1.0 (0.2)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.1 (12)27 (38)43 (79)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 17 (9)62 (102)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 53 (7)34 (25)∞∞50020/10
NSGA-II-MATLAB Auger bbob-biobj 41 (115)16 (5)25 (27)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 3.8e5 (3e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 5.7 (6)623 (33)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 15 (36)3726 (9155)∞∞5e60/10
RM-MEDA Auger bbob-biobj 394 (108)27 (6)4.2 (1)7.7 (14)0/10
SMS-EMOA-DE Auger bbob-biobj 505 (126)31 (5)3867 (3639)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 258 (172)631 (1363)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 8.8 (40)27 (4)23 (4)4.4 (0.3)9/10

Δ HVref
f23, 5-D		 1e0
1		 1e-1
59		 1e-2
618		 1e-3
4410		 #succ
10/10
DEMO Tusar bbob-biobj 6.2 (2)14 (13)7.5 (3)10 (11)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (0)1.0 (1.0)1.0 (0.6)1.0 (0.8)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.3 (0)7.0 (5)5.5 (9)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 6.1 (13)27 (13)39 (57)∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 4.3 (16)34 (36)77 (87)∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 202 (160)23 (8)11 (9)143 (199)0/10
RANDOMSEARCH-100 Auger bbob-biobj 7.0e5 (1e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.6 (8)26 (41)1343 (2010)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.7 (2)66 (94)6671 (5358)∞5e60/10
RM-MEDA Auger bbob-biobj 1475 (433)43 (7)6.0 (1)2.8 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 1683 (394)48 (12)354 (607)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 1268 (1148)117 (186)1895 (4450)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 11 (15)20 (25)21 (11)10 (3)10/10

Δ HVref
f24, 5-D		 1e0
5.0		 1e-1
2347		 1e-2
1.8e5		 1e-3
4.1e6		 #succ
0/10
DEMO Tusar bbob-biobj 3.7 (6)2.0 (2)4.5 (4)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)2.2 (4)1.0 (1)1.00 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.2 (3)1.00 (0.8)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 20 (18)6.6 (7)∞∞50040/10
MO-DIRECT-Rank Wong bbob-biobj 35 (0)10 (7)∞∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 71 (84)5.2 (9)5.7 (3)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (2)59 (64)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (9)351 (1160)∞∞5e60/10
RM-MEDA Auger bbob-biobj 317 (76)1.9 (0.6)2.8 (2)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 380 (75)2.2 (2)26 (29)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 224 (84)56 (107)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.9 (2)3.1 (2)66 (63)∞5e60/10

Δ HVref
f25, 5-D		 1e0
9.0		 1e-1
3143		 1e-2
1.2e5		 1e-3
2.5e6		 #succ
3/10
DEMO Tusar bbob-biobj 1.7 (3)1.2 (1)3.4 (2)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.7 (4)2.3 (4)1.4 (1)1.00 (1)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.1 (7)1.00 (2)∞∞50040/10
MO-DIRECT-ND Wong bbob-biobj 29 (67)2.5 (2)∞∞50020/10
MO-DIRECT-Rank Wong bbob-biobj 17 (46)2.3 (3)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 47 (79)10 (20)7.2 (7)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 2.4e5 (4e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 6.3 (2)81 (137)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.0 (3)470 (821)∞∞5e60/10
RM-MEDA Auger bbob-biobj 174 (20)1.1 (0.4)1.00 (2)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 182 (35)1.3 (0.7)3.4 (8)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 105 (55)3.2 (3)37 (62)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.3 (0.2)2.6 (3)65 (121)18 (14)0/10

Δ HVref
f26, 5-D		 1e0
7.0		 1e-1
59		 1e-2
2182		 1e-3
13673		 #succ
8/10
DEMO Tusar bbob-biobj 12 (19)20 (13)2.5 (4)6.8 (24)1/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (0.6)1.00 (0.5)7.2 (35)22 (0.1)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.3 (4)7.8 (15)1.0 (0.2)1.00 (1)0/10
MO-DIRECT-ND Wong bbob-biobj 40 (57)36 (10)3.7 (7)∞50040/10
MO-DIRECT-Rank Wong bbob-biobj 70 (81)25 (19)3.3 (3)1.1 (1)0/10
NSGA-II-MATLAB Auger bbob-biobj 88 (94)31 (53)1.5 (1.0)7.5 (12)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.2 (6)26 (20)121 (137)94 (215)0/10
RANDOMSEARCH-5 Auger bbob-biobj 6.5 (14)48 (53)677 (1178)864 (1374)0/10
RM-MEDA Auger bbob-biobj 213 (63)38 (5)1.5 (1)9.5 (9)0/10
SMS-EMOA-DE Auger bbob-biobj 266 (44)40 (4)1.6 (0.5)25 (82)0/10
SMS-EMOA-PM Auger bbob-biobj 143 (52)31 (10)29 (59)41 (37)0/10
UP-MO-CMA-ES Krause bbob-biobj 6.0 (12)12 (11)4.6 (4)42 (1)3/10

Δ HVref
f27, 5-D		 1e0
6.0		 1e-1
2631		 1e-2
21971		 1e-3
44576		 #succ
10/10
DEMO Tusar bbob-biobj 29 (3)1.1 (0.8)3.4 (18)13 (18)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1)2.7 (5)3.0 (3)2.7 (5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.0 (10)1.0 (0.2)1.0 (2)1.0 (2)0/10
MO-DIRECT-ND Wong bbob-biobj 23 (10)1.2 (0.9)1.1 (1)∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 41 (47)1.1 (1)1.1 (1)∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 386 (1627)5.5 (6)12 (18)15 (13)0/10
RANDOMSEARCH-100 Auger bbob-biobj 8.0e5 (1e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 20 (48)5.2 (11)100 (182)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 23 (5)10 (5)123 (189)1070 (1094)0/10
RM-MEDA Auger bbob-biobj 224 (37)1.1 (0.2)17 (34)29 (47)0/10
SMS-EMOA-DE Auger bbob-biobj 274 (54)22 (48)15 (23)26 (28)0/10
SMS-EMOA-PM Auger bbob-biobj 142 (50)26 (56)91 (74)101 (95)0/10
UP-MO-CMA-ES Krause bbob-biobj 8.5 (4)1.0 (1)1.00 (0.9)1.0 (0.4)9/10

Δ HVref
f28, 5-D		 1e0
2.0		 1e-1
20		 1e-2
145		 1e-3
1230		 #succ
9/10
DEMO Tusar bbob-biobj 7.8 (14)25 (15)15 (5)5.7 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (1)0.99 (0.5)★21.0 (0.7)★1.0 (0.2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.0 (0)3.9 (2)14 (21)10 (14)0/10
MO-DIRECT-ND Wong bbob-biobj 10 (28)21 (26)15 (8)∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 10 (26)23 (11)29 (29)∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 95 (192)46 (8)16 (5)27 (32)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 11 (45)16 (21)511 (946)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 18 (39)119 (523)2636 (6153)1.9e4 (2e4)0/10
RM-MEDA Auger bbob-biobj 564 (140)95 (17)20 (5)3.6 (0.7)0/10
SMS-EMOA-DE Auger bbob-biobj 727 (81)111 (18)21 (3)631 (1118)0/10
SMS-EMOA-PM Auger bbob-biobj 452 (105)86 (59)261 (588)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 6.6 (8)24 (11)26 (18)18 (3)9/10

Δ HVref
f29, 5-D		 1e0
3.0		 1e-1
114		 1e-2
1413		 1e-3
13660		 #succ
10/10
DEMO Tusar bbob-biobj 24 (58)18 (13)47 (178)51 (23)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.0 (0.6)★1.0 (0.6)1.00 (0.9)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.8 (19)12 (8)∞∞50040/10
MO-DIRECT-ND Wong bbob-biobj 36 (81)40 (23)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 26 (62)26 (11)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 140 (151)36 (59)29 (26)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 6.5 (14)889 (1150)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.6 (2)5946 (3500)∞∞5e60/10
RM-MEDA Auger bbob-biobj 418 (100)23 (2)3.5 (1)6.1 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 444 (143)25 (4)∞∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 233 (64)193 (424)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 6.7 (12)33 (17)20 (9)4.0 (0.3)10/10

Δ HVref
f30, 5-D		 1e0
1		 1e-1
33		 1e-2
366		 1e-3
2294		 #succ
10/10
DEMO Tusar bbob-biobj 26 (52)29 (21)10 (4)8.5 (10)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (0)1.0 (0.3)★1.00 (0.7)★21.00 (0.6)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.7 (9)14 (28)8.4 (4)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 14 (32)50 (42)∞∞50010/10
MO-DIRECT-Rank Wong bbob-biobj 17 (0)28 (19)136 (212)∞50020/10
NSGA-II-MATLAB Auger bbob-biobj 155 (208)92 (146)42 (63)71 (157)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.6 (2)54 (40)1109 (1991)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 15 (10)155 (301)8022 (1e4)∞5e60/10
RM-MEDA Auger bbob-biobj 1395 (289)70 (11)9.4 (2)2.7 (0.5)0/10
SMS-EMOA-DE Auger bbob-biobj 1634 (316)72 (11)162 (1025)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 833 (136)51 (21)651 (1416)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 19 (50)28 (53)25 (20)17 (4)10/10

Δ HVref
f31, 5-D		 1e0
3.0		 1e-1
2166		 1e-2
50028		 1e-3
3.2e6		 #succ
0/10
DEMO Tusar bbob-biobj 7.8 (3)2.9 (7)8.0 (7)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)2.1 (2)3.3 (5)1.00 (0.5)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.5 (12)1.00 (0.8)1 (1)∞50030/10
MO-DIRECT-ND Wong bbob-biobj 62 (13)6.3 (10)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 34 (66)3.5 (4)∞∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 71 (118)7.0 (14)10 (14)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 1.6e6 (1e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.6 (5)87 (132)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.1 (7)480 (850)∞∞5e60/10
RM-MEDA Auger bbob-biobj 463 (104)1.8 (2)8.5 (8)1.5 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 516 (83)2.0 (0.8)41 (30)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 279 (100)5.7 (8)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 7.4 (15)4.2 (5)403 (576)∞5e60/10

Δ HVref
f32, 5-D		 1e0
3.0		 1e-1
2131		 1e-2
1.1e5		 1e-3
2.4e6		 #succ
1/10
DEMO Tusar bbob-biobj 4.3 (7)1.00 (0.8)2.2 (2)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (2)6.7 (4)1.0 (0.6)1.00 (0.6)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.6 (8)1.3 (0.8)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 30 (66)3.1 (3)∞∞50040/10
MO-DIRECT-Rank Wong bbob-biobj 12 (20)1.4 (1)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 73 (107)5.1 (11)8.2 (11)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 1.5e6 (1e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (3)35 (120)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 13 (50)212 (507)∞∞5e60/10
RM-MEDA Auger bbob-biobj 442 (84)1.5 (0.5)1.1 (2)2.0 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 491 (99)1.3 (0.4)6.9 (15)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 239 (70)1.1 (0.2)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 10 (5)2.2 (2)32 (77)20 (12)0/10

Δ HVref
f33, 5-D		 1e0
6.0		 1e-1
627		 1e-2
1214		 1e-3
3730		 #succ
7/10
DEMO Tusar bbob-biobj 23 (99)3.4 (0.9)48 (0.3)17 (2)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.6)22 (55)553 (1243)180 (795)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.9 (5)1.0 (0.1)1.0 (2)1.00 (1.0)0/10
MO-DIRECT-ND Wong bbob-biobj 48 (155)2.3 (4)4.8 (2)13 (10)0/10
MO-DIRECT-Rank Wong bbob-biobj 17 (6)1.6 (0.1)2.2 (2)4.0 (3)0/10
NSGA-II-MATLAB Auger bbob-biobj 64 (71)90 (200)47 (0.1)18 (4)1/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 33 (153)89 (1)112 (110)340 (359)0/10
RANDOMSEARCH-5 Auger bbob-biobj 95 (212)890 (1999)1050 (1059)1758 (1613)0/10
RM-MEDA Auger bbob-biobj 249 (102)92 (200)59 (0.3)20 (9)1/10
SMS-EMOA-DE Auger bbob-biobj 263 (48)92 (200)105 (412)35 (67)0/10
SMS-EMOA-PM Auger bbob-biobj 142 (29)91 (399)55 (19)23 (37)0/10
UP-MO-CMA-ES Krause bbob-biobj 17 (13)27 (0.5)460 (2)153 (671)3/10

Δ HVref
f34, 5-D		 1e0
9.0		 1e-1
1376		 1e-2
15575		 1e-3
54195		 #succ
10/10
DEMO Tusar bbob-biobj 2.1 (2)1.1 (0.6)14 (24)10 (9)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (1)1.00 (0.1)5.0 (7)4.8 (4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.5 (3)1.3 (3)3.1 (3)∞50030/10
MO-DIRECT-ND Wong bbob-biobj 16 (18)2.0 (1)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 13 (15)1.2 (0.6)3.1 (4)∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 21 (32)1.6 (2)5.2 (6)15 (13)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (1)10 (36)90 (119)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.0 (2)8.5 (6)389 (804)∞5e60/10
RM-MEDA Auger bbob-biobj 151 (28)2.1 (0.7)32 (56)37 (67)0/10
SMS-EMOA-DE Auger bbob-biobj 170 (52)1.9 (0.4)14 (16)83 (69)0/10
SMS-EMOA-PM Auger bbob-biobj 83 (19)2.4 (1.0)76 (81)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.8 (3)1.5 (1)1.0 (0.7)1 (0.5)10/10

Δ HVref
f35, 5-D		 1e0
1		 1e-1
141		 1e-2
2268		 1e-3
51388		 #succ
8/10
DEMO Tusar bbob-biobj 4.9 (14)12 (6)33 (114)91 (75)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (1)1.00 (1)1.00 (1)1.0 (1)5/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.5 (8)4.7 (4)6.3 (5)∞50030/10
MO-DIRECT-ND Wong bbob-biobj 19 (45)31 (29)∞∞50040/10
MO-DIRECT-Rank Wong bbob-biobj 19 (36)22 (10)∞∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 64 (158)9.5 (2)29 (37)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 4.8e5 (6e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.9 (1)303 (36)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (0.5)665 (2158)2.1e4 (2e4)∞5e60/10
RM-MEDA Auger bbob-biobj 1079 (274)18 (2)2.2 (0.6)4.1 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 1059 (220)18 (2)362 (357)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 560 (253)28 (43)944 (826)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.5 (5)22 (19)15 (4)1.3 (0.1)8/10

Δ HVref
f36, 5-D		 1e0
2.0		 1e-1
204		 1e-2
4640		 1e-3
41237		 #succ
10/10
DEMO Tusar bbob-biobj 3.4 (2)11 (2)19 (24)14 (14)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.80 (0.5)1.0 (0.8)1.00 (3)1 (0.8)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.1 (0)2.8 (3)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 15 (36)27 (20)∞∞50040/10
MO-DIRECT-Rank Wong bbob-biobj 13 (0)16 (5)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 91 (71)12 (8)20 (54)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 2.3e6 (2e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.8 (6)320 (271)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 4.0 (5)588 (1084)∞∞5e60/10
RM-MEDA Auger bbob-biobj 696 (177)14 (1.0)1.5 (0.2)51 (56)0/10
SMS-EMOA-DE Auger bbob-biobj 680 (302)15 (2)43 (82)19 (21)0/10
SMS-EMOA-PM Auger bbob-biobj 383 (132)16 (9)98 (88)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.2 (4)18 (10)7.3 (2)1.5 (0.4)10/10

Δ HVref
f37, 5-D		 1e0
2.0		 1e-1
3956		 1e-2
1.5e5		 1e-3
2.5e6		 #succ
1/10
DEMO Tusar bbob-biobj 3.0 (4)1.2 (0.4)1.9 (2)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 0.80 (2)1.1 (1)1.0 (1.0)1.00 (0.6)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.6 (3)1 (0.9)∞∞50060/10
MO-DIRECT-ND Wong bbob-biobj 2.3 (0)2.7 (4)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 2.3 (4)5.6 (8)∞∞50020/10
NSGA-II-MATLAB Auger bbob-biobj 73 (121)1.0 (1)5.0 (6)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 1.1e6 (8e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (1)151 (306)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 4.8 (7)970 (640)∞∞5e60/10
RM-MEDA Auger bbob-biobj 624 (161)1.0 (0.5)1.8 (2)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 685 (184)1.2 (0.7)4.2 (4)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 324 (89)14 (28)32 (37)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.5 (3)2.4 (1)23 (34)∞5e60/10

Δ HVref
f38, 5-D		 1e0
4.0		 1e-1
4366		 1e-2
2.7e5		 1e-3
5.4e6		 #succ
1/10
DEMO Tusar bbob-biobj 6.9 (14)1.9 (1)2.4 (1)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.9)4.6 (7)1.00 (0.8)1.00 (1)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.5 (8)2.2 (2)∞∞50030/10
MO-DIRECT-ND Wong bbob-biobj 33 (56)11 (10)∞∞50060/10
MO-DIRECT-Rank Wong bbob-biobj 19 (34)∞∞∞50070/10
NSGA-II-MATLAB Auger bbob-biobj 88 (91)2.2 (2)5.2 (5)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 5.4e5 (8e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.5 (6)283 (485)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (4)877 (1341)∞∞5e60/10
RM-MEDA Auger bbob-biobj 334 (37)1.00 (0.4)2.4 (2)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 361 (123)1.3 (0.8)1.5 (2)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 182 (79)17 (56)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 6.8 (7)3.2 (2)44 (51)∞5e60/10

Δ HVref
f39, 5-D		 1e0
2.0		 1e-1
215		 1e-2
1160		 1e-3
47472		 #succ
8/10
DEMO Tusar bbob-biobj 1.8 (2)7.7 (4)87 (540)13 (25)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.80 (0.8)1.0 (1.0)1.00 (0.5)1.0 (0.1)7/10
MAT-DIRECT Al-Dujaili bbob-biobj 1698 (3794)∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.4 (9)8.6 (12)40 (26)∞50010/10
MO-DIRECT-ND Wong bbob-biobj 13 (12)14 (19)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 6.8 (4)12 (8)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 95 (186)6.8 (3)19 (24)31 (8)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (0.5)230 (638)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.2 (7)321 (830)∞∞5e60/10
RM-MEDA Auger bbob-biobj 597 (224)12 (2)52 (323)3.5 (4)0/10
SMS-EMOA-DE Auger bbob-biobj 665 (142)12 (0.7)1010 (1078)95 (105)0/10
SMS-EMOA-PM Auger bbob-biobj 355 (96)11 (0.8)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.6 (1)14 (4)25 (7)1.1 (0.0)8/10

Δ HVref
f40, 5-D		 1e0
2.0		 1e-1
998		 1e-2
34442		 1e-3
2.0e5		 #succ
8/10
DEMO Tusar bbob-biobj 2.9 (2)4.0 (0.6)2.3 (4)23 (17)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (2)1.00 (3)1.8 (2)1 (0.8)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.1 (8)2.7 (3)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 25 (29)5.3 (2)∞∞50010/10
MO-DIRECT-Rank Wong bbob-biobj 23 (34)4.6 (4)∞∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 161 (178)2.7 (1)4.1 (11)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (1)130 (152)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.7 (7)160 (52)∞∞5e60/10
RM-MEDA Auger bbob-biobj 550 (110)3.0 (0.5)1.8 (0.1)23 (51)0/10
SMS-EMOA-DE Auger bbob-biobj 691 (116)3.3 (0.5)7.3 (0.1)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 351 (87)70 (20)69 (51)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.9 (3)4.7 (1)1.00 (0.2)3.1 (0.0)8/10

Δ HVref
f41, 5-D		 1e0
2.0		 1e-1
48		 1e-2
708		 1e-3
6343		 #succ
10/10
DEMO Tusar bbob-biobj 19 (35)25 (14)7.0 (3)10 (10)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (0)1.0 (0.9)1.0 (0.5)1.0 (0.7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.1 (17)11 (8)6.7 (8)∞50040/10
MO-DIRECT-ND Wong bbob-biobj 26 (40)43 (36)35 (30)∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 28 (40)26 (17)22 (14)∞50010/10
NSGA-II-MATLAB Auger bbob-biobj 173 (186)29 (17)12 (16)81 (67)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.5e6 (2e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.4 (4)291 (640)988 (961)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 9.0 (17)628 (234)7688 (9091)∞5e60/10
RM-MEDA Auger bbob-biobj 798 (138)56 (8)5.5 (0.6)1.4 (0.9)0/10
SMS-EMOA-DE Auger bbob-biobj 971 (381)60 (17)7.0 (2)3.7 (2)0/10
SMS-EMOA-PM Auger bbob-biobj 482 (122)57 (24)27 (14)25 (62)0/10
UP-MO-CMA-ES Krause bbob-biobj 14 (30)42 (37)21 (11)7.8 (1)10/10

Δ HVref
f42, 5-D		 1e0
2.0		 1e-1
2525		 1e-2
3.4e5		 1e-3
4.3e6		 #succ
0/10
DEMO Tusar bbob-biobj 5.9 (11)1.4 (0.8)1.6 (0.8)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.1 (2)1.0 (1)1.00 (0.8)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.2 (9)1.00 (1)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 27 (87)3.1 (2)∞∞50020/10
MO-DIRECT-Rank Wong bbob-biobj 20 (10)4.0 (5)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 220 (230)4.3 (1)2.8 (3)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 1.1e6 (1e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (3)90 (108)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 6.5 (7)645 (1165)∞∞5e60/10
RM-MEDA Auger bbob-biobj 818 (170)1.8 (0.7)1.5 (1)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 863 (155)1.8 (0.7)2.1 (2)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 419 (146)10 (4)14 (17)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 20 (83)3.2 (5)15 (15)∞5e60/10

Δ HVref
f43, 5-D		 1e0
4.0		 1e-1
2468		 1e-2
1.5e5		 1e-3
3.8e6		 #succ
2/10
DEMO Tusar bbob-biobj 2.6 (3)1.00 (0.6)2.3 (2)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 0.95 (1)10 (12)1.3 (2)1.0 (1)2/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.9 (6)1.0 (2)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 41 (135)2.4 (3)∞∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 19 (43)2.1 (3)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 65 (99)6.8 (4)4.3 (2)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 1.2e6 (2e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (3)83 (205)33 (21)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (4)321 (759)314 (495)∞5e60/10
RM-MEDA Auger bbob-biobj 389 (80)1.3 (0.3)1.00 (1)1.3 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 416 (78)1.4 (0.4)1.3 (3)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 210 (35)3.7 (3)8.6 (10)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.0 (1)2.4 (2)35 (34)12 (14)0/10

Δ HVref
f44, 5-D		 1e0
6.0		 1e-1
86		 1e-2
513		 1e-3
1853		 #succ
10/10
DEMO Tusar bbob-biobj 18 (62)14 (12)7.7 (2)7.3 (6)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.9)1.0 (0.5)1.0 (0.5)1.00 (0.5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.9 (5)4.3 (7)4.8 (3)26 (20)0/10
MO-DIRECT-ND Wong bbob-biobj 26 (58)20 (24)45 (54)∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 21 (23)11 (6)30 (30)∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 80 (113)14 (8)7.4 (2)34 (20)0/10
RANDOMSEARCH-100 Auger bbob-biobj 7.9e5 (6e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 11 (1)116 (112)1911 (2175)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 10 (35)1500 (116)9854 (1e4)2.5e4 (2e4)0/10
RM-MEDA Auger bbob-biobj 282 (81)30 (5)6.9 (1)3.0 (0.5)0/10
SMS-EMOA-DE Auger bbob-biobj 288 (67)29 (11)7.2 (0.5)635 (809)0/10
SMS-EMOA-PM Auger bbob-biobj 163 (66)21 (7)374 (544)2522 (2428)0/10
UP-MO-CMA-ES Krause bbob-biobj 22 (53)16 (25)18 (10)21 (4)5/10

Δ HVref
f45, 5-D		 1e0
4.0		 1e-1
816		 1e-2
5730		 1e-3
53653		 #succ
10/10
DEMO Tusar bbob-biobj 10 (0.6)2.8 (0.7)2.4 (0.7)6.0 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)3.5 (0.2)6.8 (10)2.2 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.0 (4)1.0 (1.0)1.9 (2)∞50050/10
MO-DIRECT-ND Wong bbob-biobj 19 (26)3.2 (1)∞∞50020/10
MO-DIRECT-Rank Wong bbob-biobj 16 (16)2.7 (2)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 53 (90)2.3 (0.9)10 (9)90 (91)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 3.8 (3)22 (36)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.5 (5)50 (96)3989 (4915)∞5e60/10
RM-MEDA Auger bbob-biobj 357 (125)3.6 (0.4)1.0 (0.4)10 (19)0/10
SMS-EMOA-DE Auger bbob-biobj 347 (113)3.9 (0.5)40 (45)16 (12)0/10
SMS-EMOA-PM Auger bbob-biobj 213 (37)27 (6)32 (51)27 (27)0/10
UP-MO-CMA-ES Krause bbob-biobj 7.8 (16)3.6 (2)4.0 (1)1.00 (0.0)10/10

Δ HVref
f46, 5-D		 1e0
4.0		 1e-1
11925		 1e-2
4.9e5		 1e-3
5.6e7		 #succ
0/10
DEMO Tusar bbob-biobj 4.2 (3)3.1 (4)2.3 (3)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 0.95 (0.9)1.5 (2)1.00 (0.5)1 (0.9)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.4 (7)3.9 (4)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 37 (42)∞∞∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 25 (26)∞∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 88 (114)3.5 (4)∞∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.3 (7)116 (91)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.3 (11)546 (799)∞∞5e60/10
RM-MEDA Auger bbob-biobj 458 (223)1.3 (0.9)10 (12)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 424 (100)1.0 (0.6)∞∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 244 (63)14 (44)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 7.5 (4)1.6 (1)∞∞5e60/10

Δ HVref
f47, 5-D		 1e0
3.0		 1e-1
4172		 1e-2
2.7e5		 1e-3
4.4e6		 #succ
0/10
DEMO Tusar bbob-biobj 6.0 (11)2.9 (3)5.5 (8)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (2)2.8 (2)1.0 (0.6)1.0 (0.2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.7 (10)1.0 (0.7)∞∞50030/10
MO-DIRECT-ND Wong bbob-biobj 15 (13)5.6 (8)∞∞50020/10
MO-DIRECT-Rank Wong bbob-biobj 14 (29)5.4 (8)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 67 (48)6.8 (26)17 (19)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.8 (2)139 (135)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 2.6 (0.5)512 (920)∞∞5e60/10
RM-MEDA Auger bbob-biobj 452 (67)1.9 (2)7.9 (9)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 468 (128)1.7 (0.8)5.0 (3)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 246 (53)23 (61)∞∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.7 (0.5)3.4 (3)∞∞5e60/10

Δ HVref
f48, 5-D		 1e0
9.0		 1e-1
2160		 1e-2
1.1e5		 1e-3
2.1e6		 #succ
2/10
DEMO Tusar bbob-biobj 12 (4)2.9 (3)2.4 (4)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)3.9 (6)1.00 (1)1.00 (2)2/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.5 (3)1.0 (2)∞∞50030/10
MO-DIRECT-ND Wong bbob-biobj 12 (8)6.1 (8)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 12 (22)1.9 (1)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 35 (41)11 (27)2.7 (5)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 5.4e5 (5e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.8 (4)113 (119)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 14 (60)477 (1394)∞∞5e60/10
RM-MEDA Auger bbob-biobj 161 (26)5.0 (18)3.3 (2)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 174 (53)2.5 (3)18 (20)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 97 (32)5.6 (10)20 (18)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 6.8 (4)3.5 (5)175 (218)∞5e60/10

Δ HVref
f49, 5-D		 1e0
9.0		 1e-1
3737		 1e-2
2.0e5		 1e-3
1.3e6		 #succ
1/10
DEMO Tusar bbob-biobj 5.4 (5)1.8 (2)3.4 (8)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.5 (4)3.8 (0.8)1.0 (0.6)1.00 (1.0)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.8 (4)3.6 (5)∞∞50020/10
MO-DIRECT-ND Wong bbob-biobj 16 (19)2.8 (2)∞∞50010/10
MO-DIRECT-Rank Wong bbob-biobj 13 (20)1.4 (0.8)∞∞50020/10
NSGA-II-MATLAB Auger bbob-biobj 42 (54)7.9 (17)5.2 (5)3.5 (8)0/10
RANDOMSEARCH-100 Auger bbob-biobj 5.5e5 (8e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 0.96 (1)91 (98)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 9.2 (19)265 (196)∞∞5e60/10
RM-MEDA Auger bbob-biobj 144 (31)1.0 (0.3)3.3 (5)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 161 (52)1.3 (1)5.9 (6)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 96 (49)10 (13)23 (14)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 3.4 (6)150 (335)25 (55)∞5e60/10

Δ HVref
f50, 5-D		 1e0
6.0		 1e-1
3913		 1e-2
2.2e5		 1e-3
2.6e6		 #succ
1/10
DEMO Tusar bbob-biobj 4.6 (13)2.0 (0.9)2.4 (1)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)3.0 (4)1.0 (0.2)1.0 (0.4)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (3)1.4 (2)∞∞50040/10
MO-DIRECT-ND Wong bbob-biobj 23 (12)2.0 (2)∞∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 11 (7)2.5 (2)∞∞50030/10
NSGA-II-MATLAB Auger bbob-biobj 39 (68)2.4 (0.4)5.8 (3)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (3)101 (127)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 10 (10)623 (415)∞∞5e60/10
RM-MEDA Auger bbob-biobj 233 (66)1.0 (0.3)1.9 (2)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 238 (58)1.3 (2)1.2 (1)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 126 (57)21 (41)22 (15)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 2.4 (4)3.1 (3)208 (268)∞5e60/10

Δ HVref
f51, 5-D		 1e0
8.0		 1e-1
1251		 1e-2
32465		 1e-3
2.5e6		 #succ
1/10
DEMO Tusar bbob-biobj 2.3 (6)2.0 (2)3.4 (4)∞5e50/10
HMO-CMA-ES Loshchilov bbob-biobj 1.7 (4)5.3 (10)4.2 (2)1.00 (0.6)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.9 (3)1.00 (0.8)∞∞50020/10
MO-DIRECT-ND Wong bbob-biobj 13 (19)3.3 (3)∞∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 7.6 (9)2.4 (3)∞∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 53 (49)2.6 (3)11 (8)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 5.6e5 (6e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 1.8 (5)121 (263)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 8.2 (34)558 (1137)∞∞5e60/10
RM-MEDA Auger bbob-biobj 167 (24)2.1 (0.8)1.00 (1)∞5e50/10
SMS-EMOA-DE Auger bbob-biobj 176 (28)2.3 (1)3.1 (1)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 100 (36)49 (101)39 (35)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.0 (1.0)7.1 (2)156 (270)∞5e60/10

Δ HVref
f52, 5-D		 1e0
2.0		 1e-1
3027		 1e-2
1.4e5		 1e-3
2.1e6		 #succ
1/10
DEMO Tusar bbob-biobj 6.5 (11)1.2 (0.8)1.0 (1)2.3 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (2)6.3 (10)1.0 (0.7)1.0 (2)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.4 (8)1.2 (1)∞∞50050/10
MO-DIRECT-ND Wong bbob-biobj 24 (25)1.2 (1.0)∞∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 23 (26)1.5 (1)∞∞50010/10
NSGA-II-MATLAB Auger bbob-biobj 133 (168)3.2 (6)3.0 (3)∞5e50/10
RANDOMSEARCH-100 Auger bbob-biobj 1.2e6 (9e5)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (0.9)42 (69)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (5)70 (207)318 (373)∞5e60/10
RM-MEDA Auger bbob-biobj 580 (127)1.00 (0.3)1.7 (4)2.3 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 629 (201)1.1 (0.5)2.6 (2)∞5e50/10
SMS-EMOA-PM Auger bbob-biobj 345 (92)17 (38)34 (11)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 5.0 (10)1.6 (1)17 (20)∞5e60/10

Δ HVref
f53, 5-D		 1e0
2.0		 1e-1
13		 1e-2
42		 1e-3
1443		 #succ
2/10
DEMO Tusar bbob-biobj 20 (3)20 (13)23 (10)3.0 (5)2/10
HMO-CMA-ES Loshchilov bbob-biobj 0.85 (2)1 (0.4)0.99 (0.7)13 (67)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.7 (0)3.2 (0.3)2.5 (1)1.00 (0.1)0/10
MO-DIRECT-ND Wong bbob-biobj 13 (62)16 (7)18 (13)2.6 (3)0/10
MO-DIRECT-Rank Wong bbob-biobj 12 (29)12 (5)24 (15)6.3 (9)0/10
NSGA-II-MATLAB Auger bbob-biobj 116 (176)62 (8)29 (8)40 (88)1/10
RANDOMSEARCH-100 Auger bbob-biobj 2.3e6 (3e6)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 8.7 (37)7.6 (15)82 (5)63 (56)0/10
RANDOMSEARCH-5 Auger bbob-biobj 6.8 (14)11 (14)227 (490)457 (160)0/10
RM-MEDA Auger bbob-biobj 658 (167)136 (12)50 (5)2.6 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 651 (141)146 (18)53 (7)2.2 (2)1/10
SMS-EMOA-PM Auger bbob-biobj 376 (117)83 (26)38 (8)40 (87)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.0 (6)12 (12)22 (16)387 (3)1/10

Δ HVref
f54, 5-D		 1e0
98		 1e-1
1514		 1e-2
12856		 1e-3
45502		 #succ
10/10
DEMO Tusar bbob-biobj 1.0 (0.0)1.0 (0.5)6.1 (3)9.2 (15)0/10
HMO-CMA-ES Loshchilov bbob-biobj 5.7 (14)1.8 (2)6.3 (7)3.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.8 (0.2)1.1 (2)1.2 (0.8)∞50010/10
MO-DIRECT-ND Wong bbob-biobj 2.4 (9)1.0 (2)3.8 (2)∞50050/10
MO-DIRECT-Rank Wong bbob-biobj 2.5 (5)1.2 (0.2)3.9 (3)∞50040/10
NSGA-II-MATLAB Auger bbob-biobj 23 (102)2.0 (0.1)4.5 (6)7.3 (4)0/10
RANDOMSEARCH-100 Auger bbob-biobj 1.4e4 (2e4)∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 4.9 (24)12 (55)78 (139)∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 5.8 (0.1)26 (115)171 (223)∞5e60/10
RM-MEDA Auger bbob-biobj 14 (2)1.6 (0.3)26 (29)17 (41)0/10
SMS-EMOA-DE Auger bbob-biobj 15 (4)1.7 (0.3)26 (29)44 (36)0/10
SMS-EMOA-PM Auger bbob-biobj 7.8 (5)1.7 (0.9)163 (194)∞5e50/10
UP-MO-CMA-ES Krause bbob-biobj 1.8 (9)1.1 (0.4)1.00 (0.4)1.0 (0.2)9/10

Δ HVref
f55, 5-D		 1e0
3.0		 1e-1
1415		 1e-2
18086		 1e-3
51245		 #succ
9/10
DEMO Tusar bbob-biobj 7.0 (1)3.8 (2)1.0 (0.5)5.1 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 3.2 (0.2)6.8 (9)8.6 (11)7.5 (7)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞3750/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.4 (5)1.0 (1)∞∞50030/10
MO-DIRECT-ND Wong bbob-biobj 22 (43)1.2 (0.4)2.8 (2)∞50030/10
MO-DIRECT-Rank Wong bbob-biobj 37 (76)1.7 (0.9)∞∞50050/10
NSGA-II-MATLAB Auger bbob-biobj 98 (123)7.7 (11)10 (7)27 (28)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞5e50/10
RANDOMSEARCH-4 Auger bbob-biobj 12 (29)10 (10)∞∞5e50/10
RANDOMSEARCH-5 Auger bbob-biobj 1.0 (1)28 (29)1214 (2143)∞5e60/10
RM-MEDA Auger bbob-biobj 404 (68)2.2 (0.3)19 (29)23 (56)0/10
SMS-EMOA-DE Auger bbob-biobj 318 (42)2.7 (0.8)29 (28)23 (59)1/10
SMS-EMOA-PM Auger bbob-biobj 224 (27)10 (19)48 (55)88 (124)0/10
UP-MO-CMA-ES Krause bbob-biobj 12 (55)2.4 (3)1.00 (0.3)1.00 (0.0)8/10

Previous dimension | Dimension = 10 | Next dimension

Δ HVref
f1, 10-D		 1e0
1		 1e-1
93		 1e-2
886		 1e-3
5534		 #succ
10/10
DEMO Tusar bbob-biobj 5.3 (4)54 (10)19 (1)26 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.3)★21.0 (0.1)1.0 (0.0)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)14 (9)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)520 (613)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)506 (269)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)84 (76)85 (53)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (0.8)3573 (2634)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.1 (6)2.5e4 (4e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1595 (136)30 (2)4.6 (0.2)2.1 (0.1)0/10
SMS-EMOA-DE Auger bbob-biobj 2267 (188)43 (2)7.6 (0.3)2.7 (0.2)10/10
SMS-EMOA-PM Auger bbob-biobj 1149 (202)35 (8)159 (218)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.8 (5)69 (12)67 (3)21 (0.1)10/10

Δ HVref
f2, 10-D		 1e0
2.0		 1e-1
98		 1e-2
880		 1e-3
6397		 #succ
10/10
DEMO Tusar bbob-biobj 4.0 (2)45 (14)19 (2)15 (8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.2)★21.00 (0.6)1.00 (0.4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.8 (16)29 (16)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 23 (114)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 28 (70)232 (280)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 74 (184)102 (9)94 (300)1480 (2931)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (1)1130 (1228)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.1 (0.8)3056 (4421)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1331 (203)47 (4)7.4 (0.9)56 (61)0/10
SMS-EMOA-DE Auger bbob-biobj 1403 (229)4427 (5102)1.0e4 (7102)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 650 (182)35 (5)4793 (8850)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 6.7 (4)60 (24)39 (11)16 (0.3)9/10

Δ HVref
f3, 10-D		 1e0
2.0		 1e-1
114		 1e-2
1001		 1e-3
6064		 #succ
10/10
DEMO Tusar bbob-biobj 10 (11)52 (16)21 (3)32 (23)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (2)1.00 (0.3)★21.00 (0.3)1.0 (0.4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 26 (34)25 (52)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 108 (20)834 (592)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 280 (17)378 (741)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 116 (138)145 (211)143 (215)720 (886)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 6.3 (8)3765 (8443)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 14 (17)2.7e4 (2e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1023 (182)29 (3)4.7 (0.5)5.8 (10)0/10
SMS-EMOA-DE Auger bbob-biobj 1502 (297)43 (5)1507 (1498)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 860 (191)1015 (4389)2355 (4007)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 16 (26)58 (22)46 (16)17 (1)10/10

Δ HVref
f4, 10-D		 1e0
1		 1e-1
118		 1e-2
1013		 1e-3
6250		 #succ
9/10
DEMO Tusar bbob-biobj 111 (254)44 (8)17 (3)43 (52)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1 (0.1)★21 (0.3)1.0 (1.0)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)32 (19)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)400 (398)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)190 (259)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 97 (240)131 (215)53 (46)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 24 (114)3168 (3523)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 152 (336)2.1e4 (3e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1989 (278)27 (3)4.4 (0.3)42 (0.7)0/10
SMS-EMOA-DE Auger bbob-biobj 2684 (225)36 (2)2311 (3949)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1377 (337)26 (11)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 72 (286)53 (14)35 (7)17 (0.1)8/10

Δ HVref
f5, 10-D		 1e0
1		 1e-1
176		 1e-2
1805		 1e-3
20009		 #succ
9/10
DEMO Tusar bbob-biobj 3.8 (4)41 (10)22 (11)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.3)★1.00 (0.2)1 (0.6)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7420/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)13 (14)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)564 (1166)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)122 (143)413 (266)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.9 (2)1.0e4 (9532)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.7 (2)9.9e4 (5e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1687 (178)18 (2)3.7 (2)44 (61)0/10
SMS-EMOA-DE Auger bbob-biobj 2395 (350)27 (4)7.1 (3)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1019 (154)44 (29)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 6.2 (5)58 (19)50 (5)6.1 (0.3)7/10

Δ HVref
f6, 10-D		 1e0
4.0		 1e-1
120		 1e-2
1280		 1e-3
8850		 #succ
10/10
DEMO Tusar bbob-biobj 15 (28)50 (23)17 (0.9)50 (50)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (0.5)1.0 (0.4)★21.00 (0.3)1.0 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.1 (6)23 (10)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 26 (11)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 20 (66)263 (292)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 56 (108)109 (78)100 (123)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.1 (8)1.0e4 (2e4)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 28 (60)6.9e4 (2e5)∞∞1e70/10
RM-MEDA Auger bbob-biobj 569 (135)30 (4)3.9 (0.3)1.6 (0.3)0/10
SMS-EMOA-DE Auger bbob-biobj 749 (73)40 (5)6.7 (0.8)13 (23)0/10
SMS-EMOA-PM Auger bbob-biobj 369 (93)35 (17)185 (335)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 12 (23)65 (31)42 (16)13 (0.3)10/10

Δ HVref
f7, 10-D		 1e0
3.0		 1e-1
11174		 1e-2
6.9e5		 1e-3
3.7e7		 #succ
0/10
DEMO Tusar bbob-biobj 16 (25)7.8 (13)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.00 (1)1.0 (1.0)1.00 (0.8)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.4 (10)2.6 (3)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 52 (33)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 69 (44)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 104 (178)36 (45)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (5)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.0 (8)8481 (1e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 630 (82)1.3 (2)∞∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 967 (66)1.9 (3)14 (13)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 445 (88)45 (75)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 27 (22)3.7 (3)∞∞1e70/10

Δ HVref
f8, 10-D		 1e0
4.0		 1e-1
7518		 1e-2
1.4e6		 1e-3
2.0e7		 #succ
0/10
DEMO Tusar bbob-biobj 10 (12)6.7 (12)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)7.3 (10)1.00 (0.7)1.00 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (9)13 (9)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 100 (54)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 55 (70)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 144 (113)47 (208)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.4 (11)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.5 (12)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 462 (30)1.0 (0.7)∞∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 685 (108)1.6 (0.7)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 398 (125)100 (126)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 15 (4)155 (4)∞∞1e70/10

Δ HVref
f9, 10-D		 1e0
1		 1e-1
116		 1e-2
906		 1e-3
3733		 #succ
9/10
DEMO Tusar bbob-biobj 24 (34)38 (5)16 (2)12 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.2)★21.00 (0.3)1.0 (0.3)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)20 (19)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)186 (104)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)110 (131)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 39 (0)58 (49)60 (30)2651 (4153)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.8 (2)1040 (735)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 7.3 (8)4131 (3047)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1872 (176)24 (1)4.5 (0.1)1.9 (0.1)0/10
SMS-EMOA-DE Auger bbob-biobj 2470 (282)34 (3)6.8 (0.9)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1285 (196)25 (4)4426 (5519)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 14 (18)47 (16)33 (8)27 (0.4)8/10

Δ HVref
f10, 10-D		 1e0
2.0		 1e-1
3855		 1e-2
90606		 1e-3
1.5e6		 #succ
9/10
DEMO Tusar bbob-biobj 79 (191)3.7 (1)8.3 (9)3.0 (5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)3.1 (1)3.3 (4)1.00 (0.3)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7380/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.4 (0)2.7 (3)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 38 (95)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 20 (0)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 84 (171)24 (29)100 (77)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 23 (56)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 64 (153)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 864 (98)1.00 (0.1)44 (30)2.8 (4)0/10
SMS-EMOA-DE Auger bbob-biobj 1276 (182)17 (39)99 (99)6.1 (7)0/10
SMS-EMOA-PM Auger bbob-biobj 612 (188)19 (33)100 (188)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 64 (152)3.7 (0.7)1.0 (0.2)3.0 (8)6/10

Δ HVref
f11, 10-D		 1e0
7.0		 1e-1
138		 1e-2
1132		 1e-3
31641		 #succ
7/10
DEMO Tusar bbob-biobj 19 (47)7.0 (2)7.3 (11)26 (50)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.96 (0.9)1.0 (1)1.0 (0.4)1.0 (1)9/10
MAT-DIRECT Al-Dujaili bbob-biobj 496 (552)∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj 1015 (910)∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 13 (20)21 (35)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 104 (10)41 (50)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 140 (60)671 (1214)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 24 (87)29 (112)29 (9)43 (54)0/10
RANDOMSEARCH-100 Auger bbob-biobj 7.4e4 (7e4)∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.4 (0.3)806 (1812)224 (41)∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 49 (0.2)5367 (3e4)1346 (2508)3158 (3002)0/10
RM-MEDA Auger bbob-biobj 373 (635)147 (290)158 (241)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 146 (136)16 (5)3556 (1e4)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 74 (64)12 (10)752 (1457)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 12 (0.4)12 (2)20 (7)3.5 (0.1)7/10

Δ HVref
f12, 10-D		 1e0
4.0		 1e-1
59		 1e-2
447		 1e-3
4680		 #succ
10/10
DEMO Tusar bbob-biobj 1.3 (2)18 (16)15 (9)6.9 (4)1/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1)1.0 (1)1.00 (1)1.00 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7460/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 12 (21)4.9 (3)14 (26)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 59 (159)104 (35)219 (353)∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 278 (228)129 (88)104 (73)∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 79 (72)19 (5)37 (34)49 (117)0/10
RANDOMSEARCH-100 Auger bbob-biobj 2.3e6 (2e6)∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (4)280 (493)3388 (4474)2052 (2831)0/10
RANDOMSEARCH-5 Auger bbob-biobj 4.0 (6)177 (147)1.2e4 (2e4)5668 (1e4)0/10
RM-MEDA Auger bbob-biobj 592 (358)109 (70)174 (24)159 (595)0/10
SMS-EMOA-DE Auger bbob-biobj 463 (82)60 (12)13 (2)94 (160)0/10
SMS-EMOA-PM Auger bbob-biobj 221 (48)32 (10)11 (4)215 (214)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.2 (0.9)27 (21)18 (13)9.3 (6)10/10

Δ HVref
f13, 10-D		 1e0
9.0		 1e-1
32		 1e-2
220		 1e-3
3827		 #succ
10/10
DEMO Tusar bbob-biobj 17 (39)54 (75)35 (13)84 (146)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.99 (2)0.99 (1)★1.00 (0.9)★21.0 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7400/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.5 (6)8.7 (3)31 (23)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 28 (25)343 (244)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 17 (14)141 (181)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 27 (52)185 (243)91 (104)19 (22)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.6 (1)315 (1499)3716 (2512)∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.7 (12)3862 (114)1.7e4 (1e4)∞1e70/10
RM-MEDA Auger bbob-biobj 316 (59)129 (23)27 (7)34 (30)0/10
SMS-EMOA-DE Auger bbob-biobj 317 (59)119 (22)1159 (2275)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 170 (42)66 (11)197 (21)2411 (1829)0/10
UP-MO-CMA-ES Krause bbob-biobj 15 (36)41 (45)39 (35)14 (4)10/10

Δ HVref
f14, 10-D		 1e0
4.0		 1e-1
351		 1e-2
3352		 1e-3
76555		 #succ
10/10
DEMO Tusar bbob-biobj 1.9 (3)30 (6)62 (150)63 (76)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (2)1.0 (0.6)★1.0 (0.5)1.00 (0.2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (15)54 (46)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 113 (239)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 49 (71)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 65 (115)50 (25)868 (1617)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (1)7715 (7208)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.0 (2)7.7e4 (1e5)∞∞1e70/10
RM-MEDA Auger bbob-biobj 642 (163)18 (2)3.8 (2)14 (7)0/10
SMS-EMOA-DE Auger bbob-biobj 682 (133)6667 (1e4)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 325 (59)2.6e4 (3e4)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.0 (5)44 (15)27 (4)1.7 (0.7)5/10

Δ HVref
f15, 10-D		 1e0
6.0		 1e-1
102		 1e-2
1082		 1e-3
10556		 #succ
10/10
DEMO Tusar bbob-biobj 2.8 (0.9)42 (44)22 (10)29 (12)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (1)1.0 (0.6)★1.00 (0.6)1.0 (0.9)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7490/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.5 (13)37 (46)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 96 (120)438 (713)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 37 (109)98 (107)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 97 (71)29 (27)43 (40)424 (687)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 0.92 (1)1389 (2856)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 5.9 (13)9814 (4e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 629 (176)68 (16)10 (2)106 (142)0/10
SMS-EMOA-DE Auger bbob-biobj 570 (94)4254 (7359)8325 (1e4)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 354 (192)1135 (2460)8340 (1e4)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 1.6 (3)43 (50)32 (11)9.4 (1.0)10/10

Δ HVref
f16, 10-D		 1e0
4.0		 1e-1
16889		 1e-2
7.3e5		 1e-3
2.2e7		 #succ
0/10
DEMO Tusar bbob-biobj 1.7 (1)5.6 (11)13 (19)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (3)1.0 (1)1.00 (1)1.0 (2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.7 (24)1.1 (1)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 153 (382)5.8 (10)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 71 (0)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 79 (146)20 (36)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.5 (20)144 (191)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 4.5 (12)721 (768)∞∞1e70/10
RM-MEDA Auger bbob-biobj 880 (350)1.2 (2)5.6 (5)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 833 (225)139 (148)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 427 (172)60 (89)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 2.3 (5)149 (296)124 (161)∞1e70/10

Δ HVref
f17, 10-D		 1e0
8.0		 1e-1
7097		 1e-2
2.5e5		 1e-3
4.0e6		 #succ
1/10
DEMO Tusar bbob-biobj 3.2 (4)3.2 (4)12 (7)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.9)1.6 (2)1 (0.6)1.00 (0.8)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.3 (11)2.6 (2)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 150 (297)6.5 (6)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 92 (187)4.0 (2)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 55 (53)46 (67)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.4 (0.2)217 (247)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.9 (10)2125 (4231)∞∞1e70/10
RM-MEDA Auger bbob-biobj 400 (88)1.00 (0.3)2.7 (5)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 383 (78)96 (142)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 213 (78)62 (212)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 6.5 (13)161 (356)∞∞1e70/10

Δ HVref
f18, 10-D		 1e0
7.0		 1e-1
75		 1e-2
650		 1e-3
8969		 #succ
7/10
DEMO Tusar bbob-biobj 1.3 (2)27 (14)11 (2)4.9 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)1 (0.5)1 (0.4)★21 (1)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.7 (6)14 (15)146 (81)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 28 (32)50 (29)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 20 (30)347 (337)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 48 (61)15 (1)5.9 (8)69 (97)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (2)13 (11)975 (1353)∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (2)101 (160)7652 (9114)∞1e70/10
RM-MEDA Auger bbob-biobj 475 (106)72 (7)12 (4)6.7 (4)0/10
SMS-EMOA-DE Auger bbob-biobj 399 (25)1536 (6673)1546 (1539)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 200 (55)34 (4)798 (773)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 2.1 (5)16 (10)18 (4)11 (2)7/10

Δ HVref
f19, 10-D		 1e0
4.0		 1e-1
6872		 1e-2
58336		 1e-3
2.7e6		 #succ
9/10
DEMO Tusar bbob-biobj 2.5 (4)2.4 (0.6)12 (9)1.5 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.4)1.4 (2)4.7 (4)1.1 (3)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.2 (14)2.1 (3)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 50 (176)14 (10)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 28 (9)4.7 (7)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 63 (57)7.5 (4)19 (30)3.6 (3)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (3)101 (182)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (2)328 (641)∞∞1e70/10
RM-MEDA Auger bbob-biobj 733 (229)1.00 (0.3)26 (43)3.4 (4)0/10
SMS-EMOA-DE Auger bbob-biobj 689 (87)146 (218)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 339 (50)19 (41)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 1.1 (1)1.6 (0.6)1.0 (0.3)1.0 (0.1)5/10

Δ HVref
f20, 10-D		 1e0
8.0		 1e-1
84		 1e-2
640		 1e-3
4014		 #succ
10/10
DEMO Tusar bbob-biobj 17 (7)40 (57)16 (17)9.4 (9)4/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.2)0.99 (1)★1.00 (2)1.00 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7480/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.8 (11)7.3 (5)17 (21)6.4 (5)0/10
MO-DIRECT-ND Wong bbob-biobj 40 (46)101 (174)44 (66)23 (24)0/10
MO-DIRECT-Rank Wong bbob-biobj 69 (62)221 (202)148 (172)∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 64 (75)89 (337)165 (39)127 (195)0/10
RANDOMSEARCH-100 Auger bbob-biobj 5.1e5 (6e5)∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.6 (9)1928 (950)3965 (6470)1073 (1591)0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.9 (7)1.4e4 (6e4)1.4e4 (1e4)1.0e4 (1e4)0/10
RM-MEDA Auger bbob-biobj 228 (153)52 (47)18 (18)17 (7)0/10
SMS-EMOA-DE Auger bbob-biobj 335 (258)65 (35)14 (6)110 (188)0/10
SMS-EMOA-PM Auger bbob-biobj 366 (563)1037 (2204)832 (1179)299 (140)0/10
UP-MO-CMA-ES Krause bbob-biobj 5.4 (7)32 (21)22 (30)10 (12)10/10

Δ HVref
f21, 10-D		 1e0
3.0		 1e-1
105		 1e-2
826		 1e-3
24204		 #succ
8/10
DEMO Tusar bbob-biobj 27 (121)41 (30)17 (14)3.8 (6)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.00 (0.5)★21 (0.3)1.0 (0.2)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7460/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 16 (30)7.9 (15)11 (10)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 169 (39)104 (104)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 179 (435)105 (40)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 787 (187)160 (265)265 (675)185 (288)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 10 (31)2235 (2585)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 22 (67)1.9e4 (2585)1.2e5 (8e4)∞1e70/10
RM-MEDA Auger bbob-biobj 739 (186)32 (6)6.1 (2)5.1 (11)0/10
SMS-EMOA-DE Auger bbob-biobj 1147 (125)45 (10)8.6 (5)62 (72)0/10
SMS-EMOA-PM Auger bbob-biobj 710 (235)37 (8)1219 (1212)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 13 (21)38 (26)29 (43)3.3 (1)7/10

Δ HVref
f22, 10-D		 1e0
2.0		 1e-1
286		 1e-2
2633		 1e-3
22507		 #succ
10/10
DEMO Tusar bbob-biobj 15 (34)38 (9)33 (37)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (0.5)1.00 (1.0)★21.0 (0.3)1.00 (0.5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7460/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 12 (21)65 (18)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 38 (85)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 173 (52)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 347 (179)82 (116)1814 (1413)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (3)1.0e4 (9324)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 24 (11)5.5e4 (8e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1080 (84)19 (4)5.5 (3)33 (46)0/10
SMS-EMOA-DE Auger bbob-biobj 1631 (276)24 (3)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 941 (193)8186 (8736)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 10 (37)55 (14)35 (4)5.2 (0.4)4/10

Δ HVref
f23, 10-D		 1e0
1		 1e-1
115		 1e-2
943		 1e-3
5370		 #succ
10/10
DEMO Tusar bbob-biobj 36 (95)40 (16)22 (11)23 (21)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (0)1.0 (0.9)★1 (0.7)1.00 (0.6)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.4 (11)24 (33)101 (61)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 14 (32)95 (58)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 7.2 (31)138 (113)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 292 (387)19 (12)162 (199)1830 (1164)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 6.3 (8)2394 (2355)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 13 (6)2.2e4 (1019)∞∞1e70/10
RM-MEDA Auger bbob-biobj 3077 (433)97 (57)31 (27)30 (50)0/10
SMS-EMOA-DE Auger bbob-biobj 5588 (1118)74 (20)1190 (1475)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 6684 (1e4)158 (197)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 8.2 (8)45 (33)40 (12)19 (0.8)10/10

Δ HVref
f24, 10-D		 1e0
2.0		 1e-1
12987		 1e-2
4.8e5		 1e-3
2.6e7		 #succ
0/10
DEMO Tusar bbob-biobj 45 (211)6.6 (9)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (1)1.0 (0.7)1.00 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7460/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 15 (36)1.5 (2)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 29 (21)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 104 (260)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 116 (241)35 (78)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 6.7 (8)325 (302)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 16 (67)1989 (3674)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1481 (303)22 (97)19 (14)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 2187 (233)1.6 (0.9)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1748 (808)135 (184)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 7.2 (13)3.1 (2)∞∞1e70/10

Δ HVref
f25, 10-D		 1e0
2.0		 1e-1
7857		 1e-2
8.9e5		 1e-3
1.5e7		 #succ
0/10
DEMO Tusar bbob-biobj 3.0 (5)3.4 (2)4.9 (5)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)5.8 (3)1.0 (1)1.00 (0.9)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.3 (34)3.7 (2)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 27 (66)6.0 (8)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 177 (0)6.0 (5)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 100 (215)59 (65)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (4)301 (225)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 4.5 (7)2976 (2864)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1202 (181)1.00 (1)4.6 (3)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 1852 (282)15 (64)10 (16)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1159 (317)201 (414)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 2.8 (1)5.4 (8)∞∞1e70/10

Δ HVref
f26, 10-D		 1e0
3.0		 1e-1
85		 1e-2
381		 1e-3
1644		 #succ
8/10
DEMO Tusar bbob-biobj 11 (9)32 (21)21 (8)13 (9)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (0.5)★21.0 (0.2)1.00 (0.8)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7410/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 14 (59)8.4 (2)8.2 (10)16 (29)0/10
MO-DIRECT-ND Wong bbob-biobj 18 (44)74 (69)130 (185)∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 103 (504)68 (23)254 (243)∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 101 (142)18 (6)18 (19)185 (610)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.0 (0.8)180 (777)4302 (5308)∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 9.0 (25)476 (1029)3.2e4 (4e4)∞1e70/10
RM-MEDA Auger bbob-biobj 757 (137)38 (2)11 (2)3.8 (0.8)2/10
SMS-EMOA-DE Auger bbob-biobj 1144 (132)52 (10)15 (3)72 (1)0/10
SMS-EMOA-PM Auger bbob-biobj 666 (127)39 (8)127 (557)636 (789)0/10
UP-MO-CMA-ES Krause bbob-biobj 17 (33)25 (6)25 (14)22 (11)8/10

Δ HVref
f27, 10-D		 1e0
2.0		 1e-1
12590		 1e-2
78458		 1e-3
3.5e6		 #succ
8/10
DEMO Tusar bbob-biobj 14 (14)1.3 (0.5)11 (20)2.6 (4)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (2)1.3 (1)2.8 (3)1.00 (2)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 29 (117)2.5 (3)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 42 (137)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 236 (282)7.9 (14)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 237 (338)4.5 (6)38 (30)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.3 (8)207 (441)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 10 (16)819 (1087)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1085 (42)9.2 (79)30 (29)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 1751 (413)42 (91)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 980 (156)120 (119)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 11 (13)1.00 (0.3)1.0 (0.4)1.4 (3)5/10

Δ HVref
f28, 10-D		 1e0
2.0		 1e-1
24		 1e-2
358		 1e-3
5622		 #succ
7/10
DEMO Tusar bbob-biobj 75 (234)134 (47)28 (7)8.1 (2)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (3)0.98 (1)★21.00 (0.5)★21.00 (0.8)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.6 (8)11 (3)13 (7)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 10 (46)101 (68)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 12 (0)90 (63)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 92 (206)189 (185)75 (48)133 (51)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 11 (20)2269 (974)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 28 (19)9088 (2e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 960 (120)124 (10)12 (2)1.5 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 1475 (243)170 (29)17 (2)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 798 (120)188 (176)3451 (4193)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 20 (35)113 (38)40 (11)14 (1)7/10

Δ HVref
f29, 10-D		 1e0
2.0		 1e-1
108		 1e-2
7772		 1e-3
40372		 #succ
10/10
DEMO Tusar bbob-biobj 39 (168)96 (27)34 (33)73 (131)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)1 (0.3)★21.0 (0.1)1.0 (0.9)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.5 (7)265 (124)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 5.8 (13)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 5.0 (0)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 96 (252)278 (332)568 (680)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.2 (4)9.0e4 (5e4)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 9.2 (10)8.7e5 (6e5)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1009 (105)37 (4)1.1 (0.3)17 (37)0/10
SMS-EMOA-DE Auger bbob-biobj 1426 (162)1081 (4631)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 684 (65)2.2e4 (3e4)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 35 (127)147 (41)11 (2)32 (65)5/10

Δ HVref
f30, 10-D		 1e0
1		 1e-1
109		 1e-2
968		 1e-3
5577		 #succ
10/10
DEMO Tusar bbob-biobj 36 (64)43 (14)19 (5)28 (17)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.4 (0.2)1.00 (0.4)★1.00 (0.5)1.0 (0.4)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7350/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 12 (30)18 (4)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 23 (20)279 (250)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 15 (32)264 (209)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 377 (456)35 (43)54 (45)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 12 (15)1348 (2944)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 19 (25)1.1e4 (1e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 2534 (279)35 (5)5.9 (0.7)23 (45)0/10
SMS-EMOA-DE Auger bbob-biobj 3346 (216)44 (8)1557 (2325)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1791 (182)35 (6)4174 (4146)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 32 (40)49 (35)37 (14)19 (0.5)10/10

Δ HVref
f31, 10-D		 1e0
2.0		 1e-1
12618		 1e-2
5.8e5		 1e-3
9.2e6		 #succ
0/10
DEMO Tusar bbob-biobj 70 (70)7.8 (7)17 (23)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)1.00 (0.9)1.00 (2)1.00 (0.9)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.5 (20)2.0 (2)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 72 (0)7.8 (12)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 17 (0)7.6 (15)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 77 (190)21 (18)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 30 (131)318 (337)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 110 (261)3172 (2775)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1095 (186)10 (1)7.1 (8)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 1598 (257)11 (2)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 820 (131)132 (91)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 76 (151)91 (199)∞∞1e70/10

Δ HVref
f32, 10-D		 1e0
7.0		 1e-1
6372		 1e-2
3.3e5		 1e-3
8.4e6		 #succ
0/10
DEMO Tusar bbob-biobj 32 (75)4.6 (5)29 (42)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)3.2 (4)1.0 (0.5)1.00 (2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7470/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.4 (22)4.5 (7)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 88 (0)15 (15)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 23 (44)15 (27)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 35 (52)115 (115)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.8 (10)664 (271)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 51 (31)1.4e4 (2e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 309 (37)1.0 (0.4)5.6 (4)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 460 (59)1.6 (0.8)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 241 (62)146 (157)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 21 (65)1051 (1962)∞∞1e70/10

Δ HVref
f33, 10-D		 1e0
3.0		 1e-1
17		 1e-2
154		 1e-3
1507		 #succ
6/10
DEMO Tusar bbob-biobj 72 (306)171 (84)49 (13)19 (13)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (0.3)1.00 (0.4)★1.00 (0.9)6/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7460/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.9 (25)13 (7)16 (6)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 25 (64)291 (219)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 12 (30)69 (30)121 (81)∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 171 (175)69 (16)68 (97)239 (508)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 52 (2)3555 (8382)1.3e4 (2e4)∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 89 (258)2.1e4 (2e4)1.9e5 (2e5)∞1e70/10
RM-MEDA Auger bbob-biobj 702 (183)162 (20)23 (3)53 (242)0/10
SMS-EMOA-DE Auger bbob-biobj 969 (108)218 (16)31 (3)174 (337)0/10
SMS-EMOA-PM Auger bbob-biobj 529 (66)148 (20)31 (11)1576 (1966)0/10
UP-MO-CMA-ES Krause bbob-biobj 70 (58)137 (33)59 (29)33 (8)2/10

Δ HVref
f34, 10-D		 1e0
6.0		 1e-1
3602		 1e-2
54082		 1e-3
9.0e5		 #succ
10/10
DEMO Tusar bbob-biobj 7.0 (10)3.0 (0.8)28 (88)2.8 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (2)1.00 (2)3.3 (3)1.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7480/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.9 (4)3.8 (6)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 7.6 (9)28 (29)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 8.3 (17)14 (8)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 65 (80)10 (9)51 (76)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.8 (8)1289 (633)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 13 (31)4783 (9759)∞∞1e70/10
RM-MEDA Auger bbob-biobj 356 (53)1.1 (0.2)19 (9)10 (12)0/10
SMS-EMOA-DE Auger bbob-biobj 471 (53)71 (0.3)44 (42)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 262 (88)121 (210)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 4.8 (7)2.9 (1)1.0 (0.5)2.9 (6)6/10

Δ HVref
f35, 10-D		 1e0
1		 1e-1
126		 1e-2
2595		 1e-3
25274		 #succ
10/10
DEMO Tusar bbob-biobj 1.9 (1)59 (6)22 (5)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.2)★21.0 (0.3)1.00 (0.1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)19 (8)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)96 (67)3690 (5107)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.2)2.1e4 (3e4)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.2 (0)1.4e5 (1e5)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1584 (110)27 (2)2.5 (0.2)184 (230)0/10
SMS-EMOA-DE Auger bbob-biobj 2220 (344)38 (2)3562 (2216)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 908 (172)59 (30)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 2.9 (0.5)89 (19)39 (0.1)5.1 (0.3)7/10

Δ HVref
f36, 10-D		 1e0
1		 1e-1
327		 1e-2
3312		 1e-3
33699		 #succ
10/10
DEMO Tusar bbob-biobj 18 (52)30 (5)58 (81)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)1.0 (0.8)★21.00 (0.3)1.0 (0.9)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.2 (0)19 (6)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 18 (42)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 10 (43)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 141 (350)71 (72)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.8 (7)2.8e4 (5e4)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 9.1 (18)1.4e5 (9e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 2263 (378)13 (2)44 (89)134 (197)0/10
SMS-EMOA-DE Auger bbob-biobj 2975 (377)19 (3)83 (230)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1559 (544)71 (98)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 8.6 (2)48 (13)29 (2)37 (74)4/10

Δ HVref
f37, 10-D		 1e0
1		 1e-1
10070		 1e-2
4.9e5		 1e-3
1.7e7		 #succ
0/10
DEMO Tusar bbob-biobj 6.4 (4)7.0 (12)19 (15)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (0)1.5 (2)1.00 (0.8)1 (0.8)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.3 (22)2.0 (4)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 10 (0)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 7.5 (0)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 49 (241)32 (54)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (0.8)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 8.9 (22)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 2013 (320)1.0 (1)18 (37)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 2644 (337)1.4 (1)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1452 (318)41 (125)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 7.4 (11)4.0 (3)183 (290)∞1e70/10

Δ HVref
f38, 10-D		 1e0
2.0		 1e-1
11650		 1e-2
1.1e6		 1e-3
2.3e7		 #succ
0/10
DEMO Tusar bbob-biobj 47 (2)12 (13)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)2.6 (2)1.0 (1)1.00 (2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7420/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 7.5 (18)∞∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 59 (146)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 40 (199)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 137 (254)33 (17)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.8 (2)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 7.0 (7)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 980 (139)1.0 (0.7)∞∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 1373 (259)2.2 (3)8.7 (7)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 759 (205)58 (85)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 10 (0.6)5.8 (3)∞∞1e70/10

Δ HVref
f39, 10-D		 1e0
7.0		 1e-1
355		 1e-2
3656		 1e-3
19977		 #succ
8/10
DEMO Tusar bbob-biobj 4.3 (4)26 (4)15 (17)64 (60)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.1)1.0 (0.7)★21.00 (1)1 (0.8)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7480/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1.5 (1)36 (44)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 4.5 (11)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 2.9 (7)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 33 (36)84 (77)761 (1174)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 6.3 (13)2.7e4 (3e4)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 22 (52)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 272 (25)10 (2)2.0 (0.3)2.8 (6)0/10
SMS-EMOA-DE Auger bbob-biobj 375 (42)15 (2)2471 (3282)455 (413)0/10
SMS-EMOA-PM Auger bbob-biobj 183 (57)76 (37)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 7.8 (28)39 (11)31 (42)132 (132)2/10

Δ HVref
f40, 10-D		 1e0
1		 1e-1
4399		 1e-2
1.0e5		 1e-3
1.0e6		 #succ
10/10
DEMO Tusar bbob-biobj 3.1 (3)4.2 (2)4.1 (7)10 (7)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)11 (6)2.6 (3)1.00 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7360/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)11 (11)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)15 (21)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (1)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (0.5)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 1603 (106)1.00 (0.1)5.1 (5)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 2443 (248)59 (227)92 (64)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1209 (265)177 (86)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 1.6 (1)4.0 (0.5)1.00 (0.0)1.2 (0.0)8/10

Δ HVref
f41, 10-D		 1e0
5.0		 1e-1
206		 1e-2
1600		 1e-3
11425		 #succ
10/10
DEMO Tusar bbob-biobj 12 (19)32 (22)18 (5)67 (49)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (1)1.00 (0.7)★1.0 (0.4)1.0 (0.6)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7420/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.1 (6)21 (50)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 52 (138)220 (180)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 52 (97)103 (127)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 100 (101)45 (48)180 (97)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.3 (6)3785 (4022)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 12 (20)3.4e4 (4e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 646 (135)24 (5)5.1 (3)2.6 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 893 (155)32 (4)8.5 (2)37 (69)0/10
SMS-EMOA-PM Auger bbob-biobj 500 (90)31 (11)138 (80)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 19 (43)35 (15)34 (14)10 (0.6)10/10

Δ HVref
f42, 10-D		 1e0
1		 1e-1
18170		 1e-2
6.4e5		 1e-3
1.3e7		 #succ
0/10
DEMO Tusar bbob-biobj 14 (46)3.8 (8)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.3 (3)1.00 (1.0)1.00 (0.7)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)1.5 (2)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 102 (506)12 (28)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 7.6 (10)248 (233)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 8.5 (11)1423 (1603)∞∞1e70/10
RM-MEDA Auger bbob-biobj 2952 (746)1.4 (2)15 (16)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 3645 (499)1.00 (1)7.5 (9)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 2270 (1416)27 (56)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 19 (25)2.4 (3)142 (110)∞1e70/10

Δ HVref
f43, 10-D		 1e0
2.0		 1e-1
5180		 1e-2
3.2e5		 1e-3
7.3e6		 #succ
0/10
DEMO Tusar bbob-biobj 6.3 (14)3.1 (1)6.8 (8)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (3)3.8 (5)1.0 (0.8)1 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.8 (0)2.9 (4)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 24 (0)8.5 (7)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 27 (66)8.2 (12)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 100 (106)41 (71)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.7 (24)206 (201)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 5.3 (10)1433 (966)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1243 (160)1.00 (0.4)5.6 (5)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 1678 (194)1.6 (0.8)7.7 (8)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 924 (287)34 (31)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 10 (20)5.9 (9)∞∞1e70/10

Δ HVref
f44, 10-D		 1e0
6.0		 1e-1
125		 1e-2
834		 1e-3
3534		 #succ
8/10
DEMO Tusar bbob-biobj 55 (242)37 (5)19 (5)17 (5)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (2)1.0 (0.4)1.0 (0.4)1.00 (0.1)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7410/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.6 (0)19 (45)117 (138)∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 59 (137)86 (68)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 39 (148)64 (43)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 42 (102)41 (70)49 (54)2640 (1910)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 62 (154)2101 (4011)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 940 (2346)2.1e4 (2e4)∞∞1e70/10
RM-MEDA Auger bbob-biobj 428 (102)30 (5)6.6 (1)2.7 (0.7)0/10
SMS-EMOA-DE Auger bbob-biobj 527 (130)36 (4)142 (600)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 274 (113)30 (6)1967 (1950)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 33 (2)35 (32)36 (15)29 (0.9)6/10

Δ HVref
f45, 10-D		 1e0
1		 1e-1
4900		 1e-2
73882		 1e-3
7.4e5		 #succ
10/10
DEMO Tusar bbob-biobj 12 (4)2.7 (1)15 (18)12 (10)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.6 (4)3.6 (3)1 (0.7)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)1.9 (3)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)20 (14)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 50 (123)9.5 (12)21 (22)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (3)869 (1109)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 8.2 (15)3297 (3450)∞∞1e70/10
RM-MEDA Auger bbob-biobj 2137 (276)1.00 (0.3)55 (74)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 3232 (465)1.6 (0.5)55 (27)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1487 (177)34 (11)126 (61)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 10 (19)2.7 (0.6)1.0 (0.4)1.7 (7)8/10

Δ HVref
f46, 10-D		 1e0
3.0		 1e-1
29415		 1e-2
1.5e6		 1e-3
5.8e7		 #succ
0/10
DEMO Tusar bbob-biobj 4.8 (7)42 (54)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (1)1.2 (1)1.00 (0.7)1.00 (1.0)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7440/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.3 (13)∞∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 24 (21)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 12 (24)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 81 (0)34 (20)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.8 (1)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 4.2 (3)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 921 (79)2.4 (2)∞∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 1154 (161)1.0 (0.5)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 616 (128)40 (42)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 4.7 (3)40 (171)∞∞1e70/10

Δ HVref
f47, 10-D		 1e0
2.0		 1e-1
20669		 1e-2
1.1e6		 1e-3
3.5e7		 #succ
0/10
DEMO Tusar bbob-biobj 53 (250)7.3 (9)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)2.0 (2)1.0 (0.3)1 (0.5)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 11 (26)4.6 (6)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 130 (542)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 178 (648)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 173 (205)23 (45)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.7 (10)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 4.6 (12)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 1082 (116)1.00 (0.3)∞∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 1518 (164)6.3 (0.4)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 792 (133)41 (88)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 26 (61)128 (485)∞∞1e70/10

Δ HVref
f48, 10-D		 1e0
2.0		 1e-1
5403		 1e-2
3.8e5		 1e-3
9.5e6		 #succ
0/10
DEMO Tusar bbob-biobj 49 (112)13 (25)11 (8)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.75 (0.6)1.00 (1)1.0 (0.8)1.3 (2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 2.8 (0)1.8 (0.9)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 7.6 (0)3.7 (3)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 5.2 (24)3.5 (4)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 151 (210)29 (62)24 (19)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.3 (3)281 (279)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 17 (11)2785 (2782)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1068 (165)1.7 (2)3.0 (3)1 (0.4)0/10
SMS-EMOA-DE Auger bbob-biobj 1309 (186)22 (47)24 (48)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 776 (184)58 (113)24 (34)∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 11 (20)209 (464)107 (73)∞1e70/10

Δ HVref
f49, 10-D		 1e0
4.0		 1e-1
25717		 1e-2
2.4e6		 1e-3
1.5e7		 #succ
0/10
DEMO Tusar bbob-biobj 20 (41)8.2 (20)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (2)1 (0.9)1.0 (4)1.00 (2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.2 (15)∞∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 39 (41)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 47 (116)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 67 (128)10 (3)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (6)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 5.2 (12)3558 (5055)∞∞1e70/10
RM-MEDA Auger bbob-biobj 474 (106)1.7 (5)∞∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 705 (100)5.0 (20)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 347 (53)40 (78)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 41 (20)1.6 (2)38 (74)∞1e70/10

Δ HVref
f50, 10-D		 1e0
4.0		 1e-1
15119		 1e-2
9.2e5		 1e-3
1.2e7		 #succ
0/10
DEMO Tusar bbob-biobj 50 (13)13 (11)∞∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.97 (0.4)3.2 (0.9)1.00 (0.7)1.0 (0.5)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7420/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (14)∞∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 169 (338)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 49 (166)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 61 (127)118 (195)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 28 (68)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 72 (4)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 522 (67)1.00 (0.5)∞∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 730 (96)1.7 (1.0)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 394 (104)164 (149)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 11 (48)2650 (3637)∞∞1e70/10

Δ HVref
f51, 10-D		 1e0
1		 1e-1
3839		 1e-2
3.9e5		 1e-3
8.7e6		 #succ
0/10
DEMO Tusar bbob-biobj 32 (85)2.9 (4)7.2 (8)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.3 (0.5)5.5 (12)1.00 (1)1.00 (0.7)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 14 (15)3.3 (3)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 122 (141)12 (7)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 32 (36)5.1 (9)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 378 (433)8.1 (5)24 (24)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 11 (2)393 (1044)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.8 (1)3925 (7830)∞∞1e70/10
RM-MEDA Auger bbob-biobj 1959 (240)1.0 (0.3)1.3 (3)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 2578 (232)1.9 (2)24 (22)∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 1437 (344)82 (162)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 8.0 (5)298 (1305)233 (343)∞1e70/10

Δ HVref
f52, 10-D		 1e0
3.0		 1e-1
82946		 1e-2
2.8e6		 1e-3
1.4e7		 #succ
0/10
DEMO Tusar bbob-biobj 4.3 (4)1.6 (2)3.5 (6)∞1e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.1 (1)1.0 (0.7)1.00 (2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 11 (21)1.2 (2)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 59 (46)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 60 (110)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 106 (133)7.0 (6)∞∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.9)113 (102)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (5)1087 (1266)∞∞1e70/10
RM-MEDA Auger bbob-biobj 599 (49)1.6 (4)3.2 (5)∞1e60/10
SMS-EMOA-DE Auger bbob-biobj 896 (81)1.00 (0.3)∞∞1e60/10
SMS-EMOA-PM Auger bbob-biobj 468 (134)14 (21)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.8 (2)38 (30)32 (71)∞1e70/10

Δ HVref
f53, 10-D		 1e0
2.0		 1e-1
26		 1e-2
117		 1e-3
2163		 #succ
2/10
DEMO Tusar bbob-biobj 27 (21)84 (22)40 (13)5.0 (3)2/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (2)0.99 (1)★1.0 (0.9)★1.0 (2)3/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7480/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.4 (20)4.1 (2)4.2 (3)8.0 (9)0/10
MO-DIRECT-ND Wong bbob-biobj 21 (51)50 (25)47 (16)22 (27)0/10
MO-DIRECT-Rank Wong bbob-biobj 19 (0)40 (29)106 (95)44 (29)0/10
NSGA-II-MATLAB Auger bbob-biobj 132 (182)50 (16)64 (50)18 (27)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 6.7 (4)129 (195)740 (696)4287 (7050)0/10
RANDOMSEARCH-5 Auger bbob-biobj 148 (2)346 (327)7040 (4601)4.5e4 (4e4)0/10
RM-MEDA Auger bbob-biobj 1100 (213)111 (13)29 (2)2.6 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 1314 (176)135 (8)35 (1)2.4 (0.4)0/10
SMS-EMOA-PM Auger bbob-biobj 720 (107)87 (11)28 (8)91 (232)0/10
UP-MO-CMA-ES Krause bbob-biobj 74 (138)56 (19)37 (15)6.9 (5)3/10

Δ HVref
f54, 10-D		 1e0
5.0		 1e-1
8747		 1e-2
58898		 1e-3
9.5e5		 #succ
9/10
DEMO Tusar bbob-biobj 47 (8)1.4 (0.2)8.1 (17)1.2 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (3)1.6 (2)7.3 (6)1.00 (1)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7480/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 4.0 (7)1.00 (1)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 19 (27)11 (13)∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 19 (31)5.7 (5)∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 79 (99)3.7 (1)33 (11)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 17 (40)293 (315)∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 25 (55)1041 (891)∞∞1e70/10
RM-MEDA Auger bbob-biobj 384 (25)13 (0.0)68 (98)10 (13)0/10
SMS-EMOA-DE Auger bbob-biobj 540 (102)13 (29)68 (89)9.5 (7)0/10
SMS-EMOA-PM Auger bbob-biobj 276 (76)97 (200)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 18 (36)1.3 (0.3)1.0 (0.3)1.3 (11)4/10

Δ HVref
f55, 10-D		 1e0
3.0		 1e-1
22439		 1e-2
1.0e6		 1e-3
3.0e6		 #succ
8/10
DEMO Tusar bbob-biobj 1.1 (0.5)6.5 (12)1.0 (0.5)1.4 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (2)7.1 (11)1.0 (1)1.00 (2)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞7500/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (13)4.3 (5)∞∞1e40/10
MO-DIRECT-ND Wong bbob-biobj 40 (79)∞∞∞1e40/10
MO-DIRECT-Rank Wong bbob-biobj 39 (128)∞∞∞1e40/10
NSGA-II-MATLAB Auger bbob-biobj 59 (52)9.1 (5)2.8 (3)∞1e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞1e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.7 (5)∞∞∞1e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.2 (5)∞∞∞1e70/10
RM-MEDA Auger bbob-biobj 535 (133)12 (11)8.8 (8)3.0 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 869 (179)12 (11)3.9 (8)3.0 (2)0/10
SMS-EMOA-PM Auger bbob-biobj 472 (82)79 (134)∞∞1e60/10
UP-MO-CMA-ES Krause bbob-biobj 2.2 (5)1.0 (0.3)2.6 (2)2.3 (5)6/10

Previous dimension | Dimension = 20 | Next dimension

Δ HVref
f1, 20-D		 1e0
1		 1e-1
157		 1e-2
1468		 1e-3
8244		 #succ
10/10
DEMO Tusar bbob-biobj 4.9 (4)163 (10)63 (3)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.4)★21.0 (0.1)★21.00 (0.1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14880/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)171 (172)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)702 (163)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (0.2)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.8 (2)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2432 (181)26 (0.7)4.2 (0.1)6.0 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 3698 (541)44 (4)9.4 (0.2)6.0 (0.3)0/10
SMS-EMOA-PM Auger bbob-biobj 2060 (377)63 (18)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 5.0 (9)157 (20)137 (0.2)29 (0.3)10/10

Δ HVref
f2, 20-D		 1e0
5.0		 1e-1
163		 1e-2
2170		 1e-3
21153		 #succ
10/10
DEMO Tusar bbob-biobj 7.5 (21)195 (49)55 (17)46 (22)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (2)1.0 (0.5)★21.00 (0.4)1.0 (0.2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.7 (21)1123 (614)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 80 (206)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 102 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 50 (94)654 (785)8459 (1e4)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (4)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (3)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 766 (72)41 (4)4.7 (0.7)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 845 (81)2.9e4 (2e4)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 505 (79)54 (10)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.0 (2)134 (40)47 (11)10 (0.3)9/10

Δ HVref
f3, 20-D		 1e0
1		 1e-1
216		 1e-2
2384		 1e-3
11104		 #succ
10/10
DEMO Tusar bbob-biobj 25 (30)161 (35)45 (8)175 (172)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.8)★21.0 (0.3)1.00 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)295 (325)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)663 (614)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.7 (5)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 36 (63)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 3363 (499)25 (3)3.2 (0.3)4.9 (5)0/10
SMS-EMOA-DE Auger bbob-biobj 6675 (598)49 (9)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 3860 (578)9314 (9262)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 68 (57)113 (27)62 (19)20 (0.5)10/10

Δ HVref
f4, 20-D		 1e0
1		 1e-1
172		 1e-2
1682		 1e-3
10200		 #succ
9/10
DEMO Tusar bbob-biobj 35 (46)149 (7)56 (3)84 (11)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.4)★21.00 (0.2)1 (0.1)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14900/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)1108 (1687)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)504 (504)1.1e4 (9216)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 9.2 (12)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 98 (178)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2734 (280)25 (2)3.7 (0.2)57 (103)0/10
SMS-EMOA-DE Auger bbob-biobj 4535 (286)44 (6)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2603 (289)43 (13)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 183 (141)129 (9)71 (7)22 (0.3)8/10

Δ HVref
f5, 20-D		 1e0
1		 1e-1
190		 1e-2
3207		 1e-3
32860		 #succ
10/10
DEMO Tusar bbob-biobj 2.0 (2)187 (33)71 (13)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.3)★21.00 (0.1)1.0 (0.2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14940/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)498 (584)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)1635 (3196)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.7 (1)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.0 (2)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2409 (156)25 (2)2.8 (0.2)27 (51)0/10
SMS-EMOA-DE Auger bbob-biobj 3855 (365)43 (7)113 (648)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2092 (343)279 (154)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 4.7 (10)195 (38)63 (0.1)7.9 (0.2)8/10

Δ HVref
f6, 20-D		 1e0
3.0		 1e-1
236		 1e-2
2134		 1e-3
16568		 #succ
10/10
DEMO Tusar bbob-biobj 4.1 (2)118 (16)69 (14)569 (573)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (0.9)★21.00 (0.2)1.00 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (7)146 (228)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 17 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 6.0 (14)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 72 (155)459 (387)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.8 (0.5)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.1 (6)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 1055 (93)22 (1)3.8 (0.3)1.8 (0.8)0/10
SMS-EMOA-DE Auger bbob-biobj 1686 (233)37 (4)11 (0.8)208 (497)0/10
SMS-EMOA-PM Auger bbob-biobj 922 (180)49 (21)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.0 (4)103 (17)73 (15)14 (0.5)10/10

Δ HVref
f7, 20-D		 1e0
1		 1e-1
24981		 1e-2
6.8e5		 1e-3
3.8e7		 #succ
0/10
DEMO Tusar bbob-biobj 12 (14)12 (7)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.2 (1)1.0 (1)1.0 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞14980/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)326 (421)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.8 (6)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 15 (2)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2783 (198)1.00 (0.9)28 (24)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 4712 (341)1.5 (1)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2580 (429)90 (140)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 6.8 (8)3.9 (3)267 (259)∞2e70/10

Δ HVref
f8, 20-D		 1e0
4.0		 1e-1
16448		 1e-2
2.4e6		 1e-3
2.8e7		 #succ
0/10
DEMO Tusar bbob-biobj 70 (73)39 (44)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.88 (0)3.9 (3)1.00 (0.9)1.00 (0.6)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14840/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 9.2 (29)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 102 (354)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 88 (196)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 27 (68)1164 (973)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 21 (5)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 47 (230)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 677 (96)1.0 (0.6)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 1184 (150)24 (32)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 674 (118)545 (362)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 63 (151)367 (1006)∞∞2e70/10

Δ HVref
f9, 20-D		 1e0
1		 1e-1
144		 1e-2
1468		 1e-3
5944		 #succ
10/10
DEMO Tusar bbob-biobj 355 (799)158 (12)51 (5)42 (7)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.5)★21.0 (0.4)★21.0 (0.2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)640 (870)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 111 (275)375 (338)1654 (1976)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 9.4 (12)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 142 (173)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2793 (256)29 (1)4.1 (0.1)1.9 (0.2)0/10
SMS-EMOA-DE Auger bbob-biobj 4079 (329)44 (2)5457 (1e4)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2344 (257)37 (3)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 91 (148)128 (14)57 (7)35 (0.5)10/10

Δ HVref
f10, 20-D		 1e0
1		 1e-1
6124		 1e-2
2.0e5		 1e-3
2.4e5		 #succ
10/10
DEMO Tusar bbob-biobj 11 (15)10 (3)5.2 (3)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)24 (2)1.5 (0.8)2.8 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14820/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞14950/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 94 (0)223 (344)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.3 (0.8)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 10 (7)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2443 (170)1.0 (0.3)4.3 (0.1)20 (27)0/10
SMS-EMOA-DE Auger bbob-biobj 4000 (354)85 (245)15 (15)74 (109)0/10
SMS-EMOA-PM Auger bbob-biobj 2133 (260)221 (572)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 7.3 (11)7.0 (2)1.0 (8e-3)1.0 (0.1)10/10

Δ HVref
f11, 20-D		 1e0
3.0		 1e-1
246		 1e-2
3177		 1e-3
1.1e5		 #succ
1/10
DEMO Tusar bbob-biobj 6.1 (3)18 (19)29 (19)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)1.00 (0.6)1.0 (0.7)1.00 (0.8)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 75 (373)158 (101)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 172 (860)378 (192)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 388 (0)769 (630)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 84 (149)19 (27)215 (344)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (10)414 (989)6040 (1e4)∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 4.6 (3)1215 (44)5.9e4 (9e4)∞2e70/10
RM-MEDA Auger bbob-biobj 1777 (363)98 (38)200 (322)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 634 (174)17 (2)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 380 (97)14 (3)6219 (6295)∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 2.9 (4)17 (11)39 (21)2.4 (0.4)1/10

Δ HVref
f12, 20-D		 1e0
11		 1e-1
131		 1e-2
1483		 1e-3
11435		 #succ
10/10
DEMO Tusar bbob-biobj 4.6 (8)85 (85)49 (21)20 (7)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1.0)1 (0.5)★21.0 (0.9)★1.00 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 11 (14)15 (6)123 (125)∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 68 (91)1461 (1795)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 318 (271)1435 (1528)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 38 (50)142 (143)236 (346)779 (1093)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (0.8)1.7e4 (3e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.4 (3)6.0e4 (1e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 474 (175)230 (25)1041 (1678)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 374 (229)65 (25)10 (4)1576 (2711)0/10
SMS-EMOA-PM Auger bbob-biobj 206 (58)58 (40)37 (120)1577 (2011)0/10
UP-MO-CMA-ES Krause bbob-biobj 1.7 (1)63 (23)34 (10)17 (0.4)10/10

Δ HVref
f13, 20-D		 1e0
7.0		 1e-1
139		 1e-2
1087		 1e-3
17899		 #succ
7/10
DEMO Tusar bbob-biobj 89 (300)143 (64)51 (11)51 (89)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (1)1.0 (0.6)★21.00 (0.8)★21.00 (0.3)6/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14960/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 13 (10)78 (117)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 220 (940)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 74 (198)647 (441)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 77 (87)128 (234)782 (1463)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 6.3 (19)6226 (1e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 98 (229)5.9e4 (8e4)∞∞2e70/10
RM-MEDA Auger bbob-biobj 591 (116)42 (6)8.7 (2)213 (178)0/10
SMS-EMOA-DE Auger bbob-biobj 713 (50)1649 (3602)7368 (6900)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 458 (94)36 (2)1495 (4975)∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 33 (23)70 (29)55 (30)12 (0.5)7/10

Δ HVref
f14, 20-D		 1e0
3.0		 1e-1
349		 1e-2
6102		 1e-3
63789		 #succ
10/10
DEMO Tusar bbob-biobj 3.6 (5)214 (50)168 (99)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.00 (0.8)★21.00 (0.2)1.00 (0.5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 14 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 290 (725)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 311 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 112 (157)4054 (3305)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.6 (3)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.9 (7)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 1391 (139)28 (5)6.0 (4)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 1520 (230)∞∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 900 (200)∞∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 8.2 (3)154 (42)33 (1.0)40 (2)2/10

Δ HVref
f15, 20-D		 1e0
6.0		 1e-1
210		 1e-2
2099		 1e-3
25094		 #succ
9/10
DEMO Tusar bbob-biobj 33 (77)163 (79)72 (13)60 (23)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.92 (2)1.00 (0.3)★21.00 (0.3)1.0 (0.1)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14960/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 15 (37)179 (226)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 172 (430)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 293 (440)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 74 (73)192 (382)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (3)2.3e4 (2e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 8.4 (2)2.4e5 (4e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 938 (165)61 (18)66 (262)373 (391)0/10
SMS-EMOA-DE Auger bbob-biobj 1160 (98)2.2e4 (2e4)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 889 (345)4152 (1e4)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 8.3 (2)92 (47)58 (9)8.7 (0.7)9/10

Δ HVref
f16, 20-D		 1e0
6.0		 1e-1
32242		 1e-2
1.0e6		 1e-3
1.9e7		 #succ
0/10
DEMO Tusar bbob-biobj 7.6 (5)19 (63)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (2)1.0 (0.3)1.0 (3)1 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14880/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.2 (17)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 45 (89)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 378 (2509)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 85 (136)84 (85)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (4)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.1 (16)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 963 (137)2.2 (1)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 1138 (248)∞∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 701 (56)∞∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 12 (22)74 (156)∞∞2e70/10

Δ HVref
f17, 20-D		 1e0
1		 1e-1
33890		 1e-2
9.3e5		 1e-3
2.7e7		 #succ
0/10
DEMO Tusar bbob-biobj 15 (23)21 (20)21 (11)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.3 (0.8)1.00 (2)1 (1.0)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14980/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)2.6 (4)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 227 (455)139 (221)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 9.1 (30)237 (266)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 36 (73)2363 (2950)∞∞2e70/10
RM-MEDA Auger bbob-biobj 4733 (555)1.0 (0.7)9.2 (7)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 5528 (542)236 (384)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 3547 (758)138 (472)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 20 (24)593 (1180)∞∞2e70/10

Δ HVref
f18, 20-D		 1e0
6.0		 1e-1
141		 1e-2
1507		 1e-3
17541		 #succ
10/10
DEMO Tusar bbob-biobj 13 (23)117 (28)30 (3)24 (8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.8)1.00 (0.3)★21.0 (0.3)1.0 (0.4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14980/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.0 (13)35 (21)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 84 (83)279 (248)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 131 (408)181 (191)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 62 (78)27 (26)54 (23)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (0.8)1272 (1741)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 7.2 (15)4.8e4 (9e4)∞∞2e70/10
RM-MEDA Auger bbob-biobj 862 (208)56 (12)8.4 (2)87 (154)0/10
SMS-EMOA-DE Auger bbob-biobj 802 (33)2.1e4 (1e4)1.2e4 (7963)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 496 (145)40 (10)5974 (3981)∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.2 (5)61 (19)35 (12)12 (0.4)10/10

Δ HVref
f19, 20-D		 1e0
7.0		 1e-1
18889		 1e-2
2.4e5		 1e-3
5.1e6		 #succ
6/10
DEMO Tusar bbob-biobj 61 (150)19 (4)15 (27)3.6 (3)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (1)1.0 (2)5.7 (24)1.00 (2)6/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14810/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 15 (23)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 87 (112)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 158 (257)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 69 (114)42 (53)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (4)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.2 (1)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 600 (52)27 (27)20 (17)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 714 (99)∞∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 435 (37)∞∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 11 (24)2.4 (0.5)1.0 (0.7)3.9 (4)3/10

Δ HVref
f20, 20-D		 1e0
3.0		 1e-1
136		 1e-2
1870		 1e-3
8641		 #succ
10/10
DEMO Tusar bbob-biobj 34 (23)168 (48)46 (42)52 (34)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (0.5)★21.00 (1)1.0 (2)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14880/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.2 (0)19 (11)29 (26)∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 14 (33)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 14 (0)1388 (1656)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 101 (83)56 (103)570 (182)330 (158)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 7.9 (9)6.1e4 (1e5)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 19 (23)3.7e5 (5e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 994 (391)52 (38)13 (8)91 (192)1/10
SMS-EMOA-DE Auger bbob-biobj 1844 (598)788 (3400)293 (547)255 (192)0/10
SMS-EMOA-PM Auger bbob-biobj 1086 (1103)3644 (2583)1626 (2679)2094 (3414)0/10
UP-MO-CMA-ES Krause bbob-biobj 26 (43)87 (36)26 (31)13 (10)10/10

Δ HVref
f21, 20-D		 1e0
5.0		 1e-1
331		 1e-2
1753		 1e-3
9785		 #succ
8/10
DEMO Tusar bbob-biobj 181 (447)84 (36)47 (10)64 (63)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.98 (2)1.00 (0.6)★21.0 (0.7)1 (1)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 18 (40)12 (31)113 (94)∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 164 (404)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 447 (3006)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 126 (0)84 (133)1993 (3252)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.7 (14)1.1e4 (8630)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 43 (76)1.0e5 (2e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 767 (186)19 (5)6.6 (4)27 (27)0/10
SMS-EMOA-DE Auger bbob-biobj 1537 (294)31 (4)497 (856)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1162 (283)109 (359)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 55 (206)40 (18)40 (23)21 (3)8/10

Δ HVref
f22, 20-D		 1e0
1		 1e-1
313		 1e-2
4469		 1e-3
23797		 #succ
10/10
DEMO Tusar bbob-biobj 4.2 (9)196 (31)77 (19)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.7)★21.0 (0.4)1 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14960/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)6470 (3581)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.3 (6)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 15 (28)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 3270 (201)26 (3)4.9 (3)83 (155)0/10
SMS-EMOA-DE Auger bbob-biobj 6929 (1446)2788 (5)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 4826 (1330)∞∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 63 (1)154 (24)45 (0.3)10 (0.6)7/10

Δ HVref
f23, 20-D		 1e0
3.0		 1e-1
332		 1e-2
2660		 1e-3
16975		 #succ
10/10
DEMO Tusar bbob-biobj 12 (41)137 (43)66 (22)105 (155)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (4)1.00 (2)★21.00 (0.3)1.00 (0.6)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 3.3 (0)42 (102)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 12 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 6.9 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 110 (190)710 (168)7301 (3196)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 7.4 (29)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 5.1 (4)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2715 (1987)100 (53)21 (9)67 (62)0/10
SMS-EMOA-DE Auger bbob-biobj 6621 (2432)788 (34)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 5276 (2899)6389 (1e4)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 6.9 (6)65 (26)56 (21)13 (0.8)10/10

Δ HVref
f24, 20-D		 1e0
1		 1e-1
61504		 1e-2
8.1e5		 1e-3
6.5e7		 #succ
0/10
DEMO Tusar bbob-biobj 26 (42)9.3 (8)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.2 (0.5)1 (2)1.0 (2)1.00 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14940/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (0)2.9 (2)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 38 (183)3.2 (3)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 14 (33)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 589 (202)293 (203)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 6.9 (4)297 (252)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (5)2978 (3333)∞∞2e70/10
RM-MEDA Auger bbob-biobj 4914 (784)4.8 (0.9)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 1.1e4 (1816)50 (41)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1.2e4 (1e4)293 (236)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 32 (54)38 (0.9)∞∞2e70/10

Δ HVref
f25, 20-D		 1e0
1		 1e-1
34160		 1e-2
9.2e5		 1e-3
1.2e7		 #succ
0/10
DEMO Tusar bbob-biobj 9.4 (14)11 (19)20 (23)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (1.0)1.00 (0.9)1.00 (0.7)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)1.0 (3)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 193 (402)42 (120)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.0 (2)110 (136)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (6)908 (1459)∞∞2e70/10
RM-MEDA Auger bbob-biobj 3949 (607)8.0 (29)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 8568 (1306)40 (59)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 5990 (1484)88 (59)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 18 (37)253 (148)∞∞2e70/10

Δ HVref
f26, 20-D		 1e0
3.0		 1e-1
136		 1e-2
1117		 1e-3
4425		 #succ
9/10
DEMO Tusar bbob-biobj 540 (2646)191 (64)61 (49)47 (30)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)1 (0.3)★21.0 (1)★1.00 (1.0)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14940/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 25 (62)14 (9)44 (81)44 (37)0/10
MO-DIRECT-ND Wong bbob-biobj 194 (0)721 (926)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 742 (3337)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 53 (49)62 (107)315 (160)807 (776)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 392 (976)1.7e4 (1e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2783 (3)2.4e5 (4e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 1413 (411)49 (13)10 (6)10 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 2316 (381)71 (10)12 (4)681 (904)0/10
SMS-EMOA-PM Auger bbob-biobj 1726 (528)88 (44)2715 (1357)1092 (1472)0/10
UP-MO-CMA-ES Krause bbob-biobj 244 (1158)84 (47)37 (17)31 (18)8/10

Δ HVref
f27, 20-D		 1e0
1		 1e-1
60235		 1e-2
1.3e6		 1e-3
6.4e6		 #succ
7/10
DEMO Tusar bbob-biobj 6.7 (6)16 (34)6.3 (3)2.9 (1)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0)2.4 (8)1.0 (1)1.00 (1)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14840/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 25 (121)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 80 (396)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 2225 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 400 (468)58 (77)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.3 (3)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 19 (34)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 3549 (480)22 (66)3.6 (3)1.3 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 7077 (918)133 (216)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 4869 (802)299 (365)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 18 (39)1.00 (0.8)1.5 (3)1.2 (2)7/10

Δ HVref
f28, 20-D		 1e0
1		 1e-1
62		 1e-2
539		 1e-3
6898		 #succ
8/10
DEMO Tusar bbob-biobj 1492 (2186)286 (85)94 (9)45 (16)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.5)★21.0 (0.7)★21.00 (0.4)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14750/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)53 (5)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)374 (380)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)282 (211)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 195 (486)441 (438)2960 (1862)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 27 (7)1.6e5 (2e5)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1182 (971)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2944 (269)68 (4)12 (0.9)5.0 (8)0/10
SMS-EMOA-DE Auger bbob-biobj 5337 (502)117 (14)950 (1855)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 3326 (448)100 (29)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 441 (311)175 (33)85 (15)29 (0.3)8/10

Δ HVref
f29, 20-D		 1e0
1		 1e-1
359		 1e-2
3841		 1e-3
30739		 #succ
8/10
DEMO Tusar bbob-biobj 222 (529)145 (22)103 (31)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (1)★21.00 (0.2)1.0 (0.4)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 126 (623)1327 (511)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 9.1 (15)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 50 (198)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2781 (202)15 (0.8)3.5 (1)178 (144)0/10
SMS-EMOA-DE Auger bbob-biobj 4853 (322)2.2e4 (3e4)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2842 (208)∞∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 63 (131)131 (36)52 (0.1)8.2 (0.3)5/10

Δ HVref
f30, 20-D		 1e0
1		 1e-1
220		 1e-2
1777		 1e-3
10156		 #succ
9/10
DEMO Tusar bbob-biobj 182 (16)107 (39)61 (14)109 (34)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.5)1.0 (0.6)★1.00 (0.3)1.00 (0.2)9/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14840/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 10 (42)71 (50)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 20 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 13 (31)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 183 (454)293 (421)5082 (4897)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 9.2 (15)2.2e4 (2e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 41 (68)2.4e5 (3e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 3498 (285)26 (4)5.9 (2)219 (162)0/10
SMS-EMOA-DE Auger bbob-biobj 5888 (813)43 (7)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 3655 (762)2316 (6818)1.0e4 (1e4)∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 43 (14)90 (62)63 (16)21 (0.6)7/10

Δ HVref
f31, 20-D		 1e0
3.0		 1e-1
27722		 1e-2
8.6e5		 1e-3
3.0e7		 #succ
0/10
DEMO Tusar bbob-biobj 203 (32)55 (91)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (2)1.0 (0.1)1.00 (1)1.00 (0.9)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.6 (14)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 18 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 9.1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 45 (110)657 (433)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 12 (4)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 65 (188)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 1076 (85)1.9 (2)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 1971 (285)289 (253)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1122 (76)650 (595)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 64 (20)370 (461)∞∞2e70/10

Δ HVref
f32, 20-D		 1e0
6.0		 1e-1
32985		 1e-2
1.1e6		 1e-3
2.6e7		 #succ
0/10
DEMO Tusar bbob-biobj 142 (660)13 (21)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.93 (2)2.1 (2)1.0 (1)1.00 (0.2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14960/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.9 (14)6.0 (7)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 94 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 36 (84)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 99 (68)149 (171)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 11 (22)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3176 (93)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 525 (62)1.0 (1)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 917 (105)142 (273)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 535 (124)142 (409)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 95 (220)1416 (909)∞∞2e70/10

Δ HVref
f33, 20-D		 1e0
1		 1e-1
33		 1e-2
278		 1e-3
3348		 #succ
6/10
DEMO Tusar bbob-biobj 866 (1176)445 (63)135 (22)65 (69)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.4)★21.00 (0.1)★21.0 (0.4)6/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14850/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)22 (6)151 (134)∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)1005 (950)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)252 (61)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 106 (262)578 (404)2412 (2995)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 51 (85)6.7e4 (8e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 160 (114)1.6e6 (3e6)∞∞2e70/10
RM-MEDA Auger bbob-biobj 2915 (178)118 (5)18 (1)6.9 (16)0/10
SMS-EMOA-DE Auger bbob-biobj 4720 (407)191 (17)31 (3)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2902 (519)147 (13)107 (50)∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 510 (852)273 (43)116 (19)50 (15)6/10

Δ HVref
f34, 20-D		 1e0
3.0		 1e-1
45588		 1e-2
1.9e5		 1e-3
2.3e5		 #succ
8/10
DEMO Tusar bbob-biobj 82 (395)1.2 (0.5)17 (21)41 (42)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (4)1.1 (0.0)19 (70)24 (35)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.6 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 19 (48)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 13 (62)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 49 (241)17 (15)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 64 (316)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1484 (9)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 954 (80)5.0 (11)16 (18)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 1536 (413)103 (143)95 (145)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 945 (165)395 (472)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 134 (13)1.0 (0.3)1.00 (0.2)1.00 (0.1)6/10

Δ HVref
f35, 20-D		 1e0
1		 1e-1
338		 1e-2
4782		 1e-3
41429		 #succ
10/10
DEMO Tusar bbob-biobj 5.5 (10)123 (14)85 (47)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.3)★21.0 (0.2)1.00 (0.1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)590 (548)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)1968 (1968)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (1)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.1 (0.8)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2346 (211)15 (0.8)2.4 (0.7)124 (169)0/10
SMS-EMOA-DE Auger bbob-biobj 4106 (482)29 (3)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2122 (334)4039 (1247)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 1.5 (1)128 (14)43 (0.2)6.8 (0.7)2/10

Δ HVref
f36, 20-D		 1e0
1		 1e-1
454		 1e-2
4824		 1e-3
48159		 #succ
10/10
DEMO Tusar bbob-biobj 6.3 (8)125 (17)203 (76)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.6)★21.0 (0.2)1.0 (1)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14980/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)429 (529)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)4570 (5328)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.0 (4)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.4 (4)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 3279 (176)14 (1)52 (6)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 5357 (445)31 (11)652 (312)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 3476 (389)225 (225)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 10 (10)120 (40)42 (0.3)5.4 (0.2)4/10

Δ HVref
f37, 20-D		 1e0
1		 1e-1
10528		 1e-2
5.2e5		 1e-3
3.5e7		 #succ
0/10
DEMO Tusar bbob-biobj 27 (112)57 (75)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (1)1.0 (1)1.0 (2)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14880/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 7.6 (15)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 33 (152)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2982 (446)4.5 (6)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 5515 (1430)5.4 (6)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2654 (563)458 (571)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 74 (179)12 (7)∞∞2e70/10

Δ HVref
f38, 20-D		 1e0
3.0		 1e-1
36705		 1e-2
3.9e6		 1e-3
2.2e8		 #succ
0/10
DEMO Tusar bbob-biobj 4.7 (0.6)111 (160)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (4)1.6 (2)1.00 (0.9)1 (0.9)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 18 (87)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 177 (882)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 243 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 64 (0)∞∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.2 (6)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 13 (28)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 964 (49)1 (0.8)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 1706 (226)86 (102)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1009 (88)250 (286)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.8 (3)240 (139)∞∞2e70/10

Δ HVref
f39, 20-D		 1e0
1		 1e-1
457		 1e-2
3876		 1e-3
25958		 #succ
10/10
DEMO Tusar bbob-biobj 19 (31)101 (27)120 (10)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.9)★21.0 (0.3)1.00 (0.4)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14980/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)2584 (2479)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.6 (2)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 24 (59)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2688 (224)12 (1)3.2 (0.8)96 (135)0/10
SMS-EMOA-DE Auger bbob-biobj 4127 (522)508 (5)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2425 (315)1.8e4 (3e4)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 14 (6)101 (46)625 (1290)95 (0.3)5/10

Δ HVref
f40, 20-D		 1e0
1		 1e-1
26134		 1e-2
2.1e5		 1e-3
2.6e6		 #succ
7/10
DEMO Tusar bbob-biobj 1.4 (0.5)2.8 (1)10 (6)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1 (0.0)27 (55)4.3 (5)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14810/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)144 (120)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (2)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.4 (5)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2349 (251)8.9 (0.4)10 (7)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 3828 (620)13 (9)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2130 (393)328 (259)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.0 (0.2)2.1 (0.5)1 (0.0)1.00 (6)4/10

Δ HVref
f41, 20-D		 1e0
2.0		 1e-1
245		 1e-2
2389		 1e-3
15818		 #succ
10/10
DEMO Tusar bbob-biobj 15 (44)116 (35)83 (17)286 (258)0/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (0)1.00 (0.8)★1.00 (0.2)1.0 (0.5)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 18 (43)79 (76)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 60 (0)785 (735)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 48 (0)761 (572)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 107 (280)456 (1001)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.5 (6)1.3e4 (2e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 42 (99)1.4e5 (1e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 2210 (554)33 (12)8.1 (4)21 (25)0/10
SMS-EMOA-DE Auger bbob-biobj 5379 (1653)85 (26)30 (26)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 3290 (1658)88 (45)7797 (7535)∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 86 (4)86 (55)62 (21)14 (0.7)10/10

Δ HVref
f42, 20-D		 1e0
1		 1e-1
34578		 1e-2
2.9e6		 1e-3
2.3e7		 #succ
0/10
DEMO Tusar bbob-biobj 24 (35)17 (33)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (0.6)1.00 (2)1.0 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)5.7 (9)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 191 (474)149 (159)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.1 (4)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (5)5406 (6941)∞∞2e70/10
RM-MEDA Auger bbob-biobj 4502 (820)18 (5)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 8267 (280)5.6 (12)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 5042 (962)142 (140)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 18 (27)3.1 (2)∞∞2e70/10

Δ HVref
f43, 20-D		 1e0
2.0		 1e-1
55957		 1e-2
2.3e6		 1e-3
3.9e7		 #succ
0/10
DEMO Tusar bbob-biobj 3.1 (2)5.7 (10)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (0)1.3 (2)1.00 (0.8)1.0 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 25 (61)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 104 (299)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 345 (862)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 122 (214)162 (164)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.5 (6)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.0 (2)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 1935 (156)1 (1)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 3108 (390)58 (39)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2120 (391)148 (125)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 4.8 (8)242 (536)∞∞2e70/10

Δ HVref
f44, 20-D		 1e0
3.0		 1e-1
168		 1e-2
1485		 1e-3
6584		 #succ
10/10
DEMO Tusar bbob-biobj 54 (6)124 (65)56 (33)70 (88)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (1)1.00 (0.5)★21.00 (0.3)★21.00 (0.3)10/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14960/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 13 (32)74 (41)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 57 (43)342 (360)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 39 (85)315 (337)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 93 (204)261 (540)1477 (3174)∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 7.9 (18)1.7e4 (2e4)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 64 (142)4.8e5 (5e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 1246 (202)34 (5)6.1 (1)3.3 (0.9)0/10
SMS-EMOA-DE Auger bbob-biobj 1726 (262)49 (7)3151 (4713)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 989 (196)1361 (15)1.2e4 (2e4)∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 39 (80)98 (76)60 (11)30 (3)9/10

Δ HVref
f45, 20-D		 1e0
1		 1e-1
49797		 1e-2
2.4e6		 1e-3
5.2e6		 #succ
7/10
DEMO Tusar bbob-biobj 4.0 (4)6.3 (11)7.8 (4)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.1 (0.2)1.6 (1)1.2 (2)1.4 (3)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14960/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 9.1 (40)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 40 (196)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 75 (370)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 170 (467)32 (48)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.3 (5)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (22)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 3340 (194)10 (0.0)3.3 (9)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 5408 (350)5.6 (11)7.5 (14)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 3163 (362)28 (30)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 4.7 (4)1.00 (0.3)1.0 (2e-3)1.0 (2)7/10

Δ HVref
f46, 20-D		 1e0
1		 1e-1
55141		 1e-2
1.0e6		 1e-3
1.1e8		 #succ
0/10
DEMO Tusar bbob-biobj 16 (2)94 (122)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.8)1.00 (0.7)1 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 81 (0)∞∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (0.5)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (5)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 4409 (948)6.1 (10)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 7415 (1356)18 (0.9)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 4104 (496)∞∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 3.8 (4)366 (725)∞∞2e70/10

Δ HVref
f47, 20-D		 1e0
2.0		 1e-1
54618		 1e-2
2.2e6		 1e-3
5.2e7		 #succ
0/10
DEMO Tusar bbob-biobj 8.2 (21)161 (126)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.85 (0.9)1.00 (0.1)1 (0.5)1.0 (0.7)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14740/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.8 (16)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 55 (137)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 22 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 45 (112)331 (412)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.8 (4)356 (302)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 8.4 (21)3574 (4394)∞∞2e70/10
RM-MEDA Auger bbob-biobj 1715 (79)1.7 (1)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 3194 (364)87 (83)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1771 (364)330 (284)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 5.0 (4)3296 (2746)∞∞2e70/10

Δ HVref
f48, 20-D		 1e0
1		 1e-1
22301		 1e-2
3.5e5		 1e-3
3.3e7		 #succ
0/10
DEMO Tusar bbob-biobj 44 (22)15 (39)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.6 (1)1.0 (1.0)1.0 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14960/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)8.1 (11)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)8.5 (11)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 91 (450)220 (384)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 3.2 (6)829 (583)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 100 (459)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 3254 (413)1.00 (1.0)51 (41)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 4989 (554)1.8 (2)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2841 (744)223 (413)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 82 (366)229 (674)∞∞2e70/10

Δ HVref
f49, 20-D		 1e0
2.0		 1e-1
1.6e5		 1e-2
1.1e7		 1e-3
7.1e7		 #succ
0/10
DEMO Tusar bbob-biobj 3.2 (5)10 (19)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (1)1.00 (2)1.0 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14880/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 5.0 (22)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 8.2 (38)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 8.2 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 52 (256)113 (64)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.4 (4)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.2 (1)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 1414 (100)8.9 (16)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 2409 (228)5.8 (9)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1346 (236)54 (78)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 4.2 (6)32 (61)∞∞2e70/10

Δ HVref
f50, 20-D		 1e0
3.0		 1e-1
22169		 1e-2
1.3e6		 1e-3
2.0e7		 #succ
0/10
DEMO Tusar bbob-biobj 4.0 (7)71 (70)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 0.90 (0)2.9 (1)1 (0.5)1.00 (1.0)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14880/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 8.8 (11)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 102 (255)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 22 (52)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 72 (180)∞∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.7 (0.7)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.0 (4)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 1004 (53)1.0 (0.5)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 2692 (4598)365 (407)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1062 (149)∞∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 1.8 (1)∞∞∞2e70/10

Δ HVref
f51, 20-D		 1e0
1		 1e-1
8574		 1e-2
9.9e5		 1e-3
2.2e7		 #succ
0/10
DEMO Tusar bbob-biobj 57 (33)12 (5)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)4.6 (7)1.00 (0.7)1.0 (1)0/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)21 (23)∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 269 (398)566 (535)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.3 (9)2211 (2274)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 15 (8)2.2e4 (4665)∞∞2e70/10
RM-MEDA Auger bbob-biobj 2982 (239)1.00 (0.4)18 (25)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 4426 (365)161 (408)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2713 (416)596 (616)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 46 (58)1569 (1168)∞∞2e70/10

Δ HVref
f52, 20-D		 1e0
1		 1e-1
82166		 1e-2
4.9e6		 1e-3
2.4e7		 #succ
1/10
DEMO Tusar bbob-biobj 6.4 (9)16 (20)∞∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.5)1.00 (0.7)1 (1)1/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14910/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 39 (0)240 (225)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (2)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 5.5 (10)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2537 (274)1.7 (3)∞∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 4388 (440)28 (19)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 2694 (448)43 (41)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 2.3 (2)106 (183)∞∞2e70/10

Δ HVref
f53, 20-D		 1e0
1		 1e-1
28		 1e-2
159		 1e-3
1952		 #succ
8/10
DEMO Tusar bbob-biobj 123 (89)394 (61)157 (15)29 (8)0/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.0 (0.3)★21.00 (0.1)★21.0 (0.5)8/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14980/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)6.6 (0.6)54 (59)∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)157 (116)∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)89 (43)∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)123 (32)147 (119)124 (123)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 4.6 (12)5129 (2906)∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 39 (99)2.5e5 (3e5)∞∞2e70/10
RM-MEDA Auger bbob-biobj 2909 (263)142 (6)31 (2)4.1 (0.8)0/10
SMS-EMOA-DE Auger bbob-biobj 4138 (338)202 (9)44 (3)261 (0.6)0/10
SMS-EMOA-PM Auger bbob-biobj 2507 (406)147 (9)1808 (2608)2401 (2312)0/10
UP-MO-CMA-ES Krause bbob-biobj 37 (48)238 (38)138 (28)36 (22)6/10

Δ HVref
f54, 20-D		 1e0
7.0		 1e-1
6049		 1e-2
1.6e5		 1e-3
2.5e6		 #succ
8/10
DEMO Tusar bbob-biobj 165 (76)7.5 (2)29 (52)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1.0 (4)1.3 (1.0)3.0 (5)2.8 (10)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14860/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 6.0 (6)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 89 (222)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 89 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 66 (118)155 (104)∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 13 (12)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 479 (1116)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 408 (46)1.00 (0.2)49 (30)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 620 (97)222 (496)∞∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 369 (33)1330 (1904)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 70 (297)6.7 (1)1.0 (0.2)1.00 (0.0)8/10

Δ HVref
f55, 20-D		 1e0
1		 1e-1
8.4e5		 1e-2
8.8e6		 1e-3
8.8e6		 #succ
7/10
DEMO Tusar bbob-biobj 1 (0)2.5 (2)2.1 (2)∞2e60/10
HMO-CMA-ES Loshchilov bbob-biobj 1 (0)1.00 (3)1.5 (3)1.7 (2)7/10
MAT-DIRECT Al-Dujaili bbob-biobj ∞∞∞∞14760/10
MAT-SMS Al-Dujaili bbob-biobj ∞∞∞∞15000/10
MO-DIRECT-HV-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-ND Wong bbob-biobj 1 (0)∞∞∞2e40/10
MO-DIRECT-Rank Wong bbob-biobj 1 (0)∞∞∞2e40/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞2e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞2e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0)∞∞∞2e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1 (0)∞∞∞2e70/10
RM-MEDA Auger bbob-biobj 2204 (202)2.4 (2)2.1 (3)∞2e60/10
SMS-EMOA-DE Auger bbob-biobj 3955 (222)21 (17)2.1 (1)∞2e60/10
SMS-EMOA-PM Auger bbob-biobj 1785 (59)22 (26)∞∞2e60/10
UP-MO-CMA-ES Krause bbob-biobj 1 (0)2.7 (6)1 (2)1.00 (2)5/10

Previous dimension | Dimension = 40 | First dimension

Δ HVref
f1, 40-D		 1e0
1		 1e-1
6023		 1e-2
9415		 1e-3
3.4e5		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 5.1 (0.5)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 77 (169)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3510 (165)1 (0.0)★21.00 (0.0)★23.8 (3)0/10
SMS-EMOA-DE Auger bbob-biobj 2.2e4 (6e4)10 (10)9.4 (6)1.0 (0.2)0/10
SMS-EMOA-PM Auger bbob-biobj 3907 (394)12 (9)∞∞4e60/10

Δ HVref
f2, 40-D		 1e0
8.0		 1e-1
8937		 1e-2
36920		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 33 (79)1331 (947)∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.95 (1)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 7.4 (5)∞∞ 0/10
RM-MEDA Auger bbob-biobj 631 (174)1.00 (0.1)★21.0 (3)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 5022 (8867)4030 (4028)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 597 (135)114 (559)∞ 0/10

Δ HVref
f3, 40-D		 1e0
9.0		 1e-1
9126		 1e-2
14466		 1e-3
1.8e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 14 (35)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (4)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 13 (5)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 595 (56)1.0 (0.1)★21.0 (0.1)★21.00 (1)★20/10
SMS-EMOA-DE Auger bbob-biobj 2895 (1364)1758 (1753)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 1223 (164)3947 (4164)∞∞4e60/10

Δ HVref
f4, 40-D		 1e0
1		 1e-1
6096		 1e-2
9522		 1e-3
9.9e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 13 (24)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2436 (5570)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3838 (180)1.0 (0.0)★21.0 (0.1)★21 (0.7)★20/10
SMS-EMOA-DE Auger bbob-biobj 1.5e4 (5690)443 (986)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 5058 (560)288 (656)∞∞4e60/10

Δ HVref
f5, 40-D		 1e0
1		 1e-1
7059		 1e-2
18327		 1e-3
5.1e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 30 (2)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3460 (248)1.00 (0.0)★21.0 (0.2)★21 (1)★20/10
SMS-EMOA-DE Auger bbob-biobj 3.3e4 (6886)10 (12)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 3974 (505)36 (25)∞∞4e60/10

Δ HVref
f6, 40-D		 1e0
5.0		 1e-1
7488		 1e-2
13828		 1e-3
2.1e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 25 (61)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 0.96 (1)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 7.3 (13)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 862 (94)1.0 (0.1)★21.0 (0.2)★21.00 (2)★20/10
SMS-EMOA-DE Auger bbob-biobj 5568 (4763)66 (4)51 (218)∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 1181 (306)9.4 (10)∞∞4e60/10

Δ HVref
f7, 40-D		 1e0
28		 1e-1
26319		 1e-2
4.7e6		 1e-3
2.0e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 3.0 (15)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (2)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.1 (5)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 148 (7)1.00 (3)1.00 (1)★1 (0.7)★0/10
SMS-EMOA-DE Auger bbob-biobj 650 (497)28 (6)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 183 (33)250 (619)∞∞4e60/10

Δ HVref
f8, 40-D		 1e0
32		 1e-1
23042		 1e-2
1.7e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 7.5 (9)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.98 (2)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 12 (28)∞∞ 0/10
RM-MEDA Auger bbob-biobj 121 (5)1 (0.9)★1 (1)★ 0/10
SMS-EMOA-DE Auger bbob-biobj 2.3e4 (9e4)703 (1513)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 150 (21)759 (1108)∞ 0/10

Δ HVref
f9, 40-D		 1e0
1		 1e-1
6019		 1e-2
8971		 1e-3
27077		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)2906 (3210)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 38 (156)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2927 (5573)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3872 (376)1.00 (0.0)★21.0 (0.1)★21 (0.2)★20/10
SMS-EMOA-DE Auger bbob-biobj 9.5e4 (2e5)109 (201)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 4413 (538)2.4 (0.3)∞∞4e60/10

Δ HVref
f10, 40-D		 1e0
1		 1e-1
1.0e6		 1e-2
4.0e6		 1e-3
1.7e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (0.5)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.0 (1)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3435 (134)1.0 (3)1 (1)1 (2)0/10
SMS-EMOA-DE Auger bbob-biobj 1.8e4 (4e4)2.8 (5)1.6 (3)2.2 (3)0/10
SMS-EMOA-PM Auger bbob-biobj 3906 (167)18 (25)∞∞4e60/10

Δ HVref
f11, 40-D		 1e0
3.0		 1e-1
9656		 1e-2
4.4e6		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 34 (0)5.4 (8)∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.93 (0.7)523 (949)∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 5.8 (13)6863 (1e4)∞ 0/10
RM-MEDA Auger bbob-biobj 2608 (670)9.3 (9)1.0 (0.9)★ 0/10
SMS-EMOA-DE Auger bbob-biobj 1.5e5 (483)278 (518)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 867 (256)1.0 (0.1)∞ 0/10

Δ HVref
f12, 40-D		 1e0
7.0		 1e-1
20487		 1e-2
1.1e6		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 18 (45)4.7 (6)7.1 (15) 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.96 (2)1836 (1562)∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 13 (25)1.9e4 (4e4)∞ 0/10
RM-MEDA Auger bbob-biobj 990 (302)17 (36)6.6 (5) 0/10
SMS-EMOA-DE Auger bbob-biobj 1086 (400)1.2 (0.7)3.8 (11) 0/10
SMS-EMOA-PM Auger bbob-biobj 700 (342)1.00 (1.0)1.00 (2) 0/10

Δ HVref
f13, 40-D		 1e0
209		 1e-1
8106		 1e-2
17455		 1e-3
1.7e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 2.8 (3)62 (223)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (4)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 810 (4030)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 27 (2)1.00 (0.1)★21.00 (0.3)★21 (0.8)★20/10
SMS-EMOA-DE Auger bbob-biobj 156 (319)744 (864)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 31 (2)1.4 (0.2)∞∞4e60/10

Δ HVref
f14, 40-D		 1e0
9.0		 1e-1
15592		 1e-2
2.9e5		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 8.9 (22)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.97 (3)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.5 (7)∞∞ 0/10
RM-MEDA Auger bbob-biobj 577 (48)1.00 (0.4)★21 (0.4)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 1594 (787)∞∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 631 (116)∞∞ 0/10

Δ HVref
f15, 40-D		 1e0
45		 1e-1
35010		 1e-2
3.8e5		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 3.4 (8)21 (9)∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.00 (3)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.4 (3)∞∞ 0/10
RM-MEDA Auger bbob-biobj 171 (20)1.0 (0.3)1.00 (2)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 660 (239)1029 (1028)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 271 (89)172 (257)∞ 0/10

Δ HVref
f16, 40-D		 1e0
11		 1e-1
2.8e6		 1e-2
∞		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 13 (0)∞  0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞  0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (4)∞  0/10
RANDOMSEARCH-5 Auger bbob-biobj 4.9 (12)∞  0/10
RM-MEDA Auger bbob-biobj 783 (199)1.00 (2)  0/10
SMS-EMOA-DE Auger bbob-biobj 3198 (3657)∞  0/10
SMS-EMOA-PM Auger bbob-biobj 812 (96)13 (14)  0/10

Δ HVref
f17, 40-D		 1e0
11		 1e-1
4.8e5		 1e-2
∞		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 31 (66)36 (56)  0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞  0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (1)∞  0/10
RANDOMSEARCH-5 Auger bbob-biobj 6.7 (25)∞  0/10
RM-MEDA Auger bbob-biobj 573 (21)1 (2)★  0/10
SMS-EMOA-DE Auger bbob-biobj 9.6e4 (7043)∞  0/10
SMS-EMOA-PM Auger bbob-biobj 683 (73)∞  0/10

Δ HVref
f18, 40-D		 1e0
95		 1e-1
11697		 1e-2
31016		 1e-3
4.0e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 2.3 (5)3.9 (4)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (2)3136 (1710)∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 70 (305)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 72 (10)1.00 (0.1)1.00 (0.3)★21 (2)★20/10
SMS-EMOA-DE Auger bbob-biobj 659 (1314)528 (355)1186 (1096)∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 69 (6)115 (106)∞∞4e60/10

Δ HVref
f19, 40-D		 1e0
5.0		 1e-1
4.0e6		 1e-2
1.6e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 20 (50)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.92 (1)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.6 (15)∞∞ 0/10
RM-MEDA Auger bbob-biobj 1076 (102)1.0 (1)1.00 (2) 0/10
SMS-EMOA-DE Auger bbob-biobj 3.1e4 (7e4)∞∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 1134 (90)∞∞ 0/10

Δ HVref
f20, 40-D		 1e0
40		 1e-1
8896		 1e-2
2.4e5		 1e-3
2.1e6		 #succ
1/10
NSGA-II-MATLAB Auger bbob-biobj 10 (10)27 (7)45 (61)19 (8)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (2)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2.9 (5)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 107 (40)1.0 (0.5)1.00 (4)1.00 (2)1/10
SMS-EMOA-DE Auger bbob-biobj 273 (211)54 (55)26 (37)18 (19)0/10
SMS-EMOA-PM Auger bbob-biobj 338 (486)85 (68)27 (66)18 (14)0/10

Δ HVref
f21, 40-D		 1e0
140		 1e-1
9081		 1e-2
18728		 1e-3
7.5e5		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1.0 (2)83 (197)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 10 (25)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 24 (116)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 43 (8)1 (0.2)★21.00 (0.9)★21.0 (2)★20/10
SMS-EMOA-DE Auger bbob-biobj 3377 (30)53 (221)1925 (3471)∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 114 (27)6.2 (11)2016 (908)∞4e60/10

Δ HVref
f22, 40-D		 1e0
1		 1e-1
21394		 1e-2
68583		 1e-3
4.8e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (2)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 19 (0.5)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 5595 (689)1.0 (0.2)★21 (0.6)★21.00 (1)★20/10
SMS-EMOA-DE Auger bbob-biobj 3.5e4 (2e4)∞∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 1.3e4 (4698)∞∞∞4e60/10

Δ HVref
f23, 40-D		 1e0
4.0		 1e-1
54998		 1e-2
97265		 1e-3
3.3e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 24 (0)16 (7)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (2)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.9 (10)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 2368 (724)1.00 (0.5)★1.0 (0.3)★21.0 (1)★20/10
SMS-EMOA-DE Auger bbob-biobj 3.1e4 (9487)22 (73)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 5.9e4 (1e5)95 (130)∞∞4e60/10

Δ HVref
f24, 40-D		 1e0
1		 1e-1
1.9e6		 1e-2
∞		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)20 (32)  0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞  0/10
RANDOMSEARCH-4 Auger bbob-biobj 11 (19)∞  0/10
RANDOMSEARCH-5 Auger bbob-biobj 33 (90)∞  0/10
RM-MEDA Auger bbob-biobj 8486 (2371)1.00 (2)  0/10
SMS-EMOA-DE Auger bbob-biobj 9.1e4 (8e4)∞  0/10
SMS-EMOA-PM Auger bbob-biobj 4.7e4 (3e4)∞  0/10

Δ HVref
f25, 40-D		 1e0
4.0		 1e-1
6.1e6		 1e-2
∞		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 12 (30)2.7 (4)  0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞  0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (2)∞  0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.8 (2)∞  0/10
RM-MEDA Auger bbob-biobj 1956 (425)1.0 (3)  0/10
SMS-EMOA-DE Auger bbob-biobj 3.6e4 (5990)∞  0/10
SMS-EMOA-PM Auger bbob-biobj 5626 (3760)∞  0/10

Δ HVref
f26, 40-D		 1e0
59		 1e-1
13218		 1e-2
22850		 1e-3
1.3e5		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1.8 (5)12 (10)431 (239)∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.00 (3)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 34 (44)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 118 (28)1.00 (0.2)★1 (0.5)★1.00 (2)★0/10
SMS-EMOA-DE Auger bbob-biobj 414 (87)2.8 (1)119 (306)276 (268)0/10
SMS-EMOA-PM Auger bbob-biobj 278 (76)4.2 (2)237 (275)288 (452)0/10

Δ HVref
f27, 40-D		 1e0
1		 1e-1
2.0e6		 1e-2
1.9e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.8 (2)∞∞ 0/10
RM-MEDA Auger bbob-biobj 5366 (346)1.0 (1)★1.0 (0.6)★ 0/10
SMS-EMOA-DE Auger bbob-biobj 3.1e4 (2e4)∞∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 1.1e4 (2687)∞∞ 0/10

Δ HVref
f28, 40-D		 1e0
244		 1e-1
7646		 1e-2
59516		 1e-3
7.7e5		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1.0 (1)660 (794)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.1 (3)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 376 (859)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 18 (0.7)1.00 (0.6)1.00 (2)★21.00 (0.7)★20/10
SMS-EMOA-DE Auger bbob-biobj 1.5e5 (3e5)4714 (5231)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 28 (2)2.1 (0.3)∞∞4e60/10

Δ HVref
f29, 40-D		 1e0
1		 1e-1
8316		 1e-2
30254		 1e-3
6.7e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.8 (2)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 75 (134)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3887 (172)1.0 (0.1)★21.0 (0.3)★21.00 (1)★20/10
SMS-EMOA-DE Auger bbob-biobj 2.6e4 (3e4)∞∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 5997 (409)∞∞∞4e60/10

Δ HVref
f30, 40-D		 1e0
98		 1e-1
8154		 1e-2
18983		 1e-3
7.0e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1.0 (3)306 (870)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (0.1)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 4.9 (7)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 50 (5)1.00 (0.1)★21.00 (0.4)★21 (2)★20/10
SMS-EMOA-DE Auger bbob-biobj 240 (109)128 (246)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 84 (10)126 (125)∞∞4e60/10

Δ HVref
f31, 40-D		 1e0
7.0		 1e-1
55825		 1e-2
9.6e6		 1e-3
3.8e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 14 (72)657 (1164)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 0.99 (0.9)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 20 (22)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 687 (32)1.0 (2)1.0 (1)★1 (1)★0/10
SMS-EMOA-DE Auger bbob-biobj 3513 (1820)646 (358)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 1003 (108)645 (752)∞∞4e60/10

Δ HVref
f32, 40-D		 1e0
57		 1e-1
28540		 1e-2
1.6e7		 1e-3
1.9e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 7.3 (9)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 0.99 (1)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 31 (23)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 76 (2)1.00 (0.8)★21.0 (1)★21.00 (0.8)★20/10
SMS-EMOA-DE Auger bbob-biobj 8846 (8804)∞∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 111 (13)∞∞∞4e60/10

Δ HVref
f33, 40-D		 1e0
1		 1e-1
5510		 1e-2
7194		 1e-3
13709		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)32 (32)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 22 (19)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1175 (1572)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 4080 (150)1.0 (0.0)★21.0 (0.0)★21.00 (0.1)★20/10
SMS-EMOA-DE Auger bbob-biobj 3.1e4 (4e4)13 (5)2235 (4448)∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 5229 (630)1.8 (0.2)2243 (840)∞4e60/10

Δ HVref
f34, 40-D		 1e0
1		 1e-1
1.7e6		 1e-2
6.0e6		 1e-3
3.7e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 40 (97)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1e4 (3e4)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3859 (402)1.0 (2)★1 (2)★1 (0.7)★0/10
SMS-EMOA-DE Auger bbob-biobj 8.1e5 (2e6)∞∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 5390 (382)∞∞∞4e60/10

Δ HVref
f35, 40-D		 1e0
1		 1e-1
7777		 1e-2
22384		 1e-3
4.2e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.3 (2)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3312 (248)1 (0.1)★21.0 (0.3)★21 (2)★20/10
SMS-EMOA-DE Auger bbob-biobj 4.6e5 (1e6)68 (4)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 4366 (577)∞∞∞4e60/10

Δ HVref
f36, 40-D		 1e0
2.0		 1e-1
11368		 1e-2
2.4e5		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 20 (96)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.1 (0.8)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 3.2 (7)∞∞ 0/10
RM-MEDA Auger bbob-biobj 2276 (166)1.0 (0.1)★21 (0.3)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 1.3e4 (2620)10 (6)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 3142 (640)30 (14)∞ 0/10

Δ HVref
f37, 40-D		 1e0
1		 1e-1
49950		 1e-2
6.8e6		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 2.9 (2)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (19)∞∞ 0/10
RM-MEDA Auger bbob-biobj 4434 (262)1.0 (0.6)★1.0 (2)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 2.6e4 (2e4)54 (55)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 6326 (1168)∞∞ 0/10

Δ HVref
f38, 40-D		 1e0
11		 1e-1
67828		 1e-2
3.7e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 12 (46)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (2)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 8.3 (0.5)∞∞ 0/10
RM-MEDA Auger bbob-biobj 385 (19)1.0 (1)★21 (2)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 1.3e4 (9704)∞∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 532 (66)574 (457)∞ 0/10

Δ HVref
f39, 40-D		 1e0
1		 1e-1
8397		 1e-2
27400		 1e-3
4.5e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.0 (4)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 35 (15)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3849 (261)1.00 (0.2)★21.0 (0.4)★21 (0.9)★20/10
SMS-EMOA-DE Auger bbob-biobj 3.9e4 (1e5)∞∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 4781 (567)∞∞∞4e60/10

Δ HVref
f40, 40-D		 1e0
1		 1e-1
9235		 1e-2
4.0e6		 1e-3
1.0e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.1 (0.2)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3317 (127)1.0 (0.1)★21.00 (1.0)1.00 (0.5)0/10
SMS-EMOA-DE Auger bbob-biobj 2.4e5 (6e5)367 (410)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 4119 (201)∞∞∞4e60/10

Δ HVref
f41, 40-D		 1e0
42		 1e-1
29058		 1e-2
77278		 1e-3
1.1e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 4.2 (6)559 (578)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (0.9)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 3.0 (2)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 164 (22)1.0 (0.3)★1.00 (0.3)★21 (2)★20/10
SMS-EMOA-DE Auger bbob-biobj 1.3e4 (1348)22 (38)33 (38)∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 655 (436)5.1 (3)∞∞4e60/10

Δ HVref
f42, 40-D		 1e0
6.0		 1e-1
1.7e6		 1e-2
∞		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 14 (36)22 (27)  0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞  0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.98 (2)∞  0/10
RANDOMSEARCH-5 Auger bbob-biobj 1.7 (0.1)∞  0/10
RM-MEDA Auger bbob-biobj 1212 (269)1.0 (0.6)  0/10
SMS-EMOA-DE Auger bbob-biobj 8.5e4 (7411)4.2 (6)  0/10
SMS-EMOA-PM Auger bbob-biobj 2087 (527)10 (10)  0/10

Δ HVref
f43, 40-D		 1e0
8.0		 1e-1
41747		 1e-2
∞		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 32 (51)∞  0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞  0/10
RANDOMSEARCH-4 Auger bbob-biobj 0.96 (2)∞  0/10
RANDOMSEARCH-5 Auger bbob-biobj 2.9 (6)∞  0/10
RM-MEDA Auger bbob-biobj 701 (39)1.00 (2)★  0/10
SMS-EMOA-DE Auger bbob-biobj 5610 (8308)867 (599)  0/10
SMS-EMOA-PM Auger bbob-biobj 1148 (175)236 (256)  0/10

Δ HVref
f44, 40-D		 1e0
6.0		 1e-1
7836		 1e-2
19356		 1e-3
1.1e5		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 17 (42)99 (17)∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 0.93 (0.4)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 5.8 (0.8)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 811 (99)1.00 (0.2)1.0 (0.4)★21.0 (1)★20/10
SMS-EMOA-DE Auger bbob-biobj 4769 (5816)64 (384)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 965 (237)1.7 (0.5)∞∞4e60/10

Δ HVref
f45, 40-D		 1e0
3.0		 1e-1
4.0e6		 1e-2
9.4e6		 1e-3
3.8e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 56 (155)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 2.4 (1)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 1.0 (0.4)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 1441 (47)1.00 (2)1 (1)1 (0.8)0/10
SMS-EMOA-DE Auger bbob-biobj 5703 (1776)1.5 (3)4.0 (3)∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 2031 (342)4.0 (3)∞∞4e60/10

Δ HVref
f46, 40-D		 1e0
2.0		 1e-1
1.6e6		 1e-2
3.8e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 46 (0)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.2 (1)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 157 (778)∞∞ 0/10
RM-MEDA Auger bbob-biobj 3687 (582)1.00 (2)★21 (0.7)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 2.6e4 (4866)∞∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 5115 (1225)∞∞ 0/10

Δ HVref
f47, 40-D		 1e0
6.0		 1e-1
1.2e5		 1e-2
4.0e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 42 (104)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0.2)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 15 (23)∞∞ 0/10
RM-MEDA Auger bbob-biobj 906 (81)1.00 (0.6)★21 (0.7)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 6922 (3344)∞∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 1441 (268)∞∞ 0/10

Δ HVref
f48, 40-D		 1e0
1		 1e-1
69919		 1e-2
1.7e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 5.0 (7)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 31 (43)∞∞ 0/10
RM-MEDA Auger bbob-biobj 4847 (275)1.00 (2)1 (1)★ 0/10
SMS-EMOA-DE Auger bbob-biobj 1.7e4 (6322)230 (129)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 5556 (545)229 (415)∞ 0/10

Δ HVref
f49, 40-D		 1e0
1		 1e-1
1.1e6		 1e-2
7.3e6		 1e-3
1.9e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.5 (0.5)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 7.1 (14)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 4347 (250)1.00 (2)1 (0.6)1.00 (1)0/10
SMS-EMOA-DE Auger bbob-biobj 1.9e4 (9162)10 (16)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 5214 (648)∞∞∞4e60/10

Δ HVref
f50, 40-D		 1e0
1		 1e-1
5.0e5		 1e-2
3.7e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 6.1 (10)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 11 (18)∞∞ 0/10
RM-MEDA Auger bbob-biobj 4456 (176)1.00 (4)★21 (1)★2 0/10
SMS-EMOA-DE Auger bbob-biobj 1.7e7 (2e7)∞∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 6693 (642)∞∞ 0/10

Δ HVref
f51, 40-D		 1e0
10		 1e-1
25050		 1e-2
1.6e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 16 (30)387 (599)∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1.0 (2)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 5.5 (9)∞∞ 0/10
RM-MEDA Auger bbob-biobj 435 (27)1.0 (2)★1 (2)★ 0/10
SMS-EMOA-DE Auger bbob-biobj 2.0e4 (5e4)651 (758)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 518 (46)378 (402)∞ 0/10

Δ HVref
f52, 40-D		 1e0
7.0		 1e-1
1.0e6		 1e-2
6.8e6		 1e-3
1.9e7		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 12 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 0.99 (4)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 7.6 (0.3)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 555 (50)1.00 (4e-3)1.0 (2)1 (0.8)0/10
SMS-EMOA-DE Auger bbob-biobj 1.1e5 (2e5)16 (31)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 710 (80)6.4 (6)∞∞4e60/10

Δ HVref
f53, 40-D		 1e0
1		 1e-1
6336		 1e-2
8127		 1e-3
12091		 #succ
2/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)7.9 (4)150 (139)3147 (4053)0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 10 (37)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 2758 (5610)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 4446 (269)1.0 (0.1)1.00 (0.1)★21.00 (0.2)★22/10
SMS-EMOA-DE Auger bbob-biobj 3.7e4 (1e4)152 (205)410 (687)2986 (5789)0/10
SMS-EMOA-PM Auger bbob-biobj 4742 (338)1.3 (0.1)145 (66)3050 (2729)0/10

Δ HVref
f54, 40-D		 1e0
1		 1e-1
1.0e6		 1e-2
6.0e6		 1e-3
9.9e6		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞∞4e60/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞∞4e60/10
RANDOMSEARCH-4 Auger bbob-biobj 1.6 (1)∞∞∞4e60/10
RANDOMSEARCH-5 Auger bbob-biobj 6.9 (6)∞∞∞4e70/10
RM-MEDA Auger bbob-biobj 3829 (120)1.00 (2e-3)★1.0 (1)★1.00 (1)★0/10
SMS-EMOA-DE Auger bbob-biobj 3.9e4 (7e4)36 (52)∞∞4e60/10
SMS-EMOA-PM Auger bbob-biobj 4487 (700)∞∞∞4e60/10

Δ HVref
f55, 40-D		 1e0
1		 1e-1
2.7e6		 1e-2
1.6e7		 1e-3
∞		 #succ
0/10
NSGA-II-MATLAB Auger bbob-biobj 1 (0)∞∞ 0/10
RANDOMSEARCH-100 Auger bbob-biobj ∞∞∞ 0/10
RANDOMSEARCH-4 Auger bbob-biobj 1 (0)∞∞ 0/10
RANDOMSEARCH-5 Auger bbob-biobj 1 (0)∞∞ 0/10
RM-MEDA Auger bbob-biobj 3162 (89)1 (2)1 (1) 0/10
SMS-EMOA-DE Auger bbob-biobj 9022 (306)1.6 (2)∞ 0/10
SMS-EMOA-PM Auger bbob-biobj 3944 (272)13 (23)∞ 0/10

Average runtime (aRT in number of function evaluations) divided by the respective best aRT measured during BBOB-2016 in different dimensions. The aRT and in braces, as dispersion measure, the half difference between 10 and 90%-tile of bootstrapped run lengths appear for each algorithm and target, the corresponding reference aRT  in the first row. The different target ∆I-values are shown in the top row. #succ is the number of trials that reached the (final) target Iref+ 10−5. The median number of conducted function evaluations is additionally given in italics, if the target in the last column was never reached. Entries, succeeded by a star, are statistically significantly better (according to the rank-sum test) when compared to all other algorithms of the table, with p = 0.05 or p = 10−k when the number k following the star is larger than 1, with Bonferroni correction by the number of functions (55). A ↓ indicates the same tested against the best algorithm from BBOB 2016. Best results are printed in bold.
Data produced with COCO v2.0.778, hv-hash=ff0e71e8cd978373