MINGYISU commited on
Commit
5d54050
·
verified ·
1 Parent(s): e5d6b2b

Upload RzenEmbed-v1-2B.json

Browse files
Files changed (1) hide show
  1. scores/RzenEmbed-v1-2B.json +2288 -0
scores/RzenEmbed-v1-2B.json ADDED
@@ -0,0 +1,2288 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "model_name": "RzenEmbed-v1-2B",
4
+ "model_size": 2.21,
5
+ "embedding_dimension": 1536,
6
+ "max_length_tokens": null,
7
+ "model_release_date": "2025-07-15",
8
+ "data_source": "RzenAI",
9
+ "url": "",
10
+ "report_generated_date": "2025-07-16T17:36:59.567353"
11
+ },
12
+ "metrics": {
13
+ "image": {
14
+ "ImageNet-1K": {
15
+ "hit@1": 0.809,
16
+ "hit@5": 0.949,
17
+ "hit@10": 0.971,
18
+ "ndcg_linear@1": 0.809,
19
+ "ndcg_linear@5": 0.8873729216700347,
20
+ "ndcg_linear@10": 0.894245384897251,
21
+ "ndcg_exponential@1": 0.809,
22
+ "ndcg_exponential@5": 0.8873729216700347,
23
+ "ndcg_exponential@10": 0.894245384897251,
24
+ "precision@1": 0.809,
25
+ "precision@5": 0.18980000000000002,
26
+ "precision@10": 0.09710000000000002,
27
+ "recall@1": 0.809,
28
+ "recall@5": 0.949,
29
+ "recall@10": 0.971,
30
+ "f1@1": 0.809,
31
+ "f1@5": 0.31633333333333336,
32
+ "f1@10": 0.17654545454545453,
33
+ "map@1": 0.809,
34
+ "map@5": 0.8664833333333334,
35
+ "map@10": 0.8691829365079365,
36
+ "mrr@1": 0.809,
37
+ "mrr@5": 0.8664833333333334,
38
+ "mrr@10": 0.8691829365079365,
39
+ "num_pred": 1000,
40
+ "num_data": 1000
41
+ },
42
+ "N24News": {
43
+ "hit@1": 0.74,
44
+ "hit@5": 0.946,
45
+ "hit@10": 0.98,
46
+ "ndcg_linear@1": 0.74,
47
+ "ndcg_linear@5": 0.8559176011970342,
48
+ "ndcg_linear@10": 0.867149138771268,
49
+ "ndcg_exponential@1": 0.74,
50
+ "ndcg_exponential@5": 0.8559176011970342,
51
+ "ndcg_exponential@10": 0.867149138771268,
52
+ "precision@1": 0.74,
53
+ "precision@5": 0.18920000000000003,
54
+ "precision@10": 0.09800000000000002,
55
+ "recall@1": 0.74,
56
+ "recall@5": 0.946,
57
+ "recall@10": 0.98,
58
+ "f1@1": 0.74,
59
+ "f1@5": 0.3153333333333333,
60
+ "f1@10": 0.1781818181818182,
61
+ "map@1": 0.74,
62
+ "map@5": 0.8254,
63
+ "map@10": 0.8301777777777777,
64
+ "mrr@1": 0.74,
65
+ "mrr@5": 0.8254,
66
+ "mrr@10": 0.8301777777777777,
67
+ "num_pred": 1000,
68
+ "num_data": 1000
69
+ },
70
+ "HatefulMemes": {
71
+ "hit@1": 0.586,
72
+ "hit@5": 1.0,
73
+ "hit@10": 1.0,
74
+ "ndcg_linear@1": 0.586,
75
+ "ndcg_linear@5": 0.8472049179785835,
76
+ "ndcg_linear@10": 0.8472049179785835,
77
+ "ndcg_exponential@1": 0.586,
78
+ "ndcg_exponential@5": 0.8472049179785835,
79
+ "ndcg_exponential@10": 0.8472049179785835,
80
+ "precision@1": 0.586,
81
+ "precision@5": 0.20000000000000004,
82
+ "precision@10": 0.10000000000000002,
83
+ "recall@1": 0.586,
84
+ "recall@5": 1.0,
85
+ "recall@10": 1.0,
86
+ "f1@1": 0.586,
87
+ "f1@5": 0.3333333333333333,
88
+ "f1@10": 0.18181818181818182,
89
+ "map@1": 0.586,
90
+ "map@5": 0.793,
91
+ "map@10": 0.793,
92
+ "mrr@1": 0.586,
93
+ "mrr@5": 0.793,
94
+ "mrr@10": 0.793,
95
+ "num_pred": 1000,
96
+ "num_data": 1000
97
+ },
98
+ "VOC2007": {
99
+ "hit@1": 0.896,
100
+ "hit@5": 0.974,
101
+ "hit@10": 0.988,
102
+ "ndcg_linear@1": 0.896,
103
+ "ndcg_linear@5": 0.9393988570118095,
104
+ "ndcg_linear@10": 0.9439297732712962,
105
+ "ndcg_exponential@1": 0.896,
106
+ "ndcg_exponential@5": 0.9393988570118095,
107
+ "ndcg_exponential@10": 0.9439297732712962,
108
+ "precision@1": 0.896,
109
+ "precision@5": 0.19480000000000003,
110
+ "precision@10": 0.09880000000000001,
111
+ "recall@1": 0.896,
112
+ "recall@5": 0.974,
113
+ "recall@10": 0.988,
114
+ "f1@1": 0.896,
115
+ "f1@5": 0.3246666666666666,
116
+ "f1@10": 0.17963636363636362,
117
+ "map@1": 0.896,
118
+ "map@5": 0.9277333333333333,
119
+ "map@10": 0.9296075396825397,
120
+ "mrr@1": 0.896,
121
+ "mrr@5": 0.9277333333333333,
122
+ "mrr@10": 0.9296075396825397,
123
+ "num_pred": 1000,
124
+ "num_data": 1000
125
+ },
126
+ "SUN397": {
127
+ "hit@1": 0.784,
128
+ "hit@5": 0.958,
129
+ "hit@10": 0.981,
130
+ "ndcg_linear@1": 0.784,
131
+ "ndcg_linear@5": 0.8836853449086659,
132
+ "ndcg_linear@10": 0.891031065357324,
133
+ "ndcg_exponential@1": 0.784,
134
+ "ndcg_exponential@5": 0.8836853449086659,
135
+ "ndcg_exponential@10": 0.891031065357324,
136
+ "precision@1": 0.784,
137
+ "precision@5": 0.19160000000000002,
138
+ "precision@10": 0.09810000000000002,
139
+ "recall@1": 0.784,
140
+ "recall@5": 0.958,
141
+ "recall@10": 0.981,
142
+ "f1@1": 0.784,
143
+ "f1@5": 0.31933333333333336,
144
+ "f1@10": 0.1783636363636363,
145
+ "map@1": 0.784,
146
+ "map@5": 0.8583333333333334,
147
+ "map@10": 0.8613071428571429,
148
+ "mrr@1": 0.784,
149
+ "mrr@5": 0.8583333333333334,
150
+ "mrr@10": 0.8613071428571429,
151
+ "num_pred": 1000,
152
+ "num_data": 1000
153
+ },
154
+ "Place365": {
155
+ "hit@1": 0.413,
156
+ "hit@5": 0.703,
157
+ "hit@10": 0.804,
158
+ "ndcg_linear@1": 0.413,
159
+ "ndcg_linear@5": 0.5692143522459587,
160
+ "ndcg_linear@10": 0.6020969578276198,
161
+ "ndcg_exponential@1": 0.413,
162
+ "ndcg_exponential@5": 0.5692143522459587,
163
+ "ndcg_exponential@10": 0.6020969578276198,
164
+ "precision@1": 0.413,
165
+ "precision@5": 0.14059999999999997,
166
+ "precision@10": 0.0804,
167
+ "recall@1": 0.413,
168
+ "recall@5": 0.703,
169
+ "recall@10": 0.804,
170
+ "f1@1": 0.413,
171
+ "f1@5": 0.23433333333333337,
172
+ "f1@10": 0.1461818181818182,
173
+ "map@1": 0.413,
174
+ "map@5": 0.5245333333333333,
175
+ "map@10": 0.538229761904762,
176
+ "mrr@1": 0.413,
177
+ "mrr@5": 0.5245333333333333,
178
+ "mrr@10": 0.538229761904762,
179
+ "num_pred": 1000,
180
+ "num_data": 1000
181
+ },
182
+ "ImageNet-A": {
183
+ "hit@1": 0.495,
184
+ "hit@5": 0.709,
185
+ "hit@10": 0.781,
186
+ "ndcg_linear@1": 0.495,
187
+ "ndcg_linear@5": 0.6114944313148708,
188
+ "ndcg_linear@10": 0.6348665121005517,
189
+ "ndcg_exponential@1": 0.495,
190
+ "ndcg_exponential@5": 0.6114944313148708,
191
+ "ndcg_exponential@10": 0.6348665121005517,
192
+ "precision@1": 0.495,
193
+ "precision@5": 0.14179999999999998,
194
+ "precision@10": 0.07809999999999999,
195
+ "recall@1": 0.495,
196
+ "recall@5": 0.709,
197
+ "recall@10": 0.781,
198
+ "f1@1": 0.495,
199
+ "f1@5": 0.23633333333333337,
200
+ "f1@10": 0.142,
201
+ "map@1": 0.495,
202
+ "map@5": 0.5788166666666666,
203
+ "map@10": 0.5885103174603176,
204
+ "mrr@1": 0.495,
205
+ "mrr@5": 0.5788166666666666,
206
+ "mrr@10": 0.5885103174603176,
207
+ "num_pred": 1000,
208
+ "num_data": 1000
209
+ },
210
+ "ImageNet-R": {
211
+ "hit@1": 0.848,
212
+ "hit@5": 0.944,
213
+ "hit@10": 0.962,
214
+ "ndcg_linear@1": 0.848,
215
+ "ndcg_linear@5": 0.9008710351449747,
216
+ "ndcg_linear@10": 0.9068617117972817,
217
+ "ndcg_exponential@1": 0.848,
218
+ "ndcg_exponential@5": 0.9008710351449747,
219
+ "ndcg_exponential@10": 0.9068617117972817,
220
+ "precision@1": 0.848,
221
+ "precision@5": 0.18880000000000002,
222
+ "precision@10": 0.09620000000000002,
223
+ "recall@1": 0.848,
224
+ "recall@5": 0.944,
225
+ "recall@10": 0.962,
226
+ "f1@1": 0.848,
227
+ "f1@5": 0.3146666666666667,
228
+ "f1@10": 0.17490909090909088,
229
+ "map@1": 0.848,
230
+ "map@5": 0.88635,
231
+ "map@10": 0.8889230158730158,
232
+ "mrr@1": 0.848,
233
+ "mrr@5": 0.88635,
234
+ "mrr@10": 0.8889230158730158,
235
+ "num_pred": 1000,
236
+ "num_data": 1000
237
+ },
238
+ "ObjectNet": {
239
+ "hit@1": 0.713,
240
+ "hit@5": 0.863,
241
+ "hit@10": 0.906,
242
+ "ndcg_linear@1": 0.713,
243
+ "ndcg_linear@5": 0.7966827600295562,
244
+ "ndcg_linear@10": 0.8104878302902433,
245
+ "ndcg_exponential@1": 0.713,
246
+ "ndcg_exponential@5": 0.7966827600295562,
247
+ "ndcg_exponential@10": 0.8104878302902433,
248
+ "precision@1": 0.713,
249
+ "precision@5": 0.17260000000000003,
250
+ "precision@10": 0.09060000000000001,
251
+ "recall@1": 0.713,
252
+ "recall@5": 0.863,
253
+ "recall@10": 0.906,
254
+ "f1@1": 0.713,
255
+ "f1@5": 0.28766666666666674,
256
+ "f1@10": 0.16472727272727272,
257
+ "map@1": 0.713,
258
+ "map@5": 0.7742833333333333,
259
+ "map@10": 0.7799198412698413,
260
+ "mrr@1": 0.713,
261
+ "mrr@5": 0.7742833333333333,
262
+ "mrr@10": 0.7799198412698413,
263
+ "num_pred": 1000,
264
+ "num_data": 1000
265
+ },
266
+ "Country211": {
267
+ "hit@1": 0.245,
268
+ "hit@5": 0.425,
269
+ "hit@10": 0.524,
270
+ "ndcg_linear@1": 0.245,
271
+ "ndcg_linear@5": 0.34011931876670454,
272
+ "ndcg_linear@10": 0.3722522680493705,
273
+ "ndcg_exponential@1": 0.245,
274
+ "ndcg_exponential@5": 0.34011931876670454,
275
+ "ndcg_exponential@10": 0.3722522680493705,
276
+ "precision@1": 0.245,
277
+ "precision@5": 0.08499999999999999,
278
+ "precision@10": 0.052399999999999995,
279
+ "recall@1": 0.245,
280
+ "recall@5": 0.425,
281
+ "recall@10": 0.524,
282
+ "f1@1": 0.245,
283
+ "f1@5": 0.1416666666666667,
284
+ "f1@10": 0.09527272727272729,
285
+ "map@1": 0.245,
286
+ "map@5": 0.3119166666666666,
287
+ "map@10": 0.3252480158730159,
288
+ "mrr@1": 0.245,
289
+ "mrr@5": 0.3119166666666666,
290
+ "mrr@10": 0.3252480158730159,
291
+ "num_pred": 1000,
292
+ "num_data": 1000
293
+ },
294
+ "OK-VQA": {
295
+ "hit@1": 0.6,
296
+ "hit@5": 0.835,
297
+ "hit@10": 0.897,
298
+ "ndcg_linear@1": 0.6,
299
+ "ndcg_linear@5": 0.7283836592219812,
300
+ "ndcg_linear@10": 0.7483526360068199,
301
+ "ndcg_exponential@1": 0.6,
302
+ "ndcg_exponential@5": 0.7283836592219812,
303
+ "ndcg_exponential@10": 0.7483526360068199,
304
+ "precision@1": 0.6,
305
+ "precision@5": 0.167,
306
+ "precision@10": 0.08970000000000002,
307
+ "recall@1": 0.6,
308
+ "recall@5": 0.835,
309
+ "recall@10": 0.897,
310
+ "f1@1": 0.6,
311
+ "f1@5": 0.2783333333333334,
312
+ "f1@10": 0.1630909090909091,
313
+ "map@1": 0.6,
314
+ "map@5": 0.6925833333333333,
315
+ "map@10": 0.7007813492063492,
316
+ "mrr@1": 0.6,
317
+ "mrr@5": 0.6925833333333333,
318
+ "mrr@10": 0.7007813492063492,
319
+ "num_pred": 1000,
320
+ "num_data": 1000
321
+ },
322
+ "A-OKVQA": {
323
+ "hit@1": 0.532,
324
+ "hit@5": 0.833,
325
+ "hit@10": 0.897,
326
+ "ndcg_linear@1": 0.532,
327
+ "ndcg_linear@5": 0.6946159677593404,
328
+ "ndcg_linear@10": 0.7154405844345778,
329
+ "ndcg_exponential@1": 0.532,
330
+ "ndcg_exponential@5": 0.6946159677593404,
331
+ "ndcg_exponential@10": 0.7154405844345778,
332
+ "precision@1": 0.532,
333
+ "precision@5": 0.16660000000000003,
334
+ "precision@10": 0.0897,
335
+ "recall@1": 0.532,
336
+ "recall@5": 0.833,
337
+ "recall@10": 0.897,
338
+ "f1@1": 0.532,
339
+ "f1@5": 0.27766666666666673,
340
+ "f1@10": 0.1630909090909091,
341
+ "map@1": 0.532,
342
+ "map@5": 0.64835,
343
+ "map@10": 0.657015873015873,
344
+ "mrr@1": 0.532,
345
+ "mrr@5": 0.64835,
346
+ "mrr@10": 0.657015873015873,
347
+ "num_pred": 1000,
348
+ "num_data": 1000
349
+ },
350
+ "DocVQA": {
351
+ "hit@1": 0.905,
352
+ "hit@5": 0.972,
353
+ "hit@10": 0.985,
354
+ "ndcg_linear@1": 0.905,
355
+ "ndcg_linear@5": 0.9446377226583499,
356
+ "ndcg_linear@10": 0.9489408131651074,
357
+ "ndcg_exponential@1": 0.905,
358
+ "ndcg_exponential@5": 0.9446377226583499,
359
+ "ndcg_exponential@10": 0.9489408131651074,
360
+ "precision@1": 0.905,
361
+ "precision@5": 0.19440000000000004,
362
+ "precision@10": 0.09850000000000002,
363
+ "recall@1": 0.905,
364
+ "recall@5": 0.972,
365
+ "recall@10": 0.985,
366
+ "f1@1": 0.905,
367
+ "f1@5": 0.324,
368
+ "f1@10": 0.17909090909090908,
369
+ "map@1": 0.905,
370
+ "map@5": 0.9351833333333334,
371
+ "map@10": 0.9370174603174602,
372
+ "mrr@1": 0.905,
373
+ "mrr@5": 0.9351833333333334,
374
+ "mrr@10": 0.9370174603174602,
375
+ "num_pred": 1000,
376
+ "num_data": 1000
377
+ },
378
+ "InfographicsVQA": {
379
+ "hit@1": 0.593,
380
+ "hit@5": 0.787,
381
+ "hit@10": 0.849,
382
+ "ndcg_linear@1": 0.593,
383
+ "ndcg_linear@5": 0.6992302235074854,
384
+ "ndcg_linear@10": 0.719037023415279,
385
+ "ndcg_exponential@1": 0.593,
386
+ "ndcg_exponential@5": 0.6992302235074854,
387
+ "ndcg_exponential@10": 0.719037023415279,
388
+ "precision@1": 0.593,
389
+ "precision@5": 0.15739999999999998,
390
+ "precision@10": 0.0849,
391
+ "recall@1": 0.593,
392
+ "recall@5": 0.787,
393
+ "recall@10": 0.849,
394
+ "f1@1": 0.593,
395
+ "f1@5": 0.26233333333333336,
396
+ "f1@10": 0.15436363636363637,
397
+ "map@1": 0.593,
398
+ "map@5": 0.6697833333333333,
399
+ "map@10": 0.6778043650793651,
400
+ "mrr@1": 0.593,
401
+ "mrr@5": 0.6697833333333333,
402
+ "mrr@10": 0.6778043650793651,
403
+ "num_pred": 1000,
404
+ "num_data": 1000
405
+ },
406
+ "ChartQA": {
407
+ "hit@1": 0.518,
408
+ "hit@5": 0.685,
409
+ "hit@10": 0.729,
410
+ "ndcg_linear@1": 0.518,
411
+ "ndcg_linear@5": 0.6092852996177144,
412
+ "ndcg_linear@10": 0.6237752364760738,
413
+ "ndcg_exponential@1": 0.518,
414
+ "ndcg_exponential@5": 0.6092852996177144,
415
+ "ndcg_exponential@10": 0.6237752364760738,
416
+ "precision@1": 0.518,
417
+ "precision@5": 0.137,
418
+ "precision@10": 0.0729,
419
+ "recall@1": 0.518,
420
+ "recall@5": 0.685,
421
+ "recall@10": 0.729,
422
+ "f1@1": 0.518,
423
+ "f1@5": 0.22833333333333336,
424
+ "f1@10": 0.13254545454545455,
425
+ "map@1": 0.518,
426
+ "map@5": 0.5838833333333333,
427
+ "map@10": 0.5900174603174603,
428
+ "mrr@1": 0.518,
429
+ "mrr@5": 0.5838833333333333,
430
+ "mrr@10": 0.5900174603174603,
431
+ "num_pred": 1000,
432
+ "num_data": 1000
433
+ },
434
+ "Visual7W": {
435
+ "hit@1": 0.571,
436
+ "hit@5": 0.889,
437
+ "hit@10": 0.941,
438
+ "ndcg_linear@1": 0.571,
439
+ "ndcg_linear@5": 0.743669883740337,
440
+ "ndcg_linear@10": 0.7608831436573412,
441
+ "ndcg_exponential@1": 0.571,
442
+ "ndcg_exponential@5": 0.743669883740337,
443
+ "ndcg_exponential@10": 0.7608831436573412,
444
+ "precision@1": 0.571,
445
+ "precision@5": 0.1778,
446
+ "precision@10": 0.0941,
447
+ "recall@1": 0.571,
448
+ "recall@5": 0.889,
449
+ "recall@10": 0.941,
450
+ "f1@1": 0.571,
451
+ "f1@5": 0.29633333333333334,
452
+ "f1@10": 0.1710909090909091,
453
+ "map@1": 0.571,
454
+ "map@5": 0.6949666666666667,
455
+ "map@10": 0.7023039682539681,
456
+ "mrr@1": 0.571,
457
+ "mrr@5": 0.6949666666666667,
458
+ "mrr@10": 0.7023039682539681,
459
+ "num_pred": 1000,
460
+ "num_data": 1000
461
+ },
462
+ "ScienceQA": {
463
+ "hit@1": 0.475,
464
+ "hit@5": 0.765,
465
+ "hit@10": 0.864,
466
+ "ndcg_linear@1": 0.475,
467
+ "ndcg_linear@5": 0.6314453432319744,
468
+ "ndcg_linear@10": 0.6630621657284694,
469
+ "ndcg_exponential@1": 0.475,
470
+ "ndcg_exponential@5": 0.6314453432319744,
471
+ "ndcg_exponential@10": 0.6630621657284694,
472
+ "precision@1": 0.475,
473
+ "precision@5": 0.153,
474
+ "precision@10": 0.0864,
475
+ "recall@1": 0.475,
476
+ "recall@5": 0.765,
477
+ "recall@10": 0.864,
478
+ "f1@1": 0.475,
479
+ "f1@5": 0.25500000000000006,
480
+ "f1@10": 0.1570909090909091,
481
+ "map@1": 0.475,
482
+ "map@5": 0.5868666666666666,
483
+ "map@10": 0.5996865079365079,
484
+ "mrr@1": 0.475,
485
+ "mrr@5": 0.5868666666666666,
486
+ "mrr@10": 0.5996865079365079,
487
+ "num_pred": 1000,
488
+ "num_data": 1000
489
+ },
490
+ "VizWiz": {
491
+ "hit@1": 0.534,
492
+ "hit@5": 0.718,
493
+ "hit@10": 0.764,
494
+ "ndcg_linear@1": 0.534,
495
+ "ndcg_linear@5": 0.6357776354115615,
496
+ "ndcg_linear@10": 0.6503798129042785,
497
+ "ndcg_exponential@1": 0.534,
498
+ "ndcg_exponential@5": 0.6357776354115615,
499
+ "ndcg_exponential@10": 0.6503798129042785,
500
+ "precision@1": 0.534,
501
+ "precision@5": 0.1436,
502
+ "precision@10": 0.07640000000000001,
503
+ "recall@1": 0.534,
504
+ "recall@5": 0.718,
505
+ "recall@10": 0.764,
506
+ "f1@1": 0.534,
507
+ "f1@5": 0.23933333333333337,
508
+ "f1@10": 0.13890909090909093,
509
+ "map@1": 0.534,
510
+ "map@5": 0.6081,
511
+ "map@10": 0.613972619047619,
512
+ "mrr@1": 0.534,
513
+ "mrr@5": 0.6081,
514
+ "mrr@10": 0.613972619047619,
515
+ "num_pred": 1000,
516
+ "num_data": 1000
517
+ },
518
+ "GQA": {
519
+ "hit@1": 0.604,
520
+ "hit@5": 0.826,
521
+ "hit@10": 0.884,
522
+ "ndcg_linear@1": 0.604,
523
+ "ndcg_linear@5": 0.7250962738992552,
524
+ "ndcg_linear@10": 0.7439540651545449,
525
+ "ndcg_exponential@1": 0.604,
526
+ "ndcg_exponential@5": 0.7250962738992552,
527
+ "ndcg_exponential@10": 0.7439540651545449,
528
+ "precision@1": 0.604,
529
+ "precision@5": 0.1652,
530
+ "precision@10": 0.0884,
531
+ "recall@1": 0.604,
532
+ "recall@5": 0.826,
533
+ "recall@10": 0.884,
534
+ "f1@1": 0.604,
535
+ "f1@5": 0.2753333333333334,
536
+ "f1@10": 0.16072727272727272,
537
+ "map@1": 0.604,
538
+ "map@5": 0.6912,
539
+ "map@10": 0.6990444444444445,
540
+ "mrr@1": 0.604,
541
+ "mrr@5": 0.6912,
542
+ "mrr@10": 0.6990444444444445,
543
+ "num_pred": 1000,
544
+ "num_data": 1000
545
+ },
546
+ "TextVQA": {
547
+ "hit@1": 0.838,
548
+ "hit@5": 0.9,
549
+ "hit@10": 0.912,
550
+ "ndcg_linear@1": 0.838,
551
+ "ndcg_linear@5": 0.8735356970943653,
552
+ "ndcg_linear@10": 0.877449220591428,
553
+ "ndcg_exponential@1": 0.838,
554
+ "ndcg_exponential@5": 0.8735356970943653,
555
+ "ndcg_exponential@10": 0.877449220591428,
556
+ "precision@1": 0.838,
557
+ "precision@5": 0.18,
558
+ "precision@10": 0.09120000000000002,
559
+ "recall@1": 0.838,
560
+ "recall@5": 0.9,
561
+ "recall@10": 0.912,
562
+ "f1@1": 0.838,
563
+ "f1@5": 0.3,
564
+ "f1@10": 0.1658181818181818,
565
+ "map@1": 0.838,
566
+ "map@5": 0.8645166666666666,
567
+ "map@10": 0.8661539682539683,
568
+ "mrr@1": 0.838,
569
+ "mrr@5": 0.8645166666666666,
570
+ "mrr@10": 0.8661539682539683,
571
+ "num_pred": 1000,
572
+ "num_data": 1000
573
+ },
574
+ "VisDial": {
575
+ "hit@1": 0.791,
576
+ "hit@5": 0.945,
577
+ "hit@10": 0.97,
578
+ "ndcg_linear@1": 0.791,
579
+ "ndcg_linear@5": 0.8757768586840254,
580
+ "ndcg_linear@10": 0.8839019772727467,
581
+ "ndcg_exponential@1": 0.791,
582
+ "ndcg_exponential@5": 0.8757768586840254,
583
+ "ndcg_exponential@10": 0.8839019772727467,
584
+ "precision@1": 0.791,
585
+ "precision@5": 0.189,
586
+ "precision@10": 0.09700000000000002,
587
+ "recall@1": 0.791,
588
+ "recall@5": 0.945,
589
+ "recall@10": 0.97,
590
+ "f1@1": 0.791,
591
+ "f1@5": 0.315,
592
+ "f1@10": 0.17636363636363636,
593
+ "map@1": 0.791,
594
+ "map@5": 0.8524666666666667,
595
+ "map@10": 0.8558428571428571,
596
+ "mrr@1": 0.791,
597
+ "mrr@5": 0.8524666666666667,
598
+ "mrr@10": 0.8558428571428571,
599
+ "num_pred": 1000,
600
+ "num_data": 1000
601
+ },
602
+ "CIRR": {
603
+ "hit@1": 0.627,
604
+ "hit@5": 0.904,
605
+ "hit@10": 0.955,
606
+ "ndcg_linear@1": 0.627,
607
+ "ndcg_linear@5": 0.7814520061894136,
608
+ "ndcg_linear@10": 0.7982662696128282,
609
+ "ndcg_exponential@1": 0.627,
610
+ "ndcg_exponential@5": 0.7814520061894136,
611
+ "ndcg_exponential@10": 0.7982662696128282,
612
+ "precision@1": 0.627,
613
+ "precision@5": 0.18080000000000002,
614
+ "precision@10": 0.09550000000000002,
615
+ "recall@1": 0.627,
616
+ "recall@5": 0.904,
617
+ "recall@10": 0.955,
618
+ "f1@1": 0.627,
619
+ "f1@5": 0.3013333333333334,
620
+ "f1@10": 0.17363636363636362,
621
+ "map@1": 0.627,
622
+ "map@5": 0.7399833333333333,
623
+ "map@10": 0.747115873015873,
624
+ "mrr@1": 0.627,
625
+ "mrr@5": 0.7399833333333333,
626
+ "mrr@10": 0.747115873015873,
627
+ "num_pred": 1000,
628
+ "num_data": 1000
629
+ },
630
+ "VisualNews_t2i": {
631
+ "hit@1": 0.78,
632
+ "hit@5": 0.908,
633
+ "hit@10": 0.931,
634
+ "ndcg_linear@1": 0.78,
635
+ "ndcg_linear@5": 0.8500641649581272,
636
+ "ndcg_linear@10": 0.8576663310856566,
637
+ "ndcg_exponential@1": 0.78,
638
+ "ndcg_exponential@5": 0.8500641649581272,
639
+ "ndcg_exponential@10": 0.8576663310856566,
640
+ "precision@1": 0.78,
641
+ "precision@5": 0.1816,
642
+ "precision@10": 0.09309999999999999,
643
+ "recall@1": 0.78,
644
+ "recall@5": 0.908,
645
+ "recall@10": 0.931,
646
+ "f1@1": 0.78,
647
+ "f1@5": 0.3026666666666667,
648
+ "f1@10": 0.16927272727272724,
649
+ "map@1": 0.78,
650
+ "map@5": 0.8306166666666668,
651
+ "map@10": 0.8338484126984126,
652
+ "mrr@1": 0.78,
653
+ "mrr@5": 0.8306166666666668,
654
+ "mrr@10": 0.8338484126984126,
655
+ "num_pred": 1000,
656
+ "num_data": 1000
657
+ },
658
+ "VisualNews_i2t": {
659
+ "hit@1": 0.814,
660
+ "hit@5": 0.921,
661
+ "hit@10": 0.939,
662
+ "ndcg_linear@1": 0.814,
663
+ "ndcg_linear@5": 0.8728615373378912,
664
+ "ndcg_linear@10": 0.8787080770691066,
665
+ "ndcg_exponential@1": 0.814,
666
+ "ndcg_exponential@5": 0.8728615373378912,
667
+ "ndcg_exponential@10": 0.8787080770691066,
668
+ "precision@1": 0.814,
669
+ "precision@5": 0.18420000000000006,
670
+ "precision@10": 0.09390000000000001,
671
+ "recall@1": 0.814,
672
+ "recall@5": 0.921,
673
+ "recall@10": 0.939,
674
+ "f1@1": 0.814,
675
+ "f1@5": 0.307,
676
+ "f1@10": 0.17072727272727273,
677
+ "map@1": 0.814,
678
+ "map@5": 0.8566500000000001,
679
+ "map@10": 0.8590753968253969,
680
+ "mrr@1": 0.814,
681
+ "mrr@5": 0.8566500000000001,
682
+ "mrr@10": 0.8590753968253969,
683
+ "num_pred": 1000,
684
+ "num_data": 1000
685
+ },
686
+ "MSCOCO_t2i": {
687
+ "hit@1": 0.767,
688
+ "hit@5": 0.947,
689
+ "hit@10": 0.981,
690
+ "ndcg_linear@1": 0.767,
691
+ "ndcg_linear@5": 0.8674275875789338,
692
+ "ndcg_linear@10": 0.8785792123543245,
693
+ "ndcg_exponential@1": 0.767,
694
+ "ndcg_exponential@5": 0.8674275875789338,
695
+ "ndcg_exponential@10": 0.8785792123543245,
696
+ "precision@1": 0.767,
697
+ "precision@5": 0.18940000000000004,
698
+ "precision@10": 0.09810000000000002,
699
+ "recall@1": 0.767,
700
+ "recall@5": 0.947,
701
+ "recall@10": 0.981,
702
+ "f1@1": 0.767,
703
+ "f1@5": 0.3156666666666667,
704
+ "f1@10": 0.17836363636363634,
705
+ "map@1": 0.767,
706
+ "map@5": 0.8405333333333334,
707
+ "map@10": 0.8452297619047618,
708
+ "mrr@1": 0.767,
709
+ "mrr@5": 0.8405333333333334,
710
+ "mrr@10": 0.8452297619047618,
711
+ "num_pred": 1000,
712
+ "num_data": 1000
713
+ },
714
+ "MSCOCO_i2t": {
715
+ "hit@1": 0.736,
716
+ "hit@5": 0.931,
717
+ "hit@10": 0.977,
718
+ "ndcg_linear@1": 0.736,
719
+ "ndcg_linear@5": 0.846634371650986,
720
+ "ndcg_linear@10": 0.8617493616233067,
721
+ "ndcg_exponential@1": 0.736,
722
+ "ndcg_exponential@5": 0.846634371650986,
723
+ "ndcg_exponential@10": 0.8617493616233067,
724
+ "precision@1": 0.736,
725
+ "precision@5": 0.1862,
726
+ "precision@10": 0.09770000000000002,
727
+ "recall@1": 0.736,
728
+ "recall@5": 0.931,
729
+ "recall@10": 0.977,
730
+ "f1@1": 0.736,
731
+ "f1@5": 0.31033333333333335,
732
+ "f1@10": 0.17763636363636362,
733
+ "map@1": 0.736,
734
+ "map@5": 0.81795,
735
+ "map@10": 0.8243242063492062,
736
+ "mrr@1": 0.736,
737
+ "mrr@5": 0.81795,
738
+ "mrr@10": 0.8243242063492062,
739
+ "num_pred": 1000,
740
+ "num_data": 1000
741
+ },
742
+ "NIGHTS": {
743
+ "hit@1": 0.679,
744
+ "hit@5": 0.977,
745
+ "hit@10": 0.993,
746
+ "ndcg_linear@1": 0.679,
747
+ "ndcg_linear@5": 0.851460039622325,
748
+ "ndcg_linear@10": 0.8568819234846234,
749
+ "ndcg_exponential@1": 0.679,
750
+ "ndcg_exponential@5": 0.851460039622325,
751
+ "ndcg_exponential@10": 0.8568819234846234,
752
+ "precision@1": 0.679,
753
+ "precision@5": 0.19540000000000005,
754
+ "precision@10": 0.09930000000000001,
755
+ "recall@1": 0.679,
756
+ "recall@5": 0.977,
757
+ "recall@10": 0.993,
758
+ "f1@1": 0.679,
759
+ "f1@5": 0.3256666666666666,
760
+ "f1@10": 0.1805454545454545,
761
+ "map@1": 0.679,
762
+ "map@5": 0.8084166666666668,
763
+ "map@10": 0.8108003968253967,
764
+ "mrr@1": 0.679,
765
+ "mrr@5": 0.8084166666666668,
766
+ "mrr@10": 0.8108003968253967,
767
+ "num_pred": 1000,
768
+ "num_data": 1000
769
+ },
770
+ "WebQA": {
771
+ "hit@1": 0.91,
772
+ "hit@5": 0.987,
773
+ "hit@10": 0.991,
774
+ "ndcg_linear@1": 0.91,
775
+ "ndcg_linear@5": 0.9551382902817904,
776
+ "ndcg_linear@10": 0.9564731028156578,
777
+ "ndcg_exponential@1": 0.91,
778
+ "ndcg_exponential@5": 0.9551382902817904,
779
+ "ndcg_exponential@10": 0.9564731028156578,
780
+ "precision@1": 0.91,
781
+ "precision@5": 0.19740000000000005,
782
+ "precision@10": 0.09910000000000002,
783
+ "recall@1": 0.91,
784
+ "recall@5": 0.987,
785
+ "recall@10": 0.991,
786
+ "f1@1": 0.91,
787
+ "f1@5": 0.329,
788
+ "f1@10": 0.1801818181818182,
789
+ "map@1": 0.91,
790
+ "map@5": 0.9441833333333334,
791
+ "map@10": 0.9447595238095238,
792
+ "mrr@1": 0.91,
793
+ "mrr@5": 0.9441833333333334,
794
+ "mrr@10": 0.9447595238095238,
795
+ "num_pred": 1000,
796
+ "num_data": 1000
797
+ },
798
+ "FashionIQ": {
799
+ "hit@1": 0.338,
800
+ "hit@5": 0.598,
801
+ "hit@10": 0.707,
802
+ "ndcg_linear@1": 0.338,
803
+ "ndcg_linear@5": 0.4737408022870451,
804
+ "ndcg_linear@10": 0.509405039694376,
805
+ "ndcg_exponential@1": 0.338,
806
+ "ndcg_exponential@5": 0.4737408022870451,
807
+ "ndcg_exponential@10": 0.509405039694376,
808
+ "precision@1": 0.338,
809
+ "precision@5": 0.1196,
810
+ "precision@10": 0.07069999999999999,
811
+ "recall@1": 0.338,
812
+ "recall@5": 0.598,
813
+ "recall@10": 0.707,
814
+ "f1@1": 0.338,
815
+ "f1@5": 0.19933333333333336,
816
+ "f1@10": 0.12854545454545457,
817
+ "map@1": 0.338,
818
+ "map@5": 0.43256666666666665,
819
+ "map@10": 0.4475174603174603,
820
+ "mrr@1": 0.338,
821
+ "mrr@5": 0.43256666666666665,
822
+ "mrr@10": 0.4475174603174603,
823
+ "num_pred": 1000,
824
+ "num_data": 1000
825
+ },
826
+ "Wiki-SS-NQ": {
827
+ "hit@1": 0.749,
828
+ "hit@5": 0.93,
829
+ "hit@10": 0.954,
830
+ "ndcg_linear@1": 0.749,
831
+ "ndcg_linear@5": 0.8518286622685086,
832
+ "ndcg_linear@10": 0.8598218746062669,
833
+ "ndcg_exponential@1": 0.749,
834
+ "ndcg_exponential@5": 0.8518286622685086,
835
+ "ndcg_exponential@10": 0.8598218746062669,
836
+ "precision@1": 0.749,
837
+ "precision@5": 0.18600000000000003,
838
+ "precision@10": 0.09540000000000003,
839
+ "recall@1": 0.749,
840
+ "recall@5": 0.93,
841
+ "recall@10": 0.954,
842
+ "f1@1": 0.749,
843
+ "f1@5": 0.31,
844
+ "f1@10": 0.17345454545454544,
845
+ "map@1": 0.749,
846
+ "map@5": 0.8252166666666667,
847
+ "map@10": 0.8286500000000001,
848
+ "mrr@1": 0.749,
849
+ "mrr@5": 0.8252166666666667,
850
+ "mrr@10": 0.8286500000000001,
851
+ "num_pred": 1000,
852
+ "num_data": 1000
853
+ },
854
+ "OVEN": {
855
+ "hit@1": 0.746,
856
+ "hit@5": 0.925,
857
+ "hit@10": 0.958,
858
+ "ndcg_linear@1": 0.746,
859
+ "ndcg_linear@5": 0.844413209516627,
860
+ "ndcg_linear@10": 0.8552665639399527,
861
+ "ndcg_exponential@1": 0.746,
862
+ "ndcg_exponential@5": 0.844413209516627,
863
+ "ndcg_exponential@10": 0.8552665639399527,
864
+ "precision@1": 0.746,
865
+ "precision@5": 0.18500000000000003,
866
+ "precision@10": 0.09580000000000001,
867
+ "recall@1": 0.746,
868
+ "recall@5": 0.925,
869
+ "recall@10": 0.958,
870
+ "f1@1": 0.746,
871
+ "f1@5": 0.30833333333333335,
872
+ "f1@10": 0.1741818181818182,
873
+ "map@1": 0.746,
874
+ "map@5": 0.8172999999999999,
875
+ "map@10": 0.8218880952380953,
876
+ "mrr@1": 0.746,
877
+ "mrr@5": 0.8172999999999999,
878
+ "mrr@10": 0.8218880952380953,
879
+ "num_pred": 1000,
880
+ "num_data": 1000
881
+ },
882
+ "EDIS": {
883
+ "hit@1": 0.92,
884
+ "hit@5": 0.987,
885
+ "hit@10": 0.993,
886
+ "ndcg_linear@1": 0.92,
887
+ "ndcg_linear@5": 0.9581026722360495,
888
+ "ndcg_linear@10": 0.960027672213166,
889
+ "ndcg_exponential@1": 0.92,
890
+ "ndcg_exponential@5": 0.9581026722360495,
891
+ "ndcg_exponential@10": 0.960027672213166,
892
+ "precision@1": 0.92,
893
+ "precision@5": 0.19740000000000005,
894
+ "precision@10": 0.09930000000000001,
895
+ "recall@1": 0.92,
896
+ "recall@5": 0.987,
897
+ "recall@10": 0.993,
898
+ "f1@1": 0.92,
899
+ "f1@5": 0.329,
900
+ "f1@10": 0.18054545454545454,
901
+ "map@1": 0.92,
902
+ "map@5": 0.9482333333333334,
903
+ "map@10": 0.9490178571428571,
904
+ "mrr@1": 0.92,
905
+ "mrr@5": 0.9482333333333334,
906
+ "mrr@10": 0.9490178571428571,
907
+ "num_pred": 1000,
908
+ "num_data": 1000
909
+ },
910
+ "MSCOCO": {
911
+ "hit@1": 0.685,
912
+ "hit@5": 0.828,
913
+ "hit@10": 0.876,
914
+ "ndcg_linear@1": 0.685,
915
+ "ndcg_linear@5": 0.7659909484428852,
916
+ "ndcg_linear@10": 0.7811887013291692,
917
+ "ndcg_exponential@1": 0.685,
918
+ "ndcg_exponential@5": 0.7659909484428852,
919
+ "ndcg_exponential@10": 0.7811887013291692,
920
+ "precision@1": 0.685,
921
+ "precision@5": 0.1656,
922
+ "precision@10": 0.08760000000000003,
923
+ "recall@1": 0.685,
924
+ "recall@5": 0.828,
925
+ "recall@10": 0.876,
926
+ "f1@1": 0.685,
927
+ "f1@5": 0.276,
928
+ "f1@10": 0.1592727272727273,
929
+ "map@1": 0.685,
930
+ "map@5": 0.7449666666666667,
931
+ "map@10": 0.7510507936507937,
932
+ "mrr@1": 0.685,
933
+ "mrr@5": 0.7449666666666667,
934
+ "mrr@10": 0.7510507936507937,
935
+ "num_pred": 1000,
936
+ "num_data": 1000
937
+ },
938
+ "RefCOCO": {
939
+ "hit@1": 0.86,
940
+ "hit@5": 0.983,
941
+ "hit@10": 0.99,
942
+ "ndcg_linear@1": 0.86,
943
+ "ndcg_linear@5": 0.9311419574836521,
944
+ "ndcg_linear@10": 0.933415689538874,
945
+ "ndcg_exponential@1": 0.86,
946
+ "ndcg_exponential@5": 0.9311419574836521,
947
+ "ndcg_exponential@10": 0.933415689538874,
948
+ "precision@1": 0.86,
949
+ "precision@5": 0.19660000000000002,
950
+ "precision@10": 0.09900000000000002,
951
+ "recall@1": 0.86,
952
+ "recall@5": 0.983,
953
+ "recall@10": 0.99,
954
+ "f1@1": 0.86,
955
+ "f1@5": 0.32766666666666666,
956
+ "f1@10": 0.18,
957
+ "map@1": 0.86,
958
+ "map@5": 0.9133499999999999,
959
+ "map@10": 0.9142924603174603,
960
+ "mrr@1": 0.86,
961
+ "mrr@5": 0.9133499999999999,
962
+ "mrr@10": 0.9142924603174603,
963
+ "num_pred": 1000,
964
+ "num_data": 1000
965
+ },
966
+ "RefCOCO-Matching": {
967
+ "hit@1": 0.748,
968
+ "hit@5": 0.999,
969
+ "hit@10": 1.0,
970
+ "ndcg_linear@1": 0.748,
971
+ "ndcg_linear@5": 0.9073319320018682,
972
+ "ndcg_linear@10": 0.9076652653352016,
973
+ "ndcg_exponential@1": 0.748,
974
+ "ndcg_exponential@5": 0.9073319320018682,
975
+ "ndcg_exponential@10": 0.9076652653352016,
976
+ "precision@1": 0.748,
977
+ "precision@5": 0.19980000000000003,
978
+ "precision@10": 0.10000000000000002,
979
+ "recall@1": 0.748,
980
+ "recall@5": 0.999,
981
+ "recall@10": 1.0,
982
+ "f1@1": 0.748,
983
+ "f1@5": 0.333,
984
+ "f1@10": 0.18181818181818182,
985
+ "map@1": 0.748,
986
+ "map@5": 0.8753333333333333,
987
+ "map@10": 0.8754761904761905,
988
+ "mrr@1": 0.748,
989
+ "mrr@5": 0.8723333333333333,
990
+ "mrr@10": 0.8724761904761905,
991
+ "num_pred": 1000,
992
+ "num_data": 1000
993
+ },
994
+ "Visual7W-Pointing": {
995
+ "hit@1": 0.821,
996
+ "hit@5": 0.94,
997
+ "hit@10": 0.966,
998
+ "ndcg_linear@1": 0.821,
999
+ "ndcg_linear@5": 0.8866903981112136,
1000
+ "ndcg_linear@10": 0.8951395131706409,
1001
+ "ndcg_exponential@1": 0.821,
1002
+ "ndcg_exponential@5": 0.8866903981112136,
1003
+ "ndcg_exponential@10": 0.8951395131706409,
1004
+ "precision@1": 0.821,
1005
+ "precision@5": 0.18800000000000003,
1006
+ "precision@10": 0.0966,
1007
+ "recall@1": 0.821,
1008
+ "recall@5": 0.94,
1009
+ "recall@10": 0.966,
1010
+ "f1@1": 0.821,
1011
+ "f1@5": 0.31333333333333335,
1012
+ "f1@10": 0.17563636363636362,
1013
+ "map@1": 0.821,
1014
+ "map@5": 0.8687166666666667,
1015
+ "map@10": 0.872225,
1016
+ "mrr@1": 0.821,
1017
+ "mrr@5": 0.8687166666666667,
1018
+ "mrr@10": 0.872225,
1019
+ "num_pred": 1000,
1020
+ "num_data": 1000
1021
+ }
1022
+ },
1023
+ "visdoc": {
1024
+ "ViDoRe_arxivqa": {
1025
+ "hit@1": 0.778,
1026
+ "hit@5": 0.882,
1027
+ "hit@10": 0.912,
1028
+ "ndcg_linear@1": 0.778,
1029
+ "ndcg_linear@5": 0.8339775475302107,
1030
+ "ndcg_linear@10": 0.8437539073932351,
1031
+ "ndcg_exponential@1": 0.778,
1032
+ "ndcg_exponential@5": 0.8339775475302107,
1033
+ "ndcg_exponential@10": 0.8437539073932351,
1034
+ "precision@1": 0.778,
1035
+ "precision@5": 0.1764,
1036
+ "precision@10": 0.09119999999999999,
1037
+ "recall@1": 0.778,
1038
+ "recall@5": 0.882,
1039
+ "recall@10": 0.912,
1040
+ "f1@1": 0.778,
1041
+ "f1@5": 0.29400000000000004,
1042
+ "f1@10": 0.1658181818181818,
1043
+ "map@1": 0.778,
1044
+ "map@5": 0.8179333333333334,
1045
+ "map@10": 0.8220119047619048,
1046
+ "mrr@1": 0.778,
1047
+ "mrr@5": 0.8179333333333334,
1048
+ "mrr@10": 0.8220119047619048,
1049
+ "num_pred": 500,
1050
+ "num_data": 500
1051
+ },
1052
+ "ViDoRe_docvqa": {
1053
+ "hit@1": 0.4656319290465632,
1054
+ "hit@5": 0.6496674057649667,
1055
+ "hit@10": 0.7006651884700665,
1056
+ "ndcg_linear@1": 0.4656319290465632,
1057
+ "ndcg_linear@5": 0.5608930231638922,
1058
+ "ndcg_linear@10": 0.5774070949574647,
1059
+ "ndcg_exponential@1": 0.4656319290465632,
1060
+ "ndcg_exponential@5": 0.5608930231638922,
1061
+ "ndcg_exponential@10": 0.5774070949574647,
1062
+ "precision@1": 0.4656319290465632,
1063
+ "precision@5": 0.13259423503325943,
1064
+ "precision@10": 0.07161862527716185,
1065
+ "recall@1": 0.4587952697708796,
1066
+ "recall@5": 0.6418699186991871,
1067
+ "recall@10": 0.6939763488543976,
1068
+ "f1@1": 0.46042981408835065,
1069
+ "f1@5": 0.21831069456607483,
1070
+ "f1@10": 0.12924348378893835,
1071
+ "map@1": 0.4656319290465632,
1072
+ "map@5": 0.5316260162601626,
1073
+ "map@10": 0.5383579699433357,
1074
+ "mrr@1": 0.4656319290465632,
1075
+ "mrr@5": 0.5375461936437546,
1076
+ "mrr@10": 0.5441831204026326,
1077
+ "num_pred": 451,
1078
+ "num_data": 451
1079
+ },
1080
+ "ViDoRe_infovqa": {
1081
+ "hit@1": 0.8481781376518218,
1082
+ "hit@5": 0.9392712550607287,
1083
+ "hit@10": 0.9554655870445344,
1084
+ "ndcg_linear@1": 0.8481781376518218,
1085
+ "ndcg_linear@5": 0.8989267822453011,
1086
+ "ndcg_linear@10": 0.9044014059110002,
1087
+ "ndcg_exponential@1": 0.8481781376518218,
1088
+ "ndcg_exponential@5": 0.8989267822453011,
1089
+ "ndcg_exponential@10": 0.9044014059110002,
1090
+ "precision@1": 0.8481781376518218,
1091
+ "precision@5": 0.1882591093117409,
1092
+ "precision@10": 0.09574898785425101,
1093
+ "recall@1": 0.847165991902834,
1094
+ "recall@5": 0.9392712550607287,
1095
+ "recall@10": 0.9554655870445344,
1096
+ "f1@1": 0.8475033738191633,
1097
+ "f1@5": 0.313572392519761,
1098
+ "f1@10": 0.1740277266593056,
1099
+ "map@1": 0.8481781376518218,
1100
+ "map@5": 0.8850877192982456,
1101
+ "map@10": 0.8874911638069533,
1102
+ "mrr@1": 0.8481781376518218,
1103
+ "mrr@5": 0.8855937921727396,
1104
+ "mrr@10": 0.8879972366814471,
1105
+ "num_pred": 494,
1106
+ "num_data": 494
1107
+ },
1108
+ "ViDoRe_tabfquad": {
1109
+ "hit@1": 0.9035714285714286,
1110
+ "hit@5": 0.9714285714285714,
1111
+ "hit@10": 0.9821428571428571,
1112
+ "ndcg_linear@1": 0.9035714285714286,
1113
+ "ndcg_linear@5": 0.9438624131031421,
1114
+ "ndcg_linear@10": 0.9472757398638011,
1115
+ "ndcg_exponential@1": 0.9035714285714286,
1116
+ "ndcg_exponential@5": 0.9438624131031421,
1117
+ "ndcg_exponential@10": 0.9472757398638011,
1118
+ "precision@1": 0.9035714285714286,
1119
+ "precision@5": 0.19428571428571426,
1120
+ "precision@10": 0.0982142857142857,
1121
+ "recall@1": 0.9035714285714286,
1122
+ "recall@5": 0.9714285714285714,
1123
+ "recall@10": 0.9821428571428571,
1124
+ "f1@1": 0.9035714285714286,
1125
+ "f1@5": 0.3238095238095239,
1126
+ "f1@10": 0.1785714285714286,
1127
+ "map@1": 0.9035714285714286,
1128
+ "map@5": 0.9343452380952382,
1129
+ "map@10": 0.9357227891156463,
1130
+ "mrr@1": 0.9035714285714286,
1131
+ "mrr@5": 0.9343452380952382,
1132
+ "mrr@10": 0.9357227891156463,
1133
+ "num_pred": 280,
1134
+ "num_data": 280
1135
+ },
1136
+ "ViDoRe_tatdqa": {
1137
+ "hit@1": 0.5595382746051033,
1138
+ "hit@5": 0.8061968408262454,
1139
+ "hit@10": 0.8882138517618469,
1140
+ "ndcg_linear@1": 0.5595382746051033,
1141
+ "ndcg_linear@5": 0.6931574300211526,
1142
+ "ndcg_linear@10": 0.7197622132929462,
1143
+ "ndcg_exponential@1": 0.5595382746051033,
1144
+ "ndcg_exponential@5": 0.6931574300211526,
1145
+ "ndcg_exponential@10": 0.7197622132929462,
1146
+ "precision@1": 0.5595382746051033,
1147
+ "precision@5": 0.1617253948967193,
1148
+ "precision@10": 0.08906439854191979,
1149
+ "recall@1": 0.5583232077764277,
1150
+ "recall@5": 0.8058930741190765,
1151
+ "recall@10": 0.887059538274605,
1152
+ "f1@1": 0.5587282300526528,
1153
+ "f1@5": 0.269281953364578,
1154
+ "f1@10": 0.16179351227953906,
1155
+ "map@1": 0.5595382746051033,
1156
+ "map@5": 0.6552450384771162,
1157
+ "map@10": 0.6663856824625355,
1158
+ "mrr@1": 0.5595382746051033,
1159
+ "mrr@5": 0.6555994329688134,
1160
+ "mrr@10": 0.666856159231615,
1161
+ "num_pred": 1646,
1162
+ "num_data": 1646
1163
+ },
1164
+ "ViDoRe_shiftproject": {
1165
+ "hit@1": 0.82,
1166
+ "hit@5": 0.97,
1167
+ "hit@10": 0.97,
1168
+ "ndcg_linear@1": 0.82,
1169
+ "ndcg_linear@5": 0.8987681350120575,
1170
+ "ndcg_linear@10": 0.8987681350120575,
1171
+ "ndcg_exponential@1": 0.82,
1172
+ "ndcg_exponential@5": 0.8987681350120575,
1173
+ "ndcg_exponential@10": 0.8987681350120575,
1174
+ "precision@1": 0.82,
1175
+ "precision@5": 0.19399999999999995,
1176
+ "precision@10": 0.09699999999999998,
1177
+ "recall@1": 0.82,
1178
+ "recall@5": 0.97,
1179
+ "recall@10": 0.97,
1180
+ "f1@1": 0.82,
1181
+ "f1@5": 0.3233333333333334,
1182
+ "f1@10": 0.17636363636363647,
1183
+ "map@1": 0.82,
1184
+ "map@5": 0.8751666666666666,
1185
+ "map@10": 0.8751666666666666,
1186
+ "mrr@1": 0.82,
1187
+ "mrr@5": 0.8751666666666666,
1188
+ "mrr@10": 0.8751666666666666,
1189
+ "num_pred": 100,
1190
+ "num_data": 100
1191
+ },
1192
+ "ViDoRe_syntheticDocQA_artificial_intelligence": {
1193
+ "hit@1": 0.97,
1194
+ "hit@5": 1.0,
1195
+ "hit@10": 1.0,
1196
+ "ndcg_linear@1": 0.97,
1197
+ "ndcg_linear@5": 0.9876185950714291,
1198
+ "ndcg_linear@10": 0.9876185950714291,
1199
+ "ndcg_exponential@1": 0.97,
1200
+ "ndcg_exponential@5": 0.9876185950714291,
1201
+ "ndcg_exponential@10": 0.9876185950714291,
1202
+ "precision@1": 0.97,
1203
+ "precision@5": 0.19999999999999996,
1204
+ "precision@10": 0.09999999999999998,
1205
+ "recall@1": 0.97,
1206
+ "recall@5": 1.0,
1207
+ "recall@10": 1.0,
1208
+ "f1@1": 0.97,
1209
+ "f1@5": 0.3333333333333335,
1210
+ "f1@10": 0.1818181818181819,
1211
+ "map@1": 0.97,
1212
+ "map@5": 0.9833333333333333,
1213
+ "map@10": 0.9833333333333333,
1214
+ "mrr@1": 0.97,
1215
+ "mrr@5": 0.9833333333333333,
1216
+ "mrr@10": 0.9833333333333333,
1217
+ "num_pred": 100,
1218
+ "num_data": 100
1219
+ },
1220
+ "ViDoRe_syntheticDocQA_energy": {
1221
+ "hit@1": 0.89,
1222
+ "hit@5": 0.96,
1223
+ "hit@10": 0.97,
1224
+ "ndcg_linear@1": 0.89,
1225
+ "ndcg_linear@5": 0.9277964206794891,
1226
+ "ndcg_linear@10": 0.9309510694473465,
1227
+ "ndcg_exponential@1": 0.89,
1228
+ "ndcg_exponential@5": 0.9277964206794891,
1229
+ "ndcg_exponential@10": 0.9309510694473465,
1230
+ "precision@1": 0.89,
1231
+ "precision@5": 0.19199999999999995,
1232
+ "precision@10": 0.09699999999999998,
1233
+ "recall@1": 0.89,
1234
+ "recall@5": 0.96,
1235
+ "recall@10": 0.97,
1236
+ "f1@1": 0.89,
1237
+ "f1@5": 0.32000000000000006,
1238
+ "f1@10": 0.17636363636363647,
1239
+ "map@1": 0.89,
1240
+ "map@5": 0.9169999999999999,
1241
+ "map@10": 0.9182499999999999,
1242
+ "mrr@1": 0.89,
1243
+ "mrr@5": 0.9169999999999999,
1244
+ "mrr@10": 0.9182499999999999,
1245
+ "num_pred": 100,
1246
+ "num_data": 100
1247
+ },
1248
+ "ViDoRe_syntheticDocQA_government_reports": {
1249
+ "hit@1": 0.95,
1250
+ "hit@5": 0.99,
1251
+ "hit@10": 1.0,
1252
+ "ndcg_linear@1": 0.95,
1253
+ "ndcg_linear@5": 0.9719253606521631,
1254
+ "ndcg_linear@10": 0.9749356606088029,
1255
+ "ndcg_exponential@1": 0.95,
1256
+ "ndcg_exponential@5": 0.9719253606521631,
1257
+ "ndcg_exponential@10": 0.9749356606088029,
1258
+ "precision@1": 0.95,
1259
+ "precision@5": 0.19799999999999998,
1260
+ "precision@10": 0.09999999999999998,
1261
+ "recall@1": 0.95,
1262
+ "recall@5": 0.99,
1263
+ "recall@10": 1.0,
1264
+ "f1@1": 0.95,
1265
+ "f1@5": 0.3300000000000001,
1266
+ "f1@10": 0.1818181818181819,
1267
+ "map@1": 0.95,
1268
+ "map@5": 0.9658333333333333,
1269
+ "map@10": 0.9669444444444444,
1270
+ "mrr@1": 0.95,
1271
+ "mrr@5": 0.9658333333333333,
1272
+ "mrr@10": 0.9669444444444444,
1273
+ "num_pred": 100,
1274
+ "num_data": 100
1275
+ },
1276
+ "ViDoRe_syntheticDocQA_healthcare_industry": {
1277
+ "hit@1": 0.95,
1278
+ "hit@5": 1.0,
1279
+ "hit@10": 1.0,
1280
+ "ndcg_linear@1": 0.95,
1281
+ "ndcg_linear@5": 0.9815464876785728,
1282
+ "ndcg_linear@10": 0.9815464876785728,
1283
+ "ndcg_exponential@1": 0.95,
1284
+ "ndcg_exponential@5": 0.9815464876785728,
1285
+ "ndcg_exponential@10": 0.9815464876785728,
1286
+ "precision@1": 0.95,
1287
+ "precision@5": 0.19999999999999996,
1288
+ "precision@10": 0.09999999999999998,
1289
+ "recall@1": 0.95,
1290
+ "recall@5": 1.0,
1291
+ "recall@10": 1.0,
1292
+ "f1@1": 0.95,
1293
+ "f1@5": 0.3333333333333335,
1294
+ "f1@10": 0.1818181818181819,
1295
+ "map@1": 0.95,
1296
+ "map@5": 0.975,
1297
+ "map@10": 0.975,
1298
+ "mrr@1": 0.95,
1299
+ "mrr@5": 0.975,
1300
+ "mrr@10": 0.975,
1301
+ "num_pred": 100,
1302
+ "num_data": 100
1303
+ },
1304
+ "ViDoRe_esg_reports_human_labeled_v2": {
1305
+ "hit@1": 0.5384615384615384,
1306
+ "hit@5": 0.7884615384615384,
1307
+ "hit@10": 0.9038461538461539,
1308
+ "ndcg_linear@1": 0.5256410256410257,
1309
+ "ndcg_linear@5": 0.589296384542775,
1310
+ "ndcg_linear@10": 0.624478878331889,
1311
+ "ndcg_exponential@1": 0.521978021978022,
1312
+ "ndcg_exponential@5": 0.5887342206254591,
1313
+ "ndcg_exponential@10": 0.6232093836929005,
1314
+ "precision@1": 0.5384615384615384,
1315
+ "precision@5": 0.2307692307692308,
1316
+ "precision@10": 0.14807692307692308,
1317
+ "recall@1": 0.3705128205128205,
1318
+ "recall@5": 0.6426739926739926,
1319
+ "recall@10": 0.733570596070596,
1320
+ "f1@1": 0.4074457593688363,
1321
+ "f1@5": 0.30411998132586376,
1322
+ "f1@10": 0.22409146283580675,
1323
+ "map@1": 0.5384615384615384,
1324
+ "map@5": 0.5308173076923077,
1325
+ "map@10": 0.5468327737077737,
1326
+ "mrr@1": 0.5384615384615384,
1327
+ "mrr@5": 0.6416666666666667,
1328
+ "mrr@10": 0.6583562271062271,
1329
+ "num_pred": 52,
1330
+ "num_data": 52
1331
+ },
1332
+ "ViDoRe_biomedical_lectures_v2": {
1333
+ "hit@1": 0.56875,
1334
+ "hit@5": 0.85625,
1335
+ "hit@10": 0.925,
1336
+ "ndcg_linear@1": 0.56875,
1337
+ "ndcg_linear@5": 0.6121447963138754,
1338
+ "ndcg_linear@10": 0.6524179326238384,
1339
+ "ndcg_exponential@1": 0.56875,
1340
+ "ndcg_exponential@5": 0.6121447963138754,
1341
+ "ndcg_exponential@10": 0.6524179326238384,
1342
+ "precision@1": 0.56875,
1343
+ "precision@5": 0.28375000000000006,
1344
+ "precision@10": 0.18375000000000002,
1345
+ "recall@1": 0.3469893127705628,
1346
+ "recall@5": 0.6348529013878279,
1347
+ "recall@10": 0.757810152470079,
1348
+ "f1@1": 0.39619340728715724,
1349
+ "f1@5": 0.345669126012876,
1350
+ "f1@10": 0.26630844713565305,
1351
+ "map@1": 0.56875,
1352
+ "map@5": 0.5411024305555555,
1353
+ "map@10": 0.5622742996504158,
1354
+ "mrr@1": 0.56875,
1355
+ "mrr@5": 0.6859375,
1356
+ "mrr@10": 0.694858630952381,
1357
+ "num_pred": 160,
1358
+ "num_data": 160
1359
+ },
1360
+ "ViDoRe_biomedical_lectures_v2_multilingual": {
1361
+ "hit@1": 0.5421875,
1362
+ "hit@5": 0.8484375,
1363
+ "hit@10": 0.9109375,
1364
+ "ndcg_linear@1": 0.5421875,
1365
+ "ndcg_linear@5": 0.5986619666059443,
1366
+ "ndcg_linear@10": 0.6293523673040508,
1367
+ "ndcg_exponential@1": 0.5421875,
1368
+ "ndcg_exponential@5": 0.5986619666059443,
1369
+ "ndcg_exponential@10": 0.6293523673040508,
1370
+ "precision@1": 0.5421875,
1371
+ "precision@5": 0.28156250000000005,
1372
+ "precision@10": 0.1771875,
1373
+ "recall@1": 0.3202389407467533,
1374
+ "recall@5": 0.635897432836771,
1375
+ "recall@10": 0.736068589171562,
1376
+ "f1@1": 0.36738050144300144,
1377
+ "f1@5": 0.34408507724913984,
1378
+ "f1@10": 0.2570341234541602,
1379
+ "map@1": 0.5421875,
1380
+ "map@5": 0.5220464409722222,
1381
+ "map@10": 0.537115656592498,
1382
+ "mrr@1": 0.5421875,
1383
+ "mrr@5": 0.668203125,
1384
+ "mrr@10": 0.6768824404761904,
1385
+ "num_pred": 640,
1386
+ "num_data": 640
1387
+ },
1388
+ "ViDoRe_economics_reports_v2": {
1389
+ "hit@1": 0.6724137931034483,
1390
+ "hit@5": 0.8793103448275862,
1391
+ "hit@10": 0.9482758620689655,
1392
+ "ndcg_linear@1": 0.6724137931034483,
1393
+ "ndcg_linear@5": 0.6146410495038206,
1394
+ "ndcg_linear@10": 0.5781758906590995,
1395
+ "ndcg_exponential@1": 0.6724137931034483,
1396
+ "ndcg_exponential@5": 0.6146410495038206,
1397
+ "ndcg_exponential@10": 0.5781758906590994,
1398
+ "precision@1": 0.6724137931034483,
1399
+ "precision@5": 0.5275862068965518,
1400
+ "precision@10": 0.40689655172413797,
1401
+ "recall@1": 0.10390117860620204,
1402
+ "recall@5": 0.3192381710568663,
1403
+ "recall@10": 0.4408362472908934,
1404
+ "f1@1": 0.15301125939339272,
1405
+ "f1@5": 0.32327849474391646,
1406
+ "f1@10": 0.3452117138076931,
1407
+ "map@1": 0.6724137931034483,
1408
+ "map@5": 0.5211637931034483,
1409
+ "map@10": 0.44485888480551866,
1410
+ "mrr@1": 0.6724137931034483,
1411
+ "mrr@5": 0.7649425287356322,
1412
+ "mrr@10": 0.773611111111111,
1413
+ "num_pred": 58,
1414
+ "num_data": 58
1415
+ },
1416
+ "ViDoRe_economics_reports_v2_multilingual": {
1417
+ "hit@1": 0.6594827586206896,
1418
+ "hit@5": 0.853448275862069,
1419
+ "hit@10": 0.9008620689655172,
1420
+ "ndcg_linear@1": 0.6594827586206896,
1421
+ "ndcg_linear@5": 0.5798064778470571,
1422
+ "ndcg_linear@10": 0.5442831167476708,
1423
+ "ndcg_exponential@1": 0.6594827586206896,
1424
+ "ndcg_exponential@5": 0.5798064778470569,
1425
+ "ndcg_exponential@10": 0.5442831167476708,
1426
+ "precision@1": 0.6594827586206896,
1427
+ "precision@5": 0.49051724137931035,
1428
+ "precision@10": 0.3801724137931034,
1429
+ "recall@1": 0.11892481076730273,
1430
+ "recall@5": 0.2952923353492653,
1431
+ "recall@10": 0.3976602828933967,
1432
+ "f1@1": 0.16378618502482922,
1433
+ "f1@5": 0.2984260202015666,
1434
+ "f1@10": 0.315370215256687,
1435
+ "map@1": 0.6594827586206896,
1436
+ "map@5": 0.4882866379310345,
1437
+ "map@10": 0.4196400328570125,
1438
+ "mrr@1": 0.6594827586206896,
1439
+ "mrr@5": 0.739655172413793,
1440
+ "mrr@10": 0.7464063355227147,
1441
+ "num_pred": 232,
1442
+ "num_data": 232
1443
+ },
1444
+ "ViDoRe_esg_reports_v2": {
1445
+ "hit@1": 0.43859649122807015,
1446
+ "hit@5": 0.8421052631578947,
1447
+ "hit@10": 0.9122807017543859,
1448
+ "ndcg_linear@1": 0.43859649122807015,
1449
+ "ndcg_linear@5": 0.5450714742581306,
1450
+ "ndcg_linear@10": 0.5810543681505067,
1451
+ "ndcg_exponential@1": 0.43859649122807015,
1452
+ "ndcg_exponential@5": 0.5450714742581306,
1453
+ "ndcg_exponential@10": 0.5810543681505067,
1454
+ "precision@1": 0.43859649122807015,
1455
+ "precision@5": 0.2807017543859649,
1456
+ "precision@10": 0.18947368421052632,
1457
+ "recall@1": 0.21871345029239764,
1458
+ "recall@5": 0.6043233082706767,
1459
+ "recall@10": 0.7227652464494568,
1460
+ "f1@1": 0.26665186665186663,
1461
+ "f1@5": 0.33131290240823713,
1462
+ "f1@10": 0.2609817805475293,
1463
+ "map@1": 0.43859649122807015,
1464
+ "map@5": 0.4562378167641325,
1465
+ "map@10": 0.46908324050867917,
1466
+ "mrr@1": 0.43859649122807015,
1467
+ "mrr@5": 0.5976608187134502,
1468
+ "mrr@10": 0.6069896964633807,
1469
+ "num_pred": 57,
1470
+ "num_data": 57
1471
+ },
1472
+ "ViDoRe_esg_reports_v2_multilingual": {
1473
+ "hit@1": 0.4780701754385965,
1474
+ "hit@5": 0.8157894736842105,
1475
+ "hit@10": 0.8903508771929824,
1476
+ "ndcg_linear@1": 0.4780701754385965,
1477
+ "ndcg_linear@5": 0.5373421213557084,
1478
+ "ndcg_linear@10": 0.5783023724706139,
1479
+ "ndcg_exponential@1": 0.4780701754385965,
1480
+ "ndcg_exponential@5": 0.5373421213557084,
1481
+ "ndcg_exponential@10": 0.5783023724706139,
1482
+ "precision@1": 0.4780701754385965,
1483
+ "precision@5": 0.26403508771929823,
1484
+ "precision@10": 0.18333333333333338,
1485
+ "recall@1": 0.25607769423558896,
1486
+ "recall@5": 0.5779709690893902,
1487
+ "recall@10": 0.6975563909774436,
1488
+ "f1@1": 0.3051582483524407,
1489
+ "f1@5": 0.31453422392623853,
1490
+ "f1@10": 0.25245907358503084,
1491
+ "map@1": 0.4780701754385965,
1492
+ "map@5": 0.4532309941520468,
1493
+ "map@10": 0.47189643669358583,
1494
+ "mrr@1": 0.4780701754385965,
1495
+ "mrr@5": 0.6087719298245614,
1496
+ "mrr@10": 0.6179546087440825,
1497
+ "num_pred": 228,
1498
+ "num_data": 228
1499
+ },
1500
+ "VisRAG_ArxivQA": {
1501
+ "hit@1": 0.7867647058823529,
1502
+ "hit@5": 0.8811274509803921,
1503
+ "hit@10": 0.9191176470588235,
1504
+ "ndcg_linear@1": 0.7867647058823529,
1505
+ "ndcg_linear@5": 0.8368378393839788,
1506
+ "ndcg_linear@10": 0.8493028487060746,
1507
+ "ndcg_exponential@1": 0.7867647058823529,
1508
+ "ndcg_exponential@5": 0.8368378393839788,
1509
+ "ndcg_exponential@10": 0.8493028487060746,
1510
+ "precision@1": 0.7867647058823529,
1511
+ "precision@5": 0.17622549019607844,
1512
+ "precision@10": 0.09191176470588236,
1513
+ "recall@1": 0.7867647058823529,
1514
+ "recall@5": 0.8811274509803921,
1515
+ "recall@10": 0.9191176470588235,
1516
+ "f1@1": 0.7867647058823529,
1517
+ "f1@5": 0.29370915032679745,
1518
+ "f1@10": 0.16711229946524064,
1519
+ "map@1": 0.7867647058823529,
1520
+ "map@5": 0.822140522875817,
1521
+ "map@10": 0.8273916511048864,
1522
+ "mrr@1": 0.7867647058823529,
1523
+ "mrr@5": 0.822140522875817,
1524
+ "mrr@10": 0.8273916511048864,
1525
+ "num_pred": 816,
1526
+ "num_data": 816
1527
+ },
1528
+ "VisRAG_ChartQA": {
1529
+ "hit@1": 0.7936507936507936,
1530
+ "hit@5": 0.9206349206349206,
1531
+ "hit@10": 0.9206349206349206,
1532
+ "ndcg_linear@1": 0.7936507936507936,
1533
+ "ndcg_linear@5": 0.8639421330527995,
1534
+ "ndcg_linear@10": 0.8639421330527995,
1535
+ "ndcg_exponential@1": 0.7936507936507936,
1536
+ "ndcg_exponential@5": 0.8639421330527995,
1537
+ "ndcg_exponential@10": 0.8639421330527995,
1538
+ "precision@1": 0.7936507936507936,
1539
+ "precision@5": 0.18412698412698403,
1540
+ "precision@10": 0.09206349206349201,
1541
+ "recall@1": 0.7936507936507936,
1542
+ "recall@5": 0.9206349206349206,
1543
+ "recall@10": 0.9206349206349206,
1544
+ "f1@1": 0.7936507936507936,
1545
+ "f1@5": 0.3068783068783068,
1546
+ "f1@10": 0.16738816738816742,
1547
+ "map@1": 0.7936507936507936,
1548
+ "map@5": 0.844973544973545,
1549
+ "map@10": 0.844973544973545,
1550
+ "mrr@1": 0.7936507936507936,
1551
+ "mrr@5": 0.844973544973545,
1552
+ "mrr@10": 0.844973544973545,
1553
+ "num_pred": 63,
1554
+ "num_data": 63
1555
+ },
1556
+ "VisRAG_MP-DocVQA": {
1557
+ "hit@1": 0.7715736040609137,
1558
+ "hit@5": 0.922165820642978,
1559
+ "hit@10": 0.9543147208121827,
1560
+ "ndcg_linear@1": 0.7715736040609137,
1561
+ "ndcg_linear@5": 0.8554025361255457,
1562
+ "ndcg_linear@10": 0.8661922441886736,
1563
+ "ndcg_exponential@1": 0.7715736040609137,
1564
+ "ndcg_exponential@5": 0.8554025361255457,
1565
+ "ndcg_exponential@10": 0.8661922441886736,
1566
+ "precision@1": 0.7715736040609137,
1567
+ "precision@5": 0.1844331641285956,
1568
+ "precision@10": 0.09543147208121826,
1569
+ "recall@1": 0.7715736040609137,
1570
+ "recall@5": 0.922165820642978,
1571
+ "recall@10": 0.9543147208121827,
1572
+ "f1@1": 0.7715736040609137,
1573
+ "f1@5": 0.30738860688099273,
1574
+ "f1@10": 0.1735117674203969,
1575
+ "map@1": 0.7715736040609137,
1576
+ "map@5": 0.8328539199097573,
1577
+ "map@10": 0.8375386082238874,
1578
+ "mrr@1": 0.7715736040609137,
1579
+ "mrr@5": 0.8328539199097573,
1580
+ "mrr@10": 0.8375386082238874,
1581
+ "num_pred": 591,
1582
+ "num_data": 591
1583
+ },
1584
+ "VisRAG_SlideVQA": {
1585
+ "hit@1": 0.9226618705035972,
1586
+ "hit@5": 0.9802158273381295,
1587
+ "hit@10": 0.9838129496402878,
1588
+ "ndcg_linear@1": 0.9226618705035972,
1589
+ "ndcg_linear@5": 0.9406544955213136,
1590
+ "ndcg_linear@10": 0.9437533706328498,
1591
+ "ndcg_exponential@1": 0.9226618705035972,
1592
+ "ndcg_exponential@5": 0.9406544955213136,
1593
+ "ndcg_exponential@10": 0.9437533706328498,
1594
+ "precision@1": 0.9226618705035972,
1595
+ "precision@5": 0.24280575539568344,
1596
+ "precision@10": 0.122841726618705,
1597
+ "recall@1": 0.7985611510791367,
1598
+ "recall@5": 0.9685251798561151,
1599
+ "recall@10": 0.9766187050359713,
1600
+ "f1@1": 0.8399280575539568,
1601
+ "f1@5": 0.3812949640287771,
1602
+ "f1@10": 0.2157183344233704,
1603
+ "map@1": 0.9226618705035972,
1604
+ "map@5": 0.9232464028776979,
1605
+ "map@10": 0.9250995632065777,
1606
+ "mrr@1": 0.9226618705035972,
1607
+ "mrr@5": 0.946702637889688,
1608
+ "mrr@10": 0.9471522781774578,
1609
+ "num_pred": 556,
1610
+ "num_data": 556
1611
+ },
1612
+ "VisRAG_InfoVQA": {
1613
+ "hit@1": 0.8523676880222841,
1614
+ "hit@5": 0.9707520891364902,
1615
+ "hit@10": 0.9888579387186629,
1616
+ "ndcg_linear@1": 0.8523676880222841,
1617
+ "ndcg_linear@5": 0.9191675080778411,
1618
+ "ndcg_linear@10": 0.9252174145083141,
1619
+ "ndcg_exponential@1": 0.8523676880222841,
1620
+ "ndcg_exponential@5": 0.9191675080778411,
1621
+ "ndcg_exponential@10": 0.9252174145083141,
1622
+ "precision@1": 0.8523676880222841,
1623
+ "precision@5": 0.19415041782729803,
1624
+ "precision@10": 0.0988857938718663,
1625
+ "recall@1": 0.8523676880222841,
1626
+ "recall@5": 0.9707520891364902,
1627
+ "recall@10": 0.9888579387186629,
1628
+ "f1@1": 0.8523676880222841,
1629
+ "f1@5": 0.32358402971216355,
1630
+ "f1@10": 0.1797923524943024,
1631
+ "map@1": 0.8523676880222841,
1632
+ "map@5": 0.9016480965645313,
1633
+ "map@10": 0.9042606225405669,
1634
+ "mrr@1": 0.8523676880222841,
1635
+ "mrr@5": 0.9016480965645313,
1636
+ "mrr@10": 0.9042606225405669,
1637
+ "num_pred": 718,
1638
+ "num_data": 718
1639
+ },
1640
+ "VisRAG_PlotQA": {
1641
+ "hit@1": 0.5492468134414832,
1642
+ "hit@5": 0.8377752027809965,
1643
+ "hit@10": 0.9049826187717266,
1644
+ "ndcg_linear@1": 0.5492468134414832,
1645
+ "ndcg_linear@5": 0.705214238124971,
1646
+ "ndcg_linear@10": 0.7270757770493562,
1647
+ "ndcg_exponential@1": 0.5492468134414832,
1648
+ "ndcg_exponential@5": 0.705214238124971,
1649
+ "ndcg_exponential@10": 0.7270757770493562,
1650
+ "precision@1": 0.5492468134414832,
1651
+ "precision@5": 0.16755504055619927,
1652
+ "precision@10": 0.09049826187717265,
1653
+ "recall@1": 0.5492468134414832,
1654
+ "recall@5": 0.8377752027809965,
1655
+ "recall@10": 0.9049826187717266,
1656
+ "f1@1": 0.5492468134414832,
1657
+ "f1@5": 0.27925840092699894,
1658
+ "f1@10": 0.16454229432213213,
1659
+ "map@1": 0.5492468134414832,
1660
+ "map@5": 0.6608342989571262,
1661
+ "map@10": 0.6699258768783681,
1662
+ "mrr@1": 0.5492468134414832,
1663
+ "mrr@5": 0.6608342989571262,
1664
+ "mrr@10": 0.6699258768783681,
1665
+ "num_pred": 863,
1666
+ "num_data": 863
1667
+ },
1668
+ "ViDoSeek-page": {
1669
+ "hit@1": 0.020140105078809107,
1670
+ "hit@5": 0.4352014010507881,
1671
+ "hit@10": 0.6392294220665499,
1672
+ "ndcg_linear@1": 0.020140105078809107,
1673
+ "ndcg_linear@5": 0.22891788122147116,
1674
+ "ndcg_linear@10": 0.2945302515228599,
1675
+ "ndcg_exponential@1": 0.020140105078809107,
1676
+ "ndcg_exponential@5": 0.22891788122147122,
1677
+ "ndcg_exponential@10": 0.2945302515228599,
1678
+ "precision@1": 0.020140105078809107,
1679
+ "precision@5": 0.08704028021015763,
1680
+ "precision@10": 0.063922942206655,
1681
+ "recall@1": 0.020140105078809107,
1682
+ "recall@5": 0.4352014010507881,
1683
+ "recall@10": 0.6392294220665499,
1684
+ "f1@1": 0.020140105078809107,
1685
+ "f1@5": 0.14506713368359606,
1686
+ "f1@10": 0.11622353128482728,
1687
+ "map@1": 0.020140105078809107,
1688
+ "map@5": 0.161456509048453,
1689
+ "map@10": 0.18832214438606731,
1690
+ "mrr@1": 0.020140105078809107,
1691
+ "mrr@5": 0.161456509048453,
1692
+ "mrr@10": 0.18832214438606731,
1693
+ "num_pred": 1142,
1694
+ "num_data": 1142
1695
+ },
1696
+ "ViDoSeek-doc": {
1697
+ "hit@1": 0.9938704028021016,
1698
+ "hit@5": 0.9982486865148862,
1699
+ "hit@10": 0.999124343257443,
1700
+ "ndcg_linear@1": 0.6795096322241682,
1701
+ "ndcg_linear@5": 0.8226918152794956,
1702
+ "ndcg_linear@10": 0.7978366708129369,
1703
+ "ndcg_exponential@1": 0.45496622466850134,
1704
+ "ndcg_exponential@5": 0.7042590807020261,
1705
+ "ndcg_exponential@10": 0.7245604277334962,
1706
+ "precision@1": 0.9938704028021016,
1707
+ "precision@5": 0.9052539404553417,
1708
+ "precision@10": 0.8042031523642732,
1709
+ "recall@1": 0.05860866742322132,
1710
+ "recall@5": 0.2664467811756284,
1711
+ "recall@10": 0.463461685999827,
1712
+ "f1@1": 0.10994096713812122,
1713
+ "f1@5": 0.4033655836761433,
1714
+ "f1@10": 0.5748187922741919,
1715
+ "map@1": 0.9938704028021016,
1716
+ "map@5": 0.896579100992411,
1717
+ "map@10": 0.7894488850544434,
1718
+ "mrr@1": 0.9938704028021016,
1719
+ "mrr@5": 0.9957968476357268,
1720
+ "mrr@10": 0.9958844133099825,
1721
+ "num_pred": 1142,
1722
+ "num_data": 1142
1723
+ },
1724
+ "MMLongBench-page": {
1725
+ "hit@1": 0.06921241050119331,
1726
+ "hit@5": 0.324582338902148,
1727
+ "hit@10": 0.42601431980906923,
1728
+ "ndcg_linear@1": 0.06921241050119331,
1729
+ "ndcg_linear@5": 0.16207753250349682,
1730
+ "ndcg_linear@10": 0.19677211691223617,
1731
+ "ndcg_exponential@1": 0.06921241050119331,
1732
+ "ndcg_exponential@5": 0.16207753250349682,
1733
+ "ndcg_exponential@10": 0.19677211691223617,
1734
+ "precision@1": 0.06921241050119331,
1735
+ "precision@5": 0.07661097852028638,
1736
+ "precision@10": 0.05560859188544153,
1737
+ "recall@1": 0.03900159108989658,
1738
+ "recall@5": 0.2395343220820548,
1739
+ "recall@10": 0.3357341743379929,
1740
+ "f1@1": 0.04752522735816291,
1741
+ "f1@5": 0.10982885412900857,
1742
+ "f1@10": 0.0908506340699305,
1743
+ "map@1": 0.06921241050119331,
1744
+ "map@5": 0.11860547600106072,
1745
+ "map@10": 0.1342412405495213,
1746
+ "mrr@1": 0.06921241050119331,
1747
+ "mrr@5": 0.16272871917263326,
1748
+ "mrr@10": 0.17640783043527675,
1749
+ "num_pred": 838,
1750
+ "num_data": 838
1751
+ },
1752
+ "MMLongBench-doc": {
1753
+ "hit@1": 0.711217183770883,
1754
+ "hit@5": 0.8031026252983293,
1755
+ "hit@10": 0.834128878281623,
1756
+ "ndcg_linear@1": 0.4988066825775656,
1757
+ "ndcg_linear@5": 0.519541593888753,
1758
+ "ndcg_linear@10": 0.49663505664434826,
1759
+ "ndcg_exponential@1": 0.3470848960109103,
1760
+ "ndcg_exponential@5": 0.43570154580159437,
1761
+ "ndcg_exponential@10": 0.44278868712368374,
1762
+ "precision@1": 0.711217183770883,
1763
+ "precision@5": 0.5730310262529832,
1764
+ "precision@10": 0.5023866348448688,
1765
+ "recall@1": 0.023227151217942815,
1766
+ "recall@5": 0.09066544823381396,
1767
+ "recall@10": 0.1549273781103741,
1768
+ "f1@1": 0.04453672442732697,
1769
+ "f1@5": 0.15029740680454912,
1770
+ "f1@10": 0.22210925386216956,
1771
+ "map@1": 0.711217183770883,
1772
+ "map@5": 0.5397295147175815,
1773
+ "map@10": 0.45481248395230095,
1774
+ "mrr@1": 0.711217183770883,
1775
+ "mrr@5": 0.7474343675417662,
1776
+ "mrr@10": 0.7516578588475964,
1777
+ "num_pred": 838,
1778
+ "num_data": 838
1779
+ }
1780
+ },
1781
+ "video": {
1782
+ "DiDeMo": {
1783
+ "hit@1": 0.3954183266932271,
1784
+ "hit@5": 0.6454183266932271,
1785
+ "hit@10": 0.7310756972111554,
1786
+ "ndcg_linear@1": 0.3954183266932271,
1787
+ "ndcg_linear@5": 0.5292216820901535,
1788
+ "ndcg_linear@10": 0.557033676703922,
1789
+ "ndcg_exponential@1": 0.3954183266932271,
1790
+ "ndcg_exponential@5": 0.5292216820901535,
1791
+ "ndcg_exponential@10": 0.557033676703922,
1792
+ "precision@1": 0.3954183266932271,
1793
+ "precision@5": 0.1290836653386454,
1794
+ "precision@10": 0.07310756972111554,
1795
+ "recall@1": 0.3954183266932271,
1796
+ "recall@5": 0.6454183266932271,
1797
+ "recall@10": 0.7310756972111554,
1798
+ "f1@1": 0.3954183266932271,
1799
+ "f1@5": 0.21513944223107576,
1800
+ "f1@10": 0.13292285403839194,
1801
+ "map@1": 0.3954183266932271,
1802
+ "map@5": 0.4904714475431607,
1803
+ "map@10": 0.5020165370264972,
1804
+ "mrr@1": 0.3954183266932271,
1805
+ "mrr@5": 0.4904714475431607,
1806
+ "mrr@10": 0.5020165370264972,
1807
+ "num_pred": 1004,
1808
+ "num_data": 1004
1809
+ },
1810
+ "MSR-VTT": {
1811
+ "hit@1": 0.43,
1812
+ "hit@5": 0.655,
1813
+ "hit@10": 0.759,
1814
+ "ndcg_linear@1": 0.43,
1815
+ "ndcg_linear@5": 0.5477542728912066,
1816
+ "ndcg_linear@10": 0.5814164877334193,
1817
+ "ndcg_exponential@1": 0.43,
1818
+ "ndcg_exponential@5": 0.5477542728912066,
1819
+ "ndcg_exponential@10": 0.5814164877334193,
1820
+ "precision@1": 0.43,
1821
+ "precision@5": 0.131,
1822
+ "precision@10": 0.07590000000000001,
1823
+ "recall@1": 0.43,
1824
+ "recall@5": 0.655,
1825
+ "recall@10": 0.759,
1826
+ "f1@1": 0.43,
1827
+ "f1@5": 0.2183333333333334,
1828
+ "f1@10": 0.138,
1829
+ "map@1": 0.43,
1830
+ "map@5": 0.51225,
1831
+ "map@10": 0.5261535714285714,
1832
+ "mrr@1": 0.43,
1833
+ "mrr@5": 0.51225,
1834
+ "mrr@10": 0.5261535714285714,
1835
+ "num_pred": 1000,
1836
+ "num_data": 1000
1837
+ },
1838
+ "MSVD": {
1839
+ "hit@1": 0.5835820895522388,
1840
+ "hit@5": 0.8477611940298507,
1841
+ "hit@10": 0.9149253731343283,
1842
+ "ndcg_linear@1": 0.5835820895522388,
1843
+ "ndcg_linear@5": 0.7294517948333389,
1844
+ "ndcg_linear@10": 0.7515440479150944,
1845
+ "ndcg_exponential@1": 0.5835820895522388,
1846
+ "ndcg_exponential@5": 0.7294517948333389,
1847
+ "ndcg_exponential@10": 0.7515440479150944,
1848
+ "precision@1": 0.5835820895522388,
1849
+ "precision@5": 0.16955223880597012,
1850
+ "precision@10": 0.09149253731343282,
1851
+ "recall@1": 0.5835820895522388,
1852
+ "recall@5": 0.8477611940298507,
1853
+ "recall@10": 0.9149253731343283,
1854
+ "f1@1": 0.5835820895522388,
1855
+ "f1@5": 0.28258706467661704,
1856
+ "f1@10": 0.16635006784260517,
1857
+ "map@1": 0.5835820895522388,
1858
+ "map@5": 0.6896517412935323,
1859
+ "map@10": 0.6989913527600095,
1860
+ "mrr@1": 0.5835820895522388,
1861
+ "mrr@5": 0.6896517412935323,
1862
+ "mrr@10": 0.6989913527600095,
1863
+ "num_pred": 670,
1864
+ "num_data": 670
1865
+ },
1866
+ "VATEX": {
1867
+ "hit@1": 0.3378740509155873,
1868
+ "hit@5": 0.6025011165698972,
1869
+ "hit@10": 0.71572130415364,
1870
+ "ndcg_linear@1": 0.3378740509155873,
1871
+ "ndcg_linear@5": 0.47769987329524893,
1872
+ "ndcg_linear@10": 0.5144788352866694,
1873
+ "ndcg_exponential@1": 0.3378740509155873,
1874
+ "ndcg_exponential@5": 0.47769987329524893,
1875
+ "ndcg_exponential@10": 0.5144788352866694,
1876
+ "precision@1": 0.3378740509155873,
1877
+ "precision@5": 0.12050022331397943,
1878
+ "precision@10": 0.071572130415364,
1879
+ "recall@1": 0.3378740509155873,
1880
+ "recall@5": 0.6025011165698972,
1881
+ "recall@10": 0.71572130415364,
1882
+ "f1@1": 0.3378740509155873,
1883
+ "f1@5": 0.20083370552329913,
1884
+ "f1@10": 0.13013114620975275,
1885
+ "map@1": 0.3378740509155873,
1886
+ "map@5": 0.43629968736042873,
1887
+ "map@10": 0.4515752497217437,
1888
+ "mrr@1": 0.3378740509155873,
1889
+ "mrr@5": 0.43629968736042873,
1890
+ "mrr@10": 0.4515752497217437,
1891
+ "num_pred": 4478,
1892
+ "num_data": 4478
1893
+ },
1894
+ "YouCook2": {
1895
+ "hit@1": 0.1689210443535703,
1896
+ "hit@5": 0.35199748348537274,
1897
+ "hit@10": 0.443535703051274,
1898
+ "ndcg_linear@1": 0.1689210443535703,
1899
+ "ndcg_linear@5": 0.2629752832004448,
1900
+ "ndcg_linear@10": 0.29271527912337164,
1901
+ "ndcg_exponential@1": 0.1689210443535703,
1902
+ "ndcg_exponential@5": 0.2629752832004448,
1903
+ "ndcg_exponential@10": 0.29271527912337164,
1904
+ "precision@1": 0.1689210443535703,
1905
+ "precision@5": 0.07039949669707456,
1906
+ "precision@10": 0.0443535703051274,
1907
+ "recall@1": 0.1689210443535703,
1908
+ "recall@5": 0.35199748348537274,
1909
+ "recall@10": 0.443535703051274,
1910
+ "f1@1": 0.1689210443535703,
1911
+ "f1@5": 0.11733249449512427,
1912
+ "f1@10": 0.08064285510023164,
1913
+ "map@1": 0.1689210443535703,
1914
+ "map@5": 0.23367935409457902,
1915
+ "map@10": 0.24603074741483047,
1916
+ "mrr@1": 0.1689210443535703,
1917
+ "mrr@5": 0.23367935409457902,
1918
+ "mrr@10": 0.24603074741483047,
1919
+ "num_pred": 3179,
1920
+ "num_data": 3179
1921
+ },
1922
+ "QVHighlight": {
1923
+ "hit@1": 0.5152354570637119,
1924
+ "hit@5": 0.8587257617728532,
1925
+ "hit@10": 1.0,
1926
+ "ndcg_linear@1": 0.5152354570637119,
1927
+ "ndcg_linear@5": 0.6968949839565493,
1928
+ "ndcg_linear@10": 0.742731199912767,
1929
+ "ndcg_exponential@1": 0.5152354570637119,
1930
+ "ndcg_exponential@5": 0.6968949839565493,
1931
+ "ndcg_exponential@10": 0.742731199912767,
1932
+ "precision@1": 0.5152354570637119,
1933
+ "precision@5": 0.17174515235457063,
1934
+ "precision@10": 0.09999999999999998,
1935
+ "recall@1": 0.5152354570637119,
1936
+ "recall@5": 0.8587257617728532,
1937
+ "recall@10": 1.0,
1938
+ "f1@1": 0.5152354570637119,
1939
+ "f1@5": 0.28624192059095116,
1940
+ "f1@10": 0.18181818181818185,
1941
+ "map@1": 0.5152354570637119,
1942
+ "map@5": 0.6431517389966143,
1943
+ "map@10": 0.6621561212973954,
1944
+ "mrr@1": 0.5152354570637119,
1945
+ "mrr@5": 0.6431517389966143,
1946
+ "mrr@10": 0.6621561212973954,
1947
+ "num_pred": 1083,
1948
+ "num_data": 1083
1949
+ },
1950
+ "Charades-STA": {
1951
+ "hit@1": 0.1815680880330124,
1952
+ "hit@5": 0.5295735900962861,
1953
+ "hit@10": 1.0,
1954
+ "ndcg_linear@1": 0.1815680880330124,
1955
+ "ndcg_linear@5": 0.35150044729928126,
1956
+ "ndcg_linear@10": 0.49842400295841127,
1957
+ "ndcg_exponential@1": 0.1815680880330124,
1958
+ "ndcg_exponential@5": 0.35150044729928126,
1959
+ "ndcg_exponential@10": 0.49842400295841127,
1960
+ "precision@1": 0.1815680880330124,
1961
+ "precision@5": 0.1059147180192572,
1962
+ "precision@10": 0.09999999999999998,
1963
+ "recall@1": 0.1815680880330124,
1964
+ "recall@5": 0.5295735900962861,
1965
+ "recall@10": 1.0,
1966
+ "f1@1": 0.1815680880330124,
1967
+ "f1@5": 0.17652453003209542,
1968
+ "f1@10": 0.18181818181818185,
1969
+ "map@1": 0.1815680880330124,
1970
+ "map@5": 0.29371847776249427,
1971
+ "map@10": 0.35136787406388503,
1972
+ "mrr@1": 0.1815680880330124,
1973
+ "mrr@5": 0.29371847776249427,
1974
+ "mrr@10": 0.35136787406388503,
1975
+ "num_pred": 727,
1976
+ "num_data": 727
1977
+ },
1978
+ "MomentSeeker": {
1979
+ "hit@1": 0.4038888888888889,
1980
+ "hit@5": 0.8922222222222222,
1981
+ "hit@10": 1.0,
1982
+ "ndcg_linear@1": 0.4038888888888889,
1983
+ "ndcg_linear@5": 0.6653924165164706,
1984
+ "ndcg_linear@10": 0.7059801123805041,
1985
+ "ndcg_exponential@1": 0.4038888888888889,
1986
+ "ndcg_exponential@5": 0.6653924165164706,
1987
+ "ndcg_exponential@10": 0.7059801123805041,
1988
+ "precision@1": 0.4038888888888889,
1989
+ "precision@5": 0.19644444444444445,
1990
+ "precision@10": 0.11261111111111112,
1991
+ "recall@1": 0.381550925925926,
1992
+ "recall@5": 0.8777546296296297,
1993
+ "recall@10": 0.9972685185185186,
1994
+ "f1@1": 0.38838536155202824,
1995
+ "f1@5": 0.3158506154339488,
1996
+ "f1@10": 0.19978680270346938,
1997
+ "map@1": 0.4038888888888889,
1998
+ "map@5": 0.5893097222222223,
1999
+ "map@10": 0.6081910640799529,
2000
+ "mrr@1": 0.4038888888888889,
2001
+ "mrr@5": 0.5971944444444444,
2002
+ "mrr@10": 0.612231922398589,
2003
+ "num_pred": 1800,
2004
+ "num_data": 1800
2005
+ },
2006
+ "K700": {
2007
+ "hit@1": 0.464,
2008
+ "hit@5": 0.703,
2009
+ "hit@10": 0.789,
2010
+ "ndcg_linear@1": 0.464,
2011
+ "ndcg_linear@5": 0.5931449504305456,
2012
+ "ndcg_linear@10": 0.6208968507825027,
2013
+ "ndcg_exponential@1": 0.464,
2014
+ "ndcg_exponential@5": 0.5931449504305456,
2015
+ "ndcg_exponential@10": 0.6208968507825027,
2016
+ "precision@1": 0.464,
2017
+ "precision@5": 0.14059999999999997,
2018
+ "precision@10": 0.07890000000000001,
2019
+ "recall@1": 0.464,
2020
+ "recall@5": 0.703,
2021
+ "recall@10": 0.789,
2022
+ "f1@1": 0.464,
2023
+ "f1@5": 0.23433333333333337,
2024
+ "f1@10": 0.14345454545454547,
2025
+ "map@1": 0.464,
2026
+ "map@5": 0.5564,
2027
+ "map@10": 0.5678150793650794,
2028
+ "mrr@1": 0.464,
2029
+ "mrr@5": 0.5564,
2030
+ "mrr@10": 0.5678150793650794,
2031
+ "num_pred": 1000,
2032
+ "num_data": 1000
2033
+ },
2034
+ "SmthSmthV2": {
2035
+ "hit@1": 0.499,
2036
+ "hit@5": 0.781,
2037
+ "hit@10": 0.862,
2038
+ "ndcg_linear@1": 0.499,
2039
+ "ndcg_linear@5": 0.6544550514934299,
2040
+ "ndcg_linear@10": 0.6810844225527735,
2041
+ "ndcg_exponential@1": 0.499,
2042
+ "ndcg_exponential@5": 0.6544550514934299,
2043
+ "ndcg_exponential@10": 0.6810844225527735,
2044
+ "precision@1": 0.499,
2045
+ "precision@5": 0.15619999999999998,
2046
+ "precision@10": 0.08620000000000001,
2047
+ "recall@1": 0.499,
2048
+ "recall@5": 0.781,
2049
+ "recall@10": 0.862,
2050
+ "f1@1": 0.499,
2051
+ "f1@5": 0.26033333333333336,
2052
+ "f1@10": 0.15672727272727274,
2053
+ "map@1": 0.499,
2054
+ "map@5": 0.6119,
2055
+ "map@10": 0.6231444444444445,
2056
+ "mrr@1": 0.499,
2057
+ "mrr@5": 0.6119,
2058
+ "mrr@10": 0.6231444444444445,
2059
+ "num_pred": 1000,
2060
+ "num_data": 1000
2061
+ },
2062
+ "HMDB51": {
2063
+ "hit@1": 0.489,
2064
+ "hit@5": 0.774,
2065
+ "hit@10": 0.854,
2066
+ "ndcg_linear@1": 0.489,
2067
+ "ndcg_linear@5": 0.6446285128958543,
2068
+ "ndcg_linear@10": 0.6704831072791624,
2069
+ "ndcg_exponential@1": 0.489,
2070
+ "ndcg_exponential@5": 0.6446285128958543,
2071
+ "ndcg_exponential@10": 0.6704831072791624,
2072
+ "precision@1": 0.489,
2073
+ "precision@5": 0.15480000000000002,
2074
+ "precision@10": 0.0854,
2075
+ "recall@1": 0.489,
2076
+ "recall@5": 0.774,
2077
+ "recall@10": 0.854,
2078
+ "f1@1": 0.489,
2079
+ "f1@5": 0.25800000000000006,
2080
+ "f1@10": 0.15527272727272728,
2081
+ "map@1": 0.489,
2082
+ "map@5": 0.6011833333333334,
2083
+ "map@10": 0.6118492063492063,
2084
+ "mrr@1": 0.489,
2085
+ "mrr@5": 0.6011833333333334,
2086
+ "mrr@10": 0.6118492063492063,
2087
+ "num_pred": 1000,
2088
+ "num_data": 1000
2089
+ },
2090
+ "UCF101": {
2091
+ "hit@1": 0.699,
2092
+ "hit@5": 0.905,
2093
+ "hit@10": 0.939,
2094
+ "ndcg_linear@1": 0.699,
2095
+ "ndcg_linear@5": 0.8157725031603894,
2096
+ "ndcg_linear@10": 0.82703643669356,
2097
+ "ndcg_exponential@1": 0.699,
2098
+ "ndcg_exponential@5": 0.8157725031603894,
2099
+ "ndcg_exponential@10": 0.82703643669356,
2100
+ "precision@1": 0.699,
2101
+ "precision@5": 0.181,
2102
+ "precision@10": 0.09390000000000001,
2103
+ "recall@1": 0.699,
2104
+ "recall@5": 0.905,
2105
+ "recall@10": 0.939,
2106
+ "f1@1": 0.699,
2107
+ "f1@5": 0.30166666666666664,
2108
+ "f1@10": 0.17072727272727273,
2109
+ "map@1": 0.699,
2110
+ "map@5": 0.7854833333333333,
2111
+ "map@10": 0.7902900793650793,
2112
+ "mrr@1": 0.699,
2113
+ "mrr@5": 0.7854833333333333,
2114
+ "mrr@10": 0.7902900793650793,
2115
+ "num_pred": 1000,
2116
+ "num_data": 1000
2117
+ },
2118
+ "Breakfast": {
2119
+ "hit@1": 0.12702078521939955,
2120
+ "hit@5": 0.6073903002309469,
2121
+ "hit@10": 1.0,
2122
+ "ndcg_linear@1": 0.12702078521939955,
2123
+ "ndcg_linear@5": 0.3635576849613301,
2124
+ "ndcg_linear@10": 0.4911876506328746,
2125
+ "ndcg_exponential@1": 0.12702078521939955,
2126
+ "ndcg_exponential@5": 0.3635576849613301,
2127
+ "ndcg_exponential@10": 0.4911876506328746,
2128
+ "precision@1": 0.12702078521939955,
2129
+ "precision@5": 0.12147806004618936,
2130
+ "precision@10": 0.1,
2131
+ "recall@1": 0.12702078521939955,
2132
+ "recall@5": 0.6073903002309469,
2133
+ "recall@10": 1.0,
2134
+ "f1@1": 0.12702078521939955,
2135
+ "f1@5": 0.20246343341031567,
2136
+ "f1@10": 0.1818181818181818,
2137
+ "map@1": 0.12702078521939955,
2138
+ "map@5": 0.28421862971516554,
2139
+ "map@10": 0.3372346860222149,
2140
+ "mrr@1": 0.12702078521939955,
2141
+ "mrr@5": 0.28421862971516554,
2142
+ "mrr@10": 0.3372346860222149,
2143
+ "num_pred": 433,
2144
+ "num_data": 433
2145
+ },
2146
+ "MVBench": {
2147
+ "hit@1": 0.41575,
2148
+ "hit@5": 1.0,
2149
+ "hit@10": 1.0,
2150
+ "ndcg_linear@1": 0.41575,
2151
+ "ndcg_linear@5": 0.7383191847708812,
2152
+ "ndcg_linear@10": 0.7383191847708812,
2153
+ "ndcg_exponential@1": 0.41575,
2154
+ "ndcg_exponential@5": 0.7383191847708812,
2155
+ "ndcg_exponential@10": 0.7383191847708812,
2156
+ "precision@1": 0.41575,
2157
+ "precision@5": 0.20000000000000004,
2158
+ "precision@10": 0.10000000000000002,
2159
+ "recall@1": 0.41575,
2160
+ "recall@5": 1.0,
2161
+ "recall@10": 1.0,
2162
+ "f1@1": 0.41575,
2163
+ "f1@5": 0.3333333333333333,
2164
+ "f1@10": 0.18181818181818182,
2165
+ "map@1": 0.41575,
2166
+ "map@5": 0.6497583333333333,
2167
+ "map@10": 0.6497583333333333,
2168
+ "mrr@1": 0.41575,
2169
+ "mrr@5": 0.6497583333333333,
2170
+ "mrr@10": 0.6497583333333333,
2171
+ "num_pred": 4000,
2172
+ "num_data": 4000
2173
+ },
2174
+ "Video-MME": {
2175
+ "hit@1": 0.3977777777777778,
2176
+ "hit@5": 1.0,
2177
+ "hit@10": 1.0,
2178
+ "ndcg_linear@1": 0.3977777777777778,
2179
+ "ndcg_linear@5": 0.7196994876582079,
2180
+ "ndcg_linear@10": 0.7196994876582079,
2181
+ "ndcg_exponential@1": 0.3977777777777778,
2182
+ "ndcg_exponential@5": 0.7196994876582079,
2183
+ "ndcg_exponential@10": 0.7196994876582079,
2184
+ "precision@1": 0.3977777777777778,
2185
+ "precision@5": 0.19999999999999996,
2186
+ "precision@10": 0.09999999999999998,
2187
+ "recall@1": 0.3977777777777778,
2188
+ "recall@5": 1.0,
2189
+ "recall@10": 1.0,
2190
+ "f1@1": 0.3977777777777778,
2191
+ "f1@5": 0.3333333333333334,
2192
+ "f1@10": 0.18181818181818188,
2193
+ "map@1": 0.3977777777777778,
2194
+ "map@5": 0.6258024691358024,
2195
+ "map@10": 0.6258024691358024,
2196
+ "mrr@1": 0.3977777777777778,
2197
+ "mrr@5": 0.6258024691358024,
2198
+ "mrr@10": 0.6258024691358024,
2199
+ "num_pred": 2700,
2200
+ "num_data": 2700
2201
+ },
2202
+ "NExTQA": {
2203
+ "hit@1": 0.49532928538066323,
2204
+ "hit@5": 1.0,
2205
+ "hit@10": 1.0,
2206
+ "ndcg_linear@1": 0.49532928538066323,
2207
+ "ndcg_linear@5": 0.7576205695993969,
2208
+ "ndcg_linear@10": 0.7576205695993969,
2209
+ "ndcg_exponential@1": 0.49532928538066323,
2210
+ "ndcg_exponential@5": 0.7576205695993969,
2211
+ "ndcg_exponential@10": 0.7576205695993969,
2212
+ "precision@1": 0.49532928538066323,
2213
+ "precision@5": 0.20000000000000004,
2214
+ "precision@10": 0.10000000000000002,
2215
+ "recall@1": 0.49532928538066323,
2216
+ "recall@5": 1.0,
2217
+ "recall@10": 1.0,
2218
+ "f1@1": 0.49532928538066323,
2219
+ "f1@5": 0.3333333333333333,
2220
+ "f1@10": 0.1818181818181818,
2221
+ "map@1": 0.49532928538066323,
2222
+ "map@5": 0.6775903004826405,
2223
+ "map@10": 0.6775903004826405,
2224
+ "mrr@1": 0.49532928538066323,
2225
+ "mrr@5": 0.6773567647516736,
2226
+ "mrr@10": 0.6773567647516736,
2227
+ "num_pred": 8564,
2228
+ "num_data": 8564
2229
+ },
2230
+ "EgoSchema": {
2231
+ "hit@1": 0.356,
2232
+ "hit@5": 1.0,
2233
+ "hit@10": 1.0,
2234
+ "ndcg_linear@1": 0.356,
2235
+ "ndcg_linear@5": 0.6967117751574468,
2236
+ "ndcg_linear@10": 0.6967117751574468,
2237
+ "ndcg_exponential@1": 0.356,
2238
+ "ndcg_exponential@5": 0.6967117751574468,
2239
+ "ndcg_exponential@10": 0.6967117751574468,
2240
+ "precision@1": 0.356,
2241
+ "precision@5": 0.2,
2242
+ "precision@10": 0.1,
2243
+ "recall@1": 0.356,
2244
+ "recall@5": 1.0,
2245
+ "recall@10": 1.0,
2246
+ "f1@1": 0.356,
2247
+ "f1@5": 0.3333333333333333,
2248
+ "f1@10": 0.18181818181818182,
2249
+ "map@1": 0.356,
2250
+ "map@5": 0.5960333333333333,
2251
+ "map@10": 0.5960333333333333,
2252
+ "mrr@1": 0.356,
2253
+ "mrr@5": 0.5960333333333333,
2254
+ "mrr@10": 0.5960333333333333,
2255
+ "num_pred": 500,
2256
+ "num_data": 500
2257
+ },
2258
+ "ActivityNetQA": {
2259
+ "hit@1": 0.712,
2260
+ "hit@5": 1.0,
2261
+ "hit@10": 1.0,
2262
+ "ndcg_linear@1": 0.712,
2263
+ "ndcg_linear@5": 0.8937077690285798,
2264
+ "ndcg_linear@10": 0.8937077690285798,
2265
+ "ndcg_exponential@1": 0.712,
2266
+ "ndcg_exponential@5": 0.8937077690285798,
2267
+ "ndcg_exponential@10": 0.8937077690285798,
2268
+ "precision@1": 0.712,
2269
+ "precision@5": 0.20000000000000004,
2270
+ "precision@10": 0.10000000000000002,
2271
+ "recall@1": 0.712,
2272
+ "recall@5": 1.0,
2273
+ "recall@10": 1.0,
2274
+ "f1@1": 0.712,
2275
+ "f1@5": 0.3333333333333333,
2276
+ "f1@10": 0.18181818181818182,
2277
+ "map@1": 0.712,
2278
+ "map@5": 0.856,
2279
+ "map@10": 0.856,
2280
+ "mrr@1": 0.712,
2281
+ "mrr@5": 0.856,
2282
+ "mrr@10": 0.856,
2283
+ "num_pred": 1000,
2284
+ "num_data": 1000
2285
+ }
2286
+ }
2287
+ }
2288
+ }