metadata
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sparse-encoder
- sparse
- csr
- generated_from_trainer
- dataset_size:99000
- loss:CSRLoss
- loss:SparseMultipleNegativesRankingLoss
base_model: mixedbread-ai/mxbai-embed-large-v1
widget:
- text: >-
Saudi Arabia–United Arab Emirates relations However, the UAE and Saudi
Arabia continue to take somewhat differing stances on regional conflicts
such the Yemeni Civil War, where the UAE opposes Al-Islah, and supports
the Southern Movement, which has fought against Saudi-backed forces, and
the Syrian Civil War, where the UAE has disagreed with Saudi support for
Islamist movements.[4]
- text: >-
Economy of New Zealand New Zealand's diverse market economy has a sizable
service sector, accounting for 63% of all GDP activity in 2013.[17] Large
scale manufacturing industries include aluminium production, food
processing, metal fabrication, wood and paper products. Mining,
manufacturing, electricity, gas, water, and waste services accounted for
16.5% of GDP in 2013.[17] The primary sector continues to dominate New
Zealand's exports, despite accounting for 6.5% of GDP in 2013.[17]
- text: >-
who was the first president of indian science congress meeting held in
kolkata in 1914
- text: >-
Get Over It (Eagles song) "Get Over It" is a song by the Eagles released
as a single after a fourteen-year breakup. It was also the first song
written by bandmates Don Henley and Glenn Frey when the band reunited.
"Get Over It" was played live for the first time during their Hell Freezes
Over tour in 1994. It returned the band to the U.S. Top 40 after a
fourteen-year absence, peaking at No. 31 on the Billboard Hot 100 chart.
It also hit No. 4 on the Billboard Mainstream Rock Tracks chart. The song
was not played live by the Eagles after the "Hell Freezes Over" tour in
1994. It remains the group's last Top 40 hit in the U.S.
- text: >-
Cornelius the Centurion Cornelius (Greek: Κορνήλιος) was a Roman centurion
who is considered by Christians to be one of the first Gentiles to convert
to the faith, as related in Acts of the Apostles.
datasets:
- sentence-transformers/natural-questions
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
- row_non_zero_mean_query
- row_sparsity_mean_query
- row_non_zero_mean_corpus
- row_sparsity_mean_corpus
co2_eq_emissions:
emissions: 73.20361367491836
energy_consumed: 0.18832836896882021
source: codecarbon
training_type: fine-tuning
on_cloud: false
cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
ram_total_size: 31.777088165283203
hours_used: 0.525
hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
- name: Sparse CSR model trained on Natural Questions
results:
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO
type: NanoMSMARCO
metrics:
- type: dot_accuracy@1
value: 0.34
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.64
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.76
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.34
name: Dot Precision@1
- type: dot_precision@3
value: 0.2
name: Dot Precision@3
- type: dot_precision@5
value: 0.128
name: Dot Precision@5
- type: dot_precision@10
value: 0.07600000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.34
name: Dot Recall@1
- type: dot_recall@3
value: 0.6
name: Dot Recall@3
- type: dot_recall@5
value: 0.64
name: Dot Recall@5
- type: dot_recall@10
value: 0.76
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5461951956850831
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.4783333333333332
name: Dot Mrr@10
- type: dot_map@100
value: 0.4886114783173606
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.42
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.56
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.72
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.78
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.42
name: Dot Precision@1
- type: dot_precision@3
value: 0.18666666666666668
name: Dot Precision@3
- type: dot_precision@5
value: 0.14400000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.078
name: Dot Precision@10
- type: dot_recall@1
value: 0.42
name: Dot Recall@1
- type: dot_recall@3
value: 0.56
name: Dot Recall@3
- type: dot_recall@5
value: 0.72
name: Dot Recall@5
- type: dot_recall@10
value: 0.78
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5877208923649152
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5268333333333333
name: Dot Mrr@10
- type: dot_map@100
value: 0.5379890352202846
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.42
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.64
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.72
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.42
name: Dot Precision@1
- type: dot_precision@3
value: 0.21333333333333332
name: Dot Precision@3
- type: dot_precision@5
value: 0.14400000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08199999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.42
name: Dot Recall@1
- type: dot_recall@3
value: 0.64
name: Dot Recall@3
- type: dot_recall@5
value: 0.72
name: Dot Recall@5
- type: dot_recall@10
value: 0.82
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6170710567644069
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5518571428571428
name: Dot Mrr@10
- type: dot_map@100
value: 0.5582688466837231
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.38
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.66
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.72
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.38
name: Dot Precision@1
- type: dot_precision@3
value: 0.22
name: Dot Precision@3
- type: dot_precision@5
value: 0.14400000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08199999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.38
name: Dot Recall@1
- type: dot_recall@3
value: 0.66
name: Dot Recall@3
- type: dot_recall@5
value: 0.72
name: Dot Recall@5
- type: dot_recall@10
value: 0.82
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6054279406769176
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.536
name: Dot Mrr@10
- type: dot_map@100
value: 0.5451453839729702
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.42
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.66
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.76
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.42
name: Dot Precision@1
- type: dot_precision@3
value: 0.22
name: Dot Precision@3
- type: dot_precision@5
value: 0.15200000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08199999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.42
name: Dot Recall@1
- type: dot_recall@3
value: 0.66
name: Dot Recall@3
- type: dot_recall@5
value: 0.76
name: Dot Recall@5
- type: dot_recall@10
value: 0.82
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6230021505155314
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.559579365079365
name: Dot Mrr@10
- type: dot_map@100
value: 0.5681061401796429
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus
type: NanoNFCorpus
metrics:
- type: dot_accuracy@1
value: 0.24
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.44
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.54
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.24
name: Dot Precision@1
- type: dot_precision@3
value: 0.24666666666666665
name: Dot Precision@3
- type: dot_precision@5
value: 0.23600000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.16799999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.004669919461913613
name: Dot Recall@1
- type: dot_recall@3
value: 0.020427728422148655
name: Dot Recall@3
- type: dot_recall@5
value: 0.032225855738342704
name: Dot Recall@5
- type: dot_recall@10
value: 0.05379343324862708
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.19455316428596148
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.341
name: Dot Mrr@10
- type: dot_map@100
value: 0.06151820759407443
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.3
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.42
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.5
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.56
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.3
name: Dot Precision@1
- type: dot_precision@3
value: 0.26
name: Dot Precision@3
- type: dot_precision@5
value: 0.26
name: Dot Precision@5
- type: dot_precision@10
value: 0.19799999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.029542626120334554
name: Dot Recall@1
- type: dot_recall@3
value: 0.044652693879235976
name: Dot Recall@3
- type: dot_recall@5
value: 0.06028547009613851
name: Dot Recall@5
- type: dot_recall@10
value: 0.07755036888620734
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.2366214062101772
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.37585714285714283
name: Dot Mrr@10
- type: dot_map@100
value: 0.10067936596479826
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.38
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.54
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.6
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.64
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.38
name: Dot Precision@1
- type: dot_precision@3
value: 0.33333333333333326
name: Dot Precision@3
- type: dot_precision@5
value: 0.3
name: Dot Precision@5
- type: dot_precision@10
value: 0.24
name: Dot Precision@10
- type: dot_recall@1
value: 0.03179014108396205
name: Dot Recall@1
- type: dot_recall@3
value: 0.06318690279554356
name: Dot Recall@3
- type: dot_recall@5
value: 0.07796877283624153
name: Dot Recall@5
- type: dot_recall@10
value: 0.1114799837292121
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.2931027237585236
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.4740238095238095
name: Dot Mrr@10
- type: dot_map@100
value: 0.1230254488501482
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.38
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.56
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.62
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.38
name: Dot Precision@1
- type: dot_precision@3
value: 0.3466666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.30000000000000004
name: Dot Precision@5
- type: dot_precision@10
value: 0.25
name: Dot Precision@10
- type: dot_recall@1
value: 0.04457964347980141
name: Dot Recall@1
- type: dot_recall@3
value: 0.0790728975013626
name: Dot Recall@3
- type: dot_recall@5
value: 0.09009883722070462
name: Dot Recall@5
- type: dot_recall@10
value: 0.13307439776648308
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.30763421801409163
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.48096825396825393
name: Dot Mrr@10
- type: dot_map@100
value: 0.1488310784840863
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.36
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.58
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.62
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.68
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.36
name: Dot Precision@1
- type: dot_precision@3
value: 0.3399999999999999
name: Dot Precision@3
- type: dot_precision@5
value: 0.304
name: Dot Precision@5
- type: dot_precision@10
value: 0.25
name: Dot Precision@10
- type: dot_recall@1
value: 0.04421600711616505
name: Dot Recall@1
- type: dot_recall@3
value: 0.07676067662123591
name: Dot Recall@3
- type: dot_recall@5
value: 0.09031933665344426
name: Dot Recall@5
- type: dot_recall@10
value: 0.1280526321040542
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3058687263048634
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.47285714285714286
name: Dot Mrr@10
- type: dot_map@100
value: 0.14946123566548253
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ
type: NanoNQ
metrics:
- type: dot_accuracy@1
value: 0.24
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.3
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.4
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.58
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.24
name: Dot Precision@1
- type: dot_precision@3
value: 0.09999999999999998
name: Dot Precision@3
- type: dot_precision@5
value: 0.08
name: Dot Precision@5
- type: dot_precision@10
value: 0.05800000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.23
name: Dot Recall@1
- type: dot_recall@3
value: 0.28
name: Dot Recall@3
- type: dot_recall@5
value: 0.37
name: Dot Recall@5
- type: dot_recall@10
value: 0.53
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.35458104021173625
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.31448412698412687
name: Dot Mrr@10
- type: dot_map@100
value: 0.31511521921352853
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.34
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.48
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.84
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.34
name: Dot Precision@1
- type: dot_precision@3
value: 0.15999999999999998
name: Dot Precision@3
- type: dot_precision@5
value: 0.14
name: Dot Precision@5
- type: dot_precision@10
value: 0.08399999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.33
name: Dot Recall@1
- type: dot_recall@3
value: 0.47
name: Dot Recall@3
- type: dot_recall@5
value: 0.66
name: Dot Recall@5
- type: dot_recall@10
value: 0.78
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5379040930401837
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.4738174603174602
name: Dot Mrr@10
- type: dot_map@100
value: 0.4633356301557521
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.46
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.66
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.74
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.84
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.46
name: Dot Precision@1
- type: dot_precision@3
value: 0.22
name: Dot Precision@3
- type: dot_precision@5
value: 0.14800000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08399999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.43
name: Dot Recall@1
- type: dot_recall@3
value: 0.61
name: Dot Recall@3
- type: dot_recall@5
value: 0.68
name: Dot Recall@5
- type: dot_recall@10
value: 0.78
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6051159774116176
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5759365079365079
name: Dot Mrr@10
- type: dot_map@100
value: 0.5492196930626584
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.56
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.76
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.56
name: Dot Precision@1
- type: dot_precision@3
value: 0.23333333333333336
name: Dot Precision@3
- type: dot_precision@5
value: 0.15600000000000003
name: Dot Precision@5
- type: dot_precision@10
value: 0.08599999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.53
name: Dot Recall@1
- type: dot_recall@3
value: 0.65
name: Dot Recall@3
- type: dot_recall@5
value: 0.72
name: Dot Recall@5
- type: dot_recall@10
value: 0.77
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6573308671626625
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6343571428571428
name: Dot Mrr@10
- type: dot_map@100
value: 0.6230919790541427
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.6
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.66
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.76
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.6
name: Dot Precision@1
- type: dot_precision@3
value: 0.22666666666666668
name: Dot Precision@3
- type: dot_precision@5
value: 0.15600000000000003
name: Dot Precision@5
- type: dot_precision@10
value: 0.08399999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.57
name: Dot Recall@1
- type: dot_recall@3
value: 0.63
name: Dot Recall@3
- type: dot_recall@5
value: 0.72
name: Dot Recall@5
- type: dot_recall@10
value: 0.76
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6666113633916609
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6562222222222222
name: Dot Mrr@10
- type: dot_map@100
value: 0.6391801455199051
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean
type: NanoBEIR_mean
metrics:
- type: dot_accuracy@1
value: 0.2733333333333334
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.4466666666666667
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.6266666666666666
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.2733333333333334
name: Dot Precision@1
- type: dot_precision@3
value: 0.1822222222222222
name: Dot Precision@3
- type: dot_precision@5
value: 0.148
name: Dot Precision@5
- type: dot_precision@10
value: 0.10066666666666667
name: Dot Precision@10
- type: dot_recall@1
value: 0.19155663982063786
name: Dot Recall@1
- type: dot_recall@3
value: 0.3001425761407162
name: Dot Recall@3
- type: dot_recall@5
value: 0.34740861857944755
name: Dot Recall@5
- type: dot_recall@10
value: 0.447931144416209
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.36510980006092697
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.3779391534391534
name: Dot Mrr@10
- type: dot_map@100
value: 0.28841496837498787
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.35333333333333333
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.48666666666666664
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.64
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7266666666666667
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.35333333333333333
name: Dot Precision@1
- type: dot_precision@3
value: 0.20222222222222222
name: Dot Precision@3
- type: dot_precision@5
value: 0.18133333333333335
name: Dot Precision@5
- type: dot_precision@10
value: 0.12
name: Dot Precision@10
- type: dot_recall@1
value: 0.2598475420401115
name: Dot Recall@1
- type: dot_recall@3
value: 0.358217564626412
name: Dot Recall@3
- type: dot_recall@5
value: 0.48009515669871283
name: Dot Recall@5
- type: dot_recall@10
value: 0.5458501229620691
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.45408213053842533
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.45883597883597876
name: Dot Mrr@10
- type: dot_map@100
value: 0.3673346771136117
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.42
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6133333333333334
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.6866666666666665
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7666666666666666
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.42
name: Dot Precision@1
- type: dot_precision@3
value: 0.25555555555555554
name: Dot Precision@3
- type: dot_precision@5
value: 0.19733333333333336
name: Dot Precision@5
- type: dot_precision@10
value: 0.1353333333333333
name: Dot Precision@10
- type: dot_recall@1
value: 0.29393004702798736
name: Dot Recall@1
- type: dot_recall@3
value: 0.43772896759851454
name: Dot Recall@3
- type: dot_recall@5
value: 0.4926562576120805
name: Dot Recall@5
- type: dot_recall@10
value: 0.5704933279097374
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5050965859781827
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5339391534391534
name: Dot Mrr@10
- type: dot_map@100
value: 0.41017132953217655
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.44
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6333333333333334
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.6999999999999998
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7733333333333334
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.44
name: Dot Precision@1
- type: dot_precision@3
value: 0.26666666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.20000000000000004
name: Dot Precision@5
- type: dot_precision@10
value: 0.1393333333333333
name: Dot Precision@10
- type: dot_recall@1
value: 0.31819321449326715
name: Dot Recall@1
- type: dot_recall@3
value: 0.4630242991671209
name: Dot Recall@3
- type: dot_recall@5
value: 0.5100329457402348
name: Dot Recall@5
- type: dot_recall@10
value: 0.5743581325888277
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5234643419512239
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5504417989417989
name: Dot Mrr@10
- type: dot_map@100
value: 0.4390228138370664
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.5731554160125589
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.7366405023547882
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.80138147566719
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8599686028257456
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5731554160125589
name: Dot Precision@1
- type: dot_precision@3
value: 0.3453898482469911
name: Dot Precision@3
- type: dot_precision@5
value: 0.26412558869701724
name: Dot Precision@5
- type: dot_precision@10
value: 0.17818838304552587
name: Dot Precision@10
- type: dot_recall@1
value: 0.3483653457738811
name: Dot Recall@1
- type: dot_recall@3
value: 0.5056041615261917
name: Dot Recall@3
- type: dot_recall@5
value: 0.5666121938396027
name: Dot Recall@5
- type: dot_recall@10
value: 0.6365903026311346
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6044636061819838
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6701240562158929
name: Dot Mrr@10
- type: dot_map@100
value: 0.532208950979869
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoClimateFEVER
type: NanoClimateFEVER
metrics:
- type: dot_accuracy@1
value: 0.3
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.44
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.58
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.66
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.3
name: Dot Precision@1
- type: dot_precision@3
value: 0.1733333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.152
name: Dot Precision@5
- type: dot_precision@10
value: 0.096
name: Dot Precision@10
- type: dot_recall@1
value: 0.14666666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.24
name: Dot Recall@3
- type: dot_recall@5
value: 0.31666666666666665
name: Dot Recall@5
- type: dot_recall@10
value: 0.379
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3188914956894916
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.40419047619047616
name: Dot Mrr@10
- type: dot_map@100
value: 0.26051323127424997
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoDBPedia
type: NanoDBPedia
metrics:
- type: dot_accuracy@1
value: 0.74
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.86
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.94
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.74
name: Dot Precision@1
- type: dot_precision@3
value: 0.64
name: Dot Precision@3
- type: dot_precision@5
value: 0.56
name: Dot Precision@5
- type: dot_precision@10
value: 0.45399999999999996
name: Dot Precision@10
- type: dot_recall@1
value: 0.07917649980287217
name: Dot Recall@1
- type: dot_recall@3
value: 0.17349972000383132
name: Dot Recall@3
- type: dot_recall@5
value: 0.24092496180273404
name: Dot Recall@5
- type: dot_recall@10
value: 0.35114625789901227
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5896892964374201
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8141904761904764
name: Dot Mrr@10
- type: dot_map@100
value: 0.43868920344880336
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFEVER
type: NanoFEVER
metrics:
- type: dot_accuracy@1
value: 0.78
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.88
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.94
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.78
name: Dot Precision@1
- type: dot_precision@3
value: 0.30666666666666664
name: Dot Precision@3
- type: dot_precision@5
value: 0.18799999999999997
name: Dot Precision@5
- type: dot_precision@10
value: 0.09799999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.7266666666666666
name: Dot Recall@1
- type: dot_recall@3
value: 0.8466666666666666
name: Dot Recall@3
- type: dot_recall@5
value: 0.8666666666666666
name: Dot Recall@5
- type: dot_recall@10
value: 0.9066666666666666
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.8324754718241318
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8367460317460318
name: Dot Mrr@10
- type: dot_map@100
value: 0.8012874752784108
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFiQA2018
type: NanoFiQA2018
metrics:
- type: dot_accuracy@1
value: 0.54
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.62
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.66
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.72
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.54
name: Dot Precision@1
- type: dot_precision@3
value: 0.3
name: Dot Precision@3
- type: dot_precision@5
value: 0.20800000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.118
name: Dot Precision@10
- type: dot_recall@1
value: 0.29257936507936505
name: Dot Recall@1
- type: dot_recall@3
value: 0.4266587301587301
name: Dot Recall@3
- type: dot_recall@5
value: 0.47401587301587306
name: Dot Recall@5
- type: dot_recall@10
value: 0.5415952380952381
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.49829838655315356
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.599
name: Dot Mrr@10
- type: dot_map@100
value: 0.45314978299438174
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoHotpotQA
type: NanoHotpotQA
metrics:
- type: dot_accuracy@1
value: 0.84
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.96
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.98
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.98
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.84
name: Dot Precision@1
- type: dot_precision@3
value: 0.5333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.3439999999999999
name: Dot Precision@5
- type: dot_precision@10
value: 0.17599999999999993
name: Dot Precision@10
- type: dot_recall@1
value: 0.42
name: Dot Recall@1
- type: dot_recall@3
value: 0.8
name: Dot Recall@3
- type: dot_recall@5
value: 0.86
name: Dot Recall@5
- type: dot_recall@10
value: 0.88
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.831808180844114
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8983333333333333
name: Dot Mrr@10
- type: dot_map@100
value: 0.7782796284246765
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoQuoraRetrieval
type: NanoQuoraRetrieval
metrics:
- type: dot_accuracy@1
value: 0.9
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.98
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.98
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 1
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.9
name: Dot Precision@1
- type: dot_precision@3
value: 0.4133333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.25999999999999995
name: Dot Precision@5
- type: dot_precision@10
value: 0.13799999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.7906666666666666
name: Dot Recall@1
- type: dot_recall@3
value: 0.9520000000000001
name: Dot Recall@3
- type: dot_recall@5
value: 0.966
name: Dot Recall@5
- type: dot_recall@10
value: 0.9966666666666666
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.9495440482890076
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.94
name: Dot Mrr@10
- type: dot_map@100
value: 0.9292555555555555
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSCIDOCS
type: NanoSCIDOCS
metrics:
- type: dot_accuracy@1
value: 0.5
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.66
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.76
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.84
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5
name: Dot Precision@1
- type: dot_precision@3
value: 0.33333333333333326
name: Dot Precision@3
- type: dot_precision@5
value: 0.29200000000000004
name: Dot Precision@5
- type: dot_precision@10
value: 0.20400000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.10666666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.2096666666666667
name: Dot Recall@3
- type: dot_recall@5
value: 0.3016666666666667
name: Dot Recall@5
- type: dot_recall@10
value: 0.4186666666666667
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4055677447150387
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6097142857142857
name: Dot Mrr@10
- type: dot_map@100
value: 0.3297751386475111
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoArguAna
type: NanoArguAna
metrics:
- type: dot_accuracy@1
value: 0.34
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.78
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.86
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.98
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.34
name: Dot Precision@1
- type: dot_precision@3
value: 0.26
name: Dot Precision@3
- type: dot_precision@5
value: 0.17199999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.09799999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.34
name: Dot Recall@1
- type: dot_recall@3
value: 0.78
name: Dot Recall@3
- type: dot_recall@5
value: 0.86
name: Dot Recall@5
- type: dot_recall@10
value: 0.98
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6735247359369816
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.574126984126984
name: Dot Mrr@10
- type: dot_map@100
value: 0.5746532999164579
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSciFact
type: NanoSciFact
metrics:
- type: dot_accuracy@1
value: 0.58
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.76
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.84
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.58
name: Dot Precision@1
- type: dot_precision@3
value: 0.24
name: Dot Precision@3
- type: dot_precision@5
value: 0.16399999999999998
name: Dot Precision@5
- type: dot_precision@10
value: 0.09599999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.555
name: Dot Recall@1
- type: dot_recall@3
value: 0.67
name: Dot Recall@3
- type: dot_recall@5
value: 0.745
name: Dot Recall@5
- type: dot_recall@10
value: 0.84
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6982128840882104
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6553015873015873
name: Dot Mrr@10
- type: dot_map@100
value: 0.6562051918669566
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoTouche2020
type: NanoTouche2020
metrics:
- type: dot_accuracy@1
value: 0.5510204081632653
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.8163265306122449
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.8979591836734694
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.9795918367346939
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5510204081632653
name: Dot Precision@1
- type: dot_precision@3
value: 0.5034013605442177
name: Dot Precision@3
- type: dot_precision@5
value: 0.4816326530612246
name: Dot Precision@5
- type: dot_precision@10
value: 0.4224489795918367
name: Dot Precision@10
- type: dot_recall@1
value: 0.03711095639538641
name: Dot Recall@1
- type: dot_recall@3
value: 0.10760163972336141
name: Dot Recall@3
- type: dot_recall@5
value: 0.16469834844278258
name: Dot Recall@5
- type: dot_recall@10
value: 0.2738798061064456
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4645323957761827
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6913508260447035
name: Dot Mrr@10
- type: dot_map@100
value: 0.3401603339662621
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
Sparse CSR model trained on Natural Questions
This is a CSR Sparse Encoder model finetuned from mixedbread-ai/mxbai-embed-large-v1 on the natural-questions dataset using the sentence-transformers library. It maps sentences & paragraphs to a 4096-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
Model Details
Model Description
- Model Type: CSR Sparse Encoder
- Base model: mixedbread-ai/mxbai-embed-large-v1
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 4096 dimensions
- Similarity Function: Dot Product
- Training Dataset:
- Language: en
- License: apache-2.0
Model Sources
Full Model Architecture
SparseEncoder(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): CSRSparsity({'input_dim': 1024, 'hidden_dim': 4096, 'k': 256, 'k_aux': 512, 'normalize': False, 'dead_threshold': 30})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SparseEncoder
model = SparseEncoder("tomaarsen/csr-mxbai-embed-large-v1-nq-gemma5")
sentences = [
'who is cornelius in the book of acts',
'Cornelius the Centurion Cornelius (Greek: Κορνήλιος) was a Roman centurion who is considered by Christians to be one of the first Gentiles to convert to the faith, as related in Acts of the Apostles.',
"Joe Ranft Ranft reunited with Lasseter when he was hired by Pixar in 1991 as their head of story.[1] There he worked on all of their films produced up to 2006; this included Toy Story (for which he received an Academy Award nomination) and A Bug's Life, as the co-story writer and others as story supervisor. His final film was Cars. He also voiced characters in many of the films, including Heimlich the caterpillar in A Bug's Life, Wheezy the penguin in Toy Story 2, and Jacques the shrimp in Finding Nemo.[1]",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
Evaluation
Metrics
Sparse Information Retrieval
- Datasets:
NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020
- Evaluated with
SparseInformationRetrievalEvaluator
| Metric |
NanoMSMARCO |
NanoNFCorpus |
NanoNQ |
NanoClimateFEVER |
NanoDBPedia |
NanoFEVER |
NanoFiQA2018 |
NanoHotpotQA |
NanoQuoraRetrieval |
NanoSCIDOCS |
NanoArguAna |
NanoSciFact |
NanoTouche2020 |
| dot_accuracy@1 |
0.42 |
0.36 |
0.6 |
0.3 |
0.74 |
0.78 |
0.54 |
0.84 |
0.9 |
0.5 |
0.34 |
0.58 |
0.551 |
| dot_accuracy@3 |
0.66 |
0.58 |
0.66 |
0.44 |
0.86 |
0.88 |
0.62 |
0.96 |
0.98 |
0.66 |
0.78 |
0.68 |
0.8163 |
| dot_accuracy@5 |
0.76 |
0.62 |
0.76 |
0.58 |
0.9 |
0.9 |
0.66 |
0.98 |
0.98 |
0.76 |
0.86 |
0.76 |
0.898 |
| dot_accuracy@10 |
0.82 |
0.68 |
0.8 |
0.66 |
0.94 |
0.94 |
0.72 |
0.98 |
1.0 |
0.84 |
0.98 |
0.84 |
0.9796 |
| dot_precision@1 |
0.42 |
0.36 |
0.6 |
0.3 |
0.74 |
0.78 |
0.54 |
0.84 |
0.9 |
0.5 |
0.34 |
0.58 |
0.551 |
| dot_precision@3 |
0.22 |
0.34 |
0.2267 |
0.1733 |
0.64 |
0.3067 |
0.3 |
0.5333 |
0.4133 |
0.3333 |
0.26 |
0.24 |
0.5034 |
| dot_precision@5 |
0.152 |
0.304 |
0.156 |
0.152 |
0.56 |
0.188 |
0.208 |
0.344 |
0.26 |
0.292 |
0.172 |
0.164 |
0.4816 |
| dot_precision@10 |
0.082 |
0.25 |
0.084 |
0.096 |
0.454 |
0.098 |
0.118 |
0.176 |
0.138 |
0.204 |
0.098 |
0.096 |
0.4224 |
| dot_recall@1 |
0.42 |
0.0442 |
0.57 |
0.1467 |
0.0792 |
0.7267 |
0.2926 |
0.42 |
0.7907 |
0.1067 |
0.34 |
0.555 |
0.0371 |
| dot_recall@3 |
0.66 |
0.0768 |
0.63 |
0.24 |
0.1735 |
0.8467 |
0.4267 |
0.8 |
0.952 |
0.2097 |
0.78 |
0.67 |
0.1076 |
| dot_recall@5 |
0.76 |
0.0903 |
0.72 |
0.3167 |
0.2409 |
0.8667 |
0.474 |
0.86 |
0.966 |
0.3017 |
0.86 |
0.745 |
0.1647 |
| dot_recall@10 |
0.82 |
0.1281 |
0.76 |
0.379 |
0.3511 |
0.9067 |
0.5416 |
0.88 |
0.9967 |
0.4187 |
0.98 |
0.84 |
0.2739 |
| dot_ndcg@10 |
0.623 |
0.3059 |
0.6666 |
0.3189 |
0.5897 |
0.8325 |
0.4983 |
0.8318 |
0.9495 |
0.4056 |
0.6735 |
0.6982 |
0.4645 |
| dot_mrr@10 |
0.5596 |
0.4729 |
0.6562 |
0.4042 |
0.8142 |
0.8367 |
0.599 |
0.8983 |
0.94 |
0.6097 |
0.5741 |
0.6553 |
0.6914 |
| dot_map@100 |
0.5681 |
0.1495 |
0.6392 |
0.2605 |
0.4387 |
0.8013 |
0.4531 |
0.7783 |
0.9293 |
0.3298 |
0.5747 |
0.6562 |
0.3402 |
| row_non_zero_mean_query |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
| row_sparsity_mean_query |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
| row_non_zero_mean_corpus |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
| row_sparsity_mean_corpus |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.2733 |
| dot_accuracy@3 |
0.4467 |
| dot_accuracy@5 |
0.52 |
| dot_accuracy@10 |
0.6267 |
| dot_precision@1 |
0.2733 |
| dot_precision@3 |
0.1822 |
| dot_precision@5 |
0.148 |
| dot_precision@10 |
0.1007 |
| dot_recall@1 |
0.1916 |
| dot_recall@3 |
0.3001 |
| dot_recall@5 |
0.3474 |
| dot_recall@10 |
0.4479 |
| dot_ndcg@10 |
0.3651 |
| dot_mrr@10 |
0.3779 |
| dot_map@100 |
0.2884 |
| row_non_zero_mean_query |
32.0 |
| row_sparsity_mean_query |
0.9922 |
| row_non_zero_mean_corpus |
32.0 |
| row_sparsity_mean_corpus |
0.9922 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.3533 |
| dot_accuracy@3 |
0.4867 |
| dot_accuracy@5 |
0.64 |
| dot_accuracy@10 |
0.7267 |
| dot_precision@1 |
0.3533 |
| dot_precision@3 |
0.2022 |
| dot_precision@5 |
0.1813 |
| dot_precision@10 |
0.12 |
| dot_recall@1 |
0.2598 |
| dot_recall@3 |
0.3582 |
| dot_recall@5 |
0.4801 |
| dot_recall@10 |
0.5459 |
| dot_ndcg@10 |
0.4541 |
| dot_mrr@10 |
0.4588 |
| dot_map@100 |
0.3673 |
| row_non_zero_mean_query |
64.0 |
| row_sparsity_mean_query |
0.9844 |
| row_non_zero_mean_corpus |
64.0 |
| row_sparsity_mean_corpus |
0.9844 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.42 |
| dot_accuracy@3 |
0.6133 |
| dot_accuracy@5 |
0.6867 |
| dot_accuracy@10 |
0.7667 |
| dot_precision@1 |
0.42 |
| dot_precision@3 |
0.2556 |
| dot_precision@5 |
0.1973 |
| dot_precision@10 |
0.1353 |
| dot_recall@1 |
0.2939 |
| dot_recall@3 |
0.4377 |
| dot_recall@5 |
0.4927 |
| dot_recall@10 |
0.5705 |
| dot_ndcg@10 |
0.5051 |
| dot_mrr@10 |
0.5339 |
| dot_map@100 |
0.4102 |
| row_non_zero_mean_query |
128.0 |
| row_sparsity_mean_query |
0.9688 |
| row_non_zero_mean_corpus |
128.0 |
| row_sparsity_mean_corpus |
0.9688 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.44 |
| dot_accuracy@3 |
0.6333 |
| dot_accuracy@5 |
0.7 |
| dot_accuracy@10 |
0.7733 |
| dot_precision@1 |
0.44 |
| dot_precision@3 |
0.2667 |
| dot_precision@5 |
0.2 |
| dot_precision@10 |
0.1393 |
| dot_recall@1 |
0.3182 |
| dot_recall@3 |
0.463 |
| dot_recall@5 |
0.51 |
| dot_recall@10 |
0.5744 |
| dot_ndcg@10 |
0.5235 |
| dot_mrr@10 |
0.5504 |
| dot_map@100 |
0.439 |
| row_non_zero_mean_query |
256.0 |
| row_sparsity_mean_query |
0.9375 |
| row_non_zero_mean_corpus |
256.0 |
| row_sparsity_mean_corpus |
0.9375 |
Sparse Nano BEIR
- Dataset:
NanoBEIR_mean
- Evaluated with
SparseNanoBEIREvaluator with these parameters:{
"dataset_names": [
"climatefever",
"dbpedia",
"fever",
"fiqa2018",
"hotpotqa",
"msmarco",
"nfcorpus",
"nq",
"quoraretrieval",
"scidocs",
"arguana",
"scifact",
"touche2020"
]
}
| Metric |
Value |
| dot_accuracy@1 |
0.5732 |
| dot_accuracy@3 |
0.7366 |
| dot_accuracy@5 |
0.8014 |
| dot_accuracy@10 |
0.86 |
| dot_precision@1 |
0.5732 |
| dot_precision@3 |
0.3454 |
| dot_precision@5 |
0.2641 |
| dot_precision@10 |
0.1782 |
| dot_recall@1 |
0.3484 |
| dot_recall@3 |
0.5056 |
| dot_recall@5 |
0.5666 |
| dot_recall@10 |
0.6366 |
| dot_ndcg@10 |
0.6045 |
| dot_mrr@10 |
0.6701 |
| dot_map@100 |
0.5322 |
| row_non_zero_mean_query |
256.0 |
| row_sparsity_mean_query |
0.9375 |
| row_non_zero_mean_corpus |
256.0 |
| row_sparsity_mean_corpus |
0.9375 |
Training Details
Training Dataset
natural-questions
- Dataset: natural-questions at f9e894e
- Size: 99,000 training samples
- Columns:
query and answer
- Approximate statistics based on the first 1000 samples:
|
query |
answer |
| type |
string |
string |
| details |
- min: 10 tokens
- mean: 11.71 tokens
- max: 26 tokens
|
- min: 4 tokens
- mean: 131.81 tokens
- max: 450 tokens
|
- Samples:
| query |
answer |
who played the father in papa don't preach |
Alex McArthur Alex McArthur (born March 6, 1957) is an American actor. |
where was the location of the battle of hastings |
Battle of Hastings The Battle of Hastings[a] was fought on 14 October 1066 between the Norman-French army of William, the Duke of Normandy, and an English army under the Anglo-Saxon King Harold Godwinson, beginning the Norman conquest of England. It took place approximately 7 miles (11 kilometres) northwest of Hastings, close to the present-day town of Battle, East Sussex, and was a decisive Norman victory. |
how many puppies can a dog give birth to |
Canine reproduction The largest litter size to date was set by a Neapolitan Mastiff in Manea, Cambridgeshire, UK on November 29, 2004; the litter was 24 puppies.[22] |
- Loss:
CSRLoss with these parameters:{
"beta": 0.1,
"gamma": 5,
"loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')"
}
Evaluation Dataset
natural-questions
- Dataset: natural-questions at f9e894e
- Size: 1,000 evaluation samples
- Columns:
query and answer
- Approximate statistics based on the first 1000 samples:
|
query |
answer |
| type |
string |
string |
| details |
- min: 10 tokens
- mean: 11.69 tokens
- max: 23 tokens
|
- min: 15 tokens
- mean: 134.01 tokens
- max: 512 tokens
|
- Samples:
| query |
answer |
where is the tiber river located in italy |
Tiber The Tiber (/ˈtaɪbər/, Latin: Tiberis,[1] Italian: Tevere [ˈteːvere])[2] is the third-longest river in Italy, rising in the Apennine Mountains in Emilia-Romagna and flowing 406 kilometres (252 mi) through Tuscany, Umbria and Lazio, where it is joined by the river Aniene, to the Tyrrhenian Sea, between Ostia and Fiumicino.[3] It drains a basin estimated at 17,375 square kilometres (6,709 sq mi). The river has achieved lasting fame as the main watercourse of the city of Rome, founded on its eastern banks. |
what kind of car does jay gatsby drive |
Jay Gatsby At the Buchanan home, Jordan Baker, Nick, Jay, and the Buchanans decide to visit New York City. Tom borrows Gatsby's yellow Rolls Royce to drive up to the city. On the way to New York City, Tom makes a detour at a gas station in "the Valley of Ashes", a run-down part of Long Island. The owner, George Wilson, shares his concern that his wife, Myrtle, may be having an affair. This unnerves Tom, who has been having an affair with Myrtle, and he leaves in a hurry. |
who sings if i can dream about you |
I Can Dream About You "I Can Dream About You" is a song performed by American singer Dan Hartman on the soundtrack album of the film Streets of Fire. Released in 1984 as a single from the soundtrack, and included on Hartman's album I Can Dream About You, it reached number 6 on the Billboard Hot 100.[1] |
- Loss:
CSRLoss with these parameters:{
"beta": 0.1,
"gamma": 5,
"loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')"
}
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: stepsper_device_train_batch_size: 64per_device_eval_batch_size: 64learning_rate: 4e-05num_train_epochs: 1bf16: Trueload_best_model_at_end: Truebatch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: stepsprediction_loss_only: Trueper_device_train_batch_size: 64per_device_eval_batch_size: 64per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 4e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 1max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.0warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional
Training Logs
| Epoch |
Step |
Training Loss |
Validation Loss |
NanoMSMARCO_dot_ndcg@10 |
NanoNFCorpus_dot_ndcg@10 |
NanoNQ_dot_ndcg@10 |
NanoBEIR_mean_dot_ndcg@10 |
NanoClimateFEVER_dot_ndcg@10 |
NanoDBPedia_dot_ndcg@10 |
NanoFEVER_dot_ndcg@10 |
NanoFiQA2018_dot_ndcg@10 |
NanoHotpotQA_dot_ndcg@10 |
NanoQuoraRetrieval_dot_ndcg@10 |
NanoSCIDOCS_dot_ndcg@10 |
NanoArguAna_dot_ndcg@10 |
NanoSciFact_dot_ndcg@10 |
NanoTouche2020_dot_ndcg@10 |
| 0.0646 |
100 |
0.599 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.1293 |
200 |
0.69 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.1939 |
300 |
0.61 |
0.6100 |
0.6357 |
0.2858 |
0.6522 |
0.5246 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.2586 |
400 |
0.7066 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.3232 |
500 |
0.6641 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.3878 |
600 |
0.7556 |
0.5275 |
0.6150 |
0.3067 |
0.6487 |
0.5235 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.4525 |
700 |
0.664 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.5171 |
800 |
0.5407 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.5818 |
900 |
0.63 |
0.4654 |
0.623 |
0.3055 |
0.6666 |
0.5317 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.6464 |
1000 |
0.5951 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.7111 |
1100 |
0.6147 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.7757 |
1200 |
0.7111 |
0.5087 |
0.6125 |
0.3061 |
0.6757 |
0.5314 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.8403 |
1300 |
0.6415 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.9050 |
1400 |
0.592 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.9696 |
1500 |
0.5953 |
0.5013 |
0.6054 |
0.3076 |
0.6573 |
0.5235 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| -1 |
-1 |
- |
- |
0.6230 |
0.3059 |
0.6666 |
0.6045 |
0.3189 |
0.5897 |
0.8325 |
0.4983 |
0.8318 |
0.9495 |
0.4056 |
0.6735 |
0.6982 |
0.4645 |
- The bold row denotes the saved checkpoint.
Environmental Impact
Carbon emissions were measured using CodeCarbon.
- Energy Consumed: 0.188 kWh
- Carbon Emitted: 0.073 kg of CO2
- Hours Used: 0.525 hours
Training Hardware
- On Cloud: No
- GPU Model: 1 x NVIDIA GeForce RTX 3090
- CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
- RAM Size: 31.78 GB
Framework Versions
- Python: 3.11.6
- Sentence Transformers: 4.2.0.dev0
- Transformers: 4.49.0
- PyTorch: 2.6.0+cu124
- Accelerate: 1.5.1
- Datasets: 2.21.0
- Tokenizers: 0.21.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
CSRLoss
@misc{wen2025matryoshkarevisitingsparsecoding,
title={Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation},
author={Tiansheng Wen and Yifei Wang and Zequn Zeng and Zhong Peng and Yudi Su and Xinyang Liu and Bo Chen and Hongwei Liu and Stefanie Jegelka and Chenyu You},
year={2025},
eprint={2503.01776},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2503.01776},
}
SparseMultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
