metadata
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sparse-encoder
- sparse
- csr
- generated_from_trainer
- dataset_size:99000
- loss:CSRLoss
- loss:SparseMultipleNegativesRankingLoss
base_model: mixedbread-ai/mxbai-embed-large-v1
widget:
- text: >-
Saudi Arabia–United Arab Emirates relations However, the UAE and Saudi
Arabia continue to take somewhat differing stances on regional conflicts
such the Yemeni Civil War, where the UAE opposes Al-Islah, and supports
the Southern Movement, which has fought against Saudi-backed forces, and
the Syrian Civil War, where the UAE has disagreed with Saudi support for
Islamist movements.[4]
- text: >-
Economy of New Zealand New Zealand's diverse market economy has a sizable
service sector, accounting for 63% of all GDP activity in 2013.[17] Large
scale manufacturing industries include aluminium production, food
processing, metal fabrication, wood and paper products. Mining,
manufacturing, electricity, gas, water, and waste services accounted for
16.5% of GDP in 2013.[17] The primary sector continues to dominate New
Zealand's exports, despite accounting for 6.5% of GDP in 2013.[17]
- text: >-
who was the first president of indian science congress meeting held in
kolkata in 1914
- text: >-
Get Over It (Eagles song) "Get Over It" is a song by the Eagles released
as a single after a fourteen-year breakup. It was also the first song
written by bandmates Don Henley and Glenn Frey when the band reunited.
"Get Over It" was played live for the first time during their Hell Freezes
Over tour in 1994. It returned the band to the U.S. Top 40 after a
fourteen-year absence, peaking at No. 31 on the Billboard Hot 100 chart.
It also hit No. 4 on the Billboard Mainstream Rock Tracks chart. The song
was not played live by the Eagles after the "Hell Freezes Over" tour in
1994. It remains the group's last Top 40 hit in the U.S.
- text: >-
Cornelius the Centurion Cornelius (Greek: Κορνήλιος) was a Roman centurion
who is considered by Christians to be one of the first Gentiles to convert
to the faith, as related in Acts of the Apostles.
datasets:
- sentence-transformers/natural-questions
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
- row_non_zero_mean_query
- row_sparsity_mean_query
- row_non_zero_mean_corpus
- row_sparsity_mean_corpus
co2_eq_emissions:
emissions: 78.63547133575128
energy_consumed: 0.20230271862699775
source: codecarbon
training_type: fine-tuning
on_cloud: false
cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
ram_total_size: 31.777088165283203
hours_used: 0.571
hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
- name: Sparse CSR model trained on Natural Questions
results:
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO
type: NanoMSMARCO
metrics:
- type: dot_accuracy@1
value: 0.34
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.44
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.62
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.34
name: Dot Precision@1
- type: dot_precision@3
value: 0.14666666666666667
name: Dot Precision@3
- type: dot_precision@5
value: 0.12400000000000003
name: Dot Precision@5
- type: dot_precision@10
value: 0.08199999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.34
name: Dot Recall@1
- type: dot_recall@3
value: 0.44
name: Dot Recall@3
- type: dot_recall@5
value: 0.62
name: Dot Recall@5
- type: dot_recall@10
value: 0.82
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.535047397862425
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.4492380952380952
name: Dot Mrr@10
- type: dot_map@100
value: 0.4565956812862131
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.4
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.64
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.74
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.4
name: Dot Precision@1
- type: dot_precision@3
value: 0.21333333333333332
name: Dot Precision@3
- type: dot_precision@5
value: 0.14800000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08199999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.4
name: Dot Recall@1
- type: dot_recall@3
value: 0.64
name: Dot Recall@3
- type: dot_recall@5
value: 0.74
name: Dot Recall@5
- type: dot_recall@10
value: 0.82
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6142058022889539
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5477142857142856
name: Dot Mrr@10
- type: dot_map@100
value: 0.5535645073071618
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.36
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.72
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.8
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.36
name: Dot Precision@1
- type: dot_precision@3
value: 0.24
name: Dot Precision@3
- type: dot_precision@5
value: 0.16
name: Dot Precision@5
- type: dot_precision@10
value: 0.08
name: Dot Precision@10
- type: dot_recall@1
value: 0.36
name: Dot Recall@1
- type: dot_recall@3
value: 0.72
name: Dot Recall@3
- type: dot_recall@5
value: 0.8
name: Dot Recall@5
- type: dot_recall@10
value: 0.8
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6119801006837546
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5479999999999999
name: Dot Mrr@10
- type: dot_map@100
value: 0.5570329635790349
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.38
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.74
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.84
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.38
name: Dot Precision@1
- type: dot_precision@3
value: 0.22666666666666668
name: Dot Precision@3
- type: dot_precision@5
value: 0.14800000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08399999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.38
name: Dot Recall@1
- type: dot_recall@3
value: 0.68
name: Dot Recall@3
- type: dot_recall@5
value: 0.74
name: Dot Recall@5
- type: dot_recall@10
value: 0.84
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6202495574521795
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5495
name: Dot Mrr@10
- type: dot_map@100
value: 0.5567587644744507
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.4
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.76
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.4
name: Dot Precision@1
- type: dot_precision@3
value: 0.22666666666666668
name: Dot Precision@3
- type: dot_precision@5
value: 0.15200000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08199999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.4
name: Dot Recall@1
- type: dot_recall@3
value: 0.68
name: Dot Recall@3
- type: dot_recall@5
value: 0.76
name: Dot Recall@5
- type: dot_recall@10
value: 0.82
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6233479483972318
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5590238095238095
name: Dot Mrr@10
- type: dot_map@100
value: 0.5667471833817065
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus
type: NanoNFCorpus
metrics:
- type: dot_accuracy@1
value: 0.22
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.3
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.36
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.52
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.22
name: Dot Precision@1
- type: dot_precision@3
value: 0.16666666666666663
name: Dot Precision@3
- type: dot_precision@5
value: 0.156
name: Dot Precision@5
- type: dot_precision@10
value: 0.15
name: Dot Precision@10
- type: dot_recall@1
value: 0.005369382143489658
name: Dot Recall@1
- type: dot_recall@3
value: 0.016195110222025074
name: Dot Recall@3
- type: dot_recall@5
value: 0.049293570620457035
name: Dot Recall@5
- type: dot_recall@10
value: 0.0806937671045514
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.17174320910928226
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.2927619047619048
name: Dot Mrr@10
- type: dot_map@100
value: 0.05298975181660711
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.26
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.4
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.46
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.56
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.26
name: Dot Precision@1
- type: dot_precision@3
value: 0.2333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.21599999999999994
name: Dot Precision@5
- type: dot_precision@10
value: 0.18
name: Dot Precision@10
- type: dot_recall@1
value: 0.010097102114744272
name: Dot Recall@1
- type: dot_recall@3
value: 0.04537644219647232
name: Dot Recall@3
- type: dot_recall@5
value: 0.06148760758910991
name: Dot Recall@5
- type: dot_recall@10
value: 0.09415095559842784
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.2096821639525137
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.34343650793650793
name: Dot Mrr@10
- type: dot_map@100
value: 0.08064284502822883
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.34
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.48
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.58
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.34
name: Dot Precision@1
- type: dot_precision@3
value: 0.3133333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.27599999999999997
name: Dot Precision@5
- type: dot_precision@10
value: 0.23
name: Dot Precision@10
- type: dot_recall@1
value: 0.03101859044799731
name: Dot Recall@1
- type: dot_recall@3
value: 0.06237480359765744
name: Dot Recall@3
- type: dot_recall@5
value: 0.07386821785513752
name: Dot Recall@5
- type: dot_recall@10
value: 0.10186854211536649
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.27455891665154974
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.42166666666666663
name: Dot Mrr@10
- type: dot_map@100
value: 0.11672912090576673
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.3
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.46
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.64
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.74
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.3
name: Dot Precision@1
- type: dot_precision@3
value: 0.3
name: Dot Precision@3
- type: dot_precision@5
value: 0.324
name: Dot Precision@5
- type: dot_precision@10
value: 0.28600000000000003
name: Dot Precision@10
- type: dot_recall@1
value: 0.010179819259573217
name: Dot Recall@1
- type: dot_recall@3
value: 0.04444946823515787
name: Dot Recall@3
- type: dot_recall@5
value: 0.07791010802255334
name: Dot Recall@5
- type: dot_recall@10
value: 0.13377621691836752
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3108609159740967
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.43744444444444447
name: Dot Mrr@10
- type: dot_map@100
value: 0.12265426034977883
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.42
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.56
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.6
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.68
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.42
name: Dot Precision@1
- type: dot_precision@3
value: 0.35999999999999993
name: Dot Precision@3
- type: dot_precision@5
value: 0.32
name: Dot Precision@5
- type: dot_precision@10
value: 0.27
name: Dot Precision@10
- type: dot_recall@1
value: 0.04635628984780851
name: Dot Recall@1
- type: dot_recall@3
value: 0.07762856181796872
name: Dot Recall@3
- type: dot_recall@5
value: 0.09496420727524445
name: Dot Recall@5
- type: dot_recall@10
value: 0.12650888877020955
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3261739681282223
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5003888888888889
name: Dot Mrr@10
- type: dot_map@100
value: 0.15272488982108906
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ
type: NanoNQ
metrics:
- type: dot_accuracy@1
value: 0.28
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.5
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.58
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.68
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.28
name: Dot Precision@1
- type: dot_precision@3
value: 0.16666666666666669
name: Dot Precision@3
- type: dot_precision@5
value: 0.11599999999999999
name: Dot Precision@5
- type: dot_precision@10
value: 0.07
name: Dot Precision@10
- type: dot_recall@1
value: 0.28
name: Dot Recall@1
- type: dot_recall@3
value: 0.47
name: Dot Recall@3
- type: dot_recall@5
value: 0.55
name: Dot Recall@5
- type: dot_recall@10
value: 0.64
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.45947191204401955
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.40702380952380957
name: Dot Mrr@10
- type: dot_map@100
value: 0.40647141879184173
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.32
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.62
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.76
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.32
name: Dot Precision@1
- type: dot_precision@3
value: 0.20666666666666667
name: Dot Precision@3
- type: dot_precision@5
value: 0.14
name: Dot Precision@5
- type: dot_precision@10
value: 0.07800000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.32
name: Dot Recall@1
- type: dot_recall@3
value: 0.59
name: Dot Recall@3
- type: dot_recall@5
value: 0.65
name: Dot Recall@5
- type: dot_recall@10
value: 0.72
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5338423179297352
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.47974603174603175
name: Dot Mrr@10
- type: dot_map@100
value: 0.4773890418843979
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.5
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.7
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.72
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.78
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5
name: Dot Precision@1
- type: dot_precision@3
value: 0.23333333333333336
name: Dot Precision@3
- type: dot_precision@5
value: 0.14800000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08199999999999999
name: Dot Precision@10
- type: dot_recall@1
value: 0.49
name: Dot Recall@1
- type: dot_recall@3
value: 0.65
name: Dot Recall@3
- type: dot_recall@5
value: 0.68
name: Dot Recall@5
- type: dot_recall@10
value: 0.74
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6242982941698777
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5989682539682539
name: Dot Mrr@10
- type: dot_map@100
value: 0.5901794633844323
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.48
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.72
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.86
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.48
name: Dot Precision@1
- type: dot_precision@3
value: 0.22666666666666668
name: Dot Precision@3
- type: dot_precision@5
value: 0.14800000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.092
name: Dot Precision@10
- type: dot_recall@1
value: 0.47
name: Dot Recall@1
- type: dot_recall@3
value: 0.64
name: Dot Recall@3
- type: dot_recall@5
value: 0.69
name: Dot Recall@5
- type: dot_recall@10
value: 0.82
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6403993438837419
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5924126984126983
name: Dot Mrr@10
- type: dot_map@100
value: 0.5839678374146947
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.48
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.72
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.76
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.84
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.48
name: Dot Precision@1
- type: dot_precision@3
value: 0.24666666666666665
name: Dot Precision@3
- type: dot_precision@5
value: 0.15600000000000003
name: Dot Precision@5
- type: dot_precision@10
value: 0.08999999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.47
name: Dot Recall@1
- type: dot_recall@3
value: 0.68
name: Dot Recall@3
- type: dot_recall@5
value: 0.71
name: Dot Recall@5
- type: dot_recall@10
value: 0.8
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6448325805638914
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6067142857142857
name: Dot Mrr@10
- type: dot_map@100
value: 0.5961039318128456
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean
type: NanoBEIR_mean
metrics:
- type: dot_accuracy@1
value: 0.28
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.41333333333333333
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.6733333333333333
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.28
name: Dot Precision@1
- type: dot_precision@3
value: 0.16
name: Dot Precision@3
- type: dot_precision@5
value: 0.132
name: Dot Precision@5
- type: dot_precision@10
value: 0.10066666666666667
name: Dot Precision@10
- type: dot_recall@1
value: 0.20845646071449656
name: Dot Recall@1
- type: dot_recall@3
value: 0.3087317034073417
name: Dot Recall@3
- type: dot_recall@5
value: 0.406431190206819
name: Dot Recall@5
- type: dot_recall@10
value: 0.5135645890348505
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.38875417300524223
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.38300793650793646
name: Dot Mrr@10
- type: dot_map@100
value: 0.3053522839648873
name: Dot Map@100
- type: row_non_zero_mean_query
value: 32
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9921875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 32
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9921875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.32666666666666666
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.5533333333333333
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.6333333333333333
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7133333333333333
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.32666666666666666
name: Dot Precision@1
- type: dot_precision@3
value: 0.21777777777777776
name: Dot Precision@3
- type: dot_precision@5
value: 0.168
name: Dot Precision@5
- type: dot_precision@10
value: 0.11333333333333334
name: Dot Precision@10
- type: dot_recall@1
value: 0.24336570070491478
name: Dot Recall@1
- type: dot_recall@3
value: 0.42512548073215745
name: Dot Recall@3
- type: dot_recall@5
value: 0.4838292025297033
name: Dot Recall@5
- type: dot_recall@10
value: 0.5447169851994759
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.452576761390401
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.4569656084656084
name: Dot Mrr@10
- type: dot_map@100
value: 0.37053213140659613
name: Dot Map@100
- type: row_non_zero_mean_query
value: 64
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.984375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 64
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.984375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.39999999999999997
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6333333333333333
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.68
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7200000000000001
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.39999999999999997
name: Dot Precision@1
- type: dot_precision@3
value: 0.2622222222222222
name: Dot Precision@3
- type: dot_precision@5
value: 0.19466666666666665
name: Dot Precision@5
- type: dot_precision@10
value: 0.13066666666666668
name: Dot Precision@10
- type: dot_recall@1
value: 0.29367286348266575
name: Dot Recall@1
- type: dot_recall@3
value: 0.47745826786588585
name: Dot Recall@3
- type: dot_recall@5
value: 0.5179560726183792
name: Dot Recall@5
- type: dot_recall@10
value: 0.5472895140384555
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.503612437168394
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5228783068783068
name: Dot Mrr@10
- type: dot_map@100
value: 0.42131384928974464
name: Dot Map@100
- type: row_non_zero_mean_query
value: 128
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.96875
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 128
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.96875
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.38666666666666666
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6066666666666668
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.6999999999999998
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8133333333333334
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.38666666666666666
name: Dot Precision@1
- type: dot_precision@3
value: 0.2511111111111111
name: Dot Precision@3
- type: dot_precision@5
value: 0.2066666666666667
name: Dot Precision@5
- type: dot_precision@10
value: 0.154
name: Dot Precision@10
- type: dot_recall@1
value: 0.28672660641985775
name: Dot Recall@1
- type: dot_recall@3
value: 0.45481648941171926
name: Dot Recall@3
- type: dot_recall@5
value: 0.5026367026741845
name: Dot Recall@5
- type: dot_recall@10
value: 0.5979254056394558
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.523836605770006
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5264523809523809
name: Dot Mrr@10
- type: dot_map@100
value: 0.42112695407964146
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- type: dot_accuracy@1
value: 0.5592778649921507
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.7628571428571431
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.8106122448979591
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8722448979591836
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5592778649921507
name: Dot Precision@1
- type: dot_precision@3
value: 0.35674515960230246
name: Dot Precision@3
- type: dot_precision@5
value: 0.26938147566718995
name: Dot Precision@5
- type: dot_precision@10
value: 0.1812558869701727
name: Dot Precision@10
- type: dot_recall@1
value: 0.34109493852292166
name: Dot Recall@1
- type: dot_recall@3
value: 0.5189062733737264
name: Dot Recall@3
- type: dot_recall@5
value: 0.5724982683825325
name: Dot Recall@5
- type: dot_recall@10
value: 0.6452176942587184
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6079916454695821
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6703401734320101
name: Dot Mrr@10
- type: dot_map@100
value: 0.5307417107665151
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoClimateFEVER
type: NanoClimateFEVER
metrics:
- type: dot_accuracy@1
value: 0.28
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.48
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.56
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.64
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.28
name: Dot Precision@1
- type: dot_precision@3
value: 0.18
name: Dot Precision@3
- type: dot_precision@5
value: 0.136
name: Dot Precision@5
- type: dot_precision@10
value: 0.086
name: Dot Precision@10
- type: dot_recall@1
value: 0.115
name: Dot Recall@1
- type: dot_recall@3
value: 0.21166666666666664
name: Dot Recall@3
- type: dot_recall@5
value: 0.2756666666666666
name: Dot Recall@5
- type: dot_recall@10
value: 0.33399999999999996
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.2808719551174852
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.39607936507936503
name: Dot Mrr@10
- type: dot_map@100
value: 0.22053769794247585
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoDBPedia
type: NanoDBPedia
metrics:
- type: dot_accuracy@1
value: 0.74
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.9
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.92
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.98
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.74
name: Dot Precision@1
- type: dot_precision@3
value: 0.64
name: Dot Precision@3
- type: dot_precision@5
value: 0.5920000000000001
name: Dot Precision@5
- type: dot_precision@10
value: 0.468
name: Dot Precision@10
- type: dot_recall@1
value: 0.08983751675202471
name: Dot Recall@1
- type: dot_recall@3
value: 0.1711487813957697
name: Dot Recall@3
- type: dot_recall@5
value: 0.23824154407745554
name: Dot Recall@5
- type: dot_recall@10
value: 0.3593446163014364
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6048782764547271
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8311904761904763
name: Dot Mrr@10
- type: dot_map@100
value: 0.44329574170124053
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFEVER
type: NanoFEVER
metrics:
- type: dot_accuracy@1
value: 0.84
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.96
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.96
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.96
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.84
name: Dot Precision@1
- type: dot_precision@3
value: 0.32666666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.19999999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.102
name: Dot Precision@10
- type: dot_recall@1
value: 0.7866666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.9166666666666667
name: Dot Recall@3
- type: dot_recall@5
value: 0.9233333333333333
name: Dot Recall@5
- type: dot_recall@10
value: 0.9333333333333332
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.8812058128870981
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.89
name: Dot Mrr@10
- type: dot_map@100
value: 0.8538462377203007
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFiQA2018
type: NanoFiQA2018
metrics:
- type: dot_accuracy@1
value: 0.4
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.64
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.78
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.4
name: Dot Precision@1
- type: dot_precision@3
value: 0.29333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.22399999999999998
name: Dot Precision@5
- type: dot_precision@10
value: 0.13599999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.20724603174603173
name: Dot Recall@1
- type: dot_recall@3
value: 0.4124603174603174
name: Dot Recall@3
- type: dot_recall@5
value: 0.5158968253968254
name: Dot Recall@5
- type: dot_recall@10
value: 0.6268412698412699
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4880473026320133
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5356349206349206
name: Dot Mrr@10
- type: dot_map@100
value: 0.4061457504951077
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoHotpotQA
type: NanoHotpotQA
metrics:
- type: dot_accuracy@1
value: 0.78
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.94
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.98
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 1
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.78
name: Dot Precision@1
- type: dot_precision@3
value: 0.5266666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.33599999999999997
name: Dot Precision@5
- type: dot_precision@10
value: 0.17999999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.39
name: Dot Recall@1
- type: dot_recall@3
value: 0.79
name: Dot Recall@3
- type: dot_recall@5
value: 0.84
name: Dot Recall@5
- type: dot_recall@10
value: 0.9
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.8241120096573138
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8728571428571428
name: Dot Mrr@10
- type: dot_map@100
value: 0.7643662862369045
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoQuoraRetrieval
type: NanoQuoraRetrieval
metrics:
- type: dot_accuracy@1
value: 0.92
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.98
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 1
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 1
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.92
name: Dot Precision@1
- type: dot_precision@3
value: 0.40666666666666657
name: Dot Precision@3
- type: dot_precision@5
value: 0.25999999999999995
name: Dot Precision@5
- type: dot_precision@10
value: 0.13599999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.8073333333333333
name: Dot Recall@1
- type: dot_recall@3
value: 0.9420000000000001
name: Dot Recall@3
- type: dot_recall@5
value: 0.976
name: Dot Recall@5
- type: dot_recall@10
value: 0.9933333333333334
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.9567316042376142
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.955
name: Dot Mrr@10
- type: dot_map@100
value: 0.9393269841269841
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSCIDOCS
type: NanoSCIDOCS
metrics:
- type: dot_accuracy@1
value: 0.46
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.66
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.74
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.86
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.46
name: Dot Precision@1
- type: dot_precision@3
value: 0.34
name: Dot Precision@3
- type: dot_precision@5
value: 0.28
name: Dot Precision@5
- type: dot_precision@10
value: 0.198
name: Dot Precision@10
- type: dot_recall@1
value: 0.09766666666666665
name: Dot Recall@1
- type: dot_recall@3
value: 0.21366666666666667
name: Dot Recall@3
- type: dot_recall@5
value: 0.28966666666666663
name: Dot Recall@5
- type: dot_recall@10
value: 0.4056666666666666
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3897243669463839
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5808015873015874
name: Dot Mrr@10
- type: dot_map@100
value: 0.3103398502941357
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoArguAna
type: NanoArguAna
metrics:
- type: dot_accuracy@1
value: 0.32
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.82
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.88
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.96
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.32
name: Dot Precision@1
- type: dot_precision@3
value: 0.2733333333333334
name: Dot Precision@3
- type: dot_precision@5
value: 0.176
name: Dot Precision@5
- type: dot_precision@10
value: 0.09599999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.32
name: Dot Recall@1
- type: dot_recall@3
value: 0.82
name: Dot Recall@3
- type: dot_recall@5
value: 0.88
name: Dot Recall@5
- type: dot_recall@10
value: 0.96
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.661824665356718
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.563047619047619
name: Dot Mrr@10
- type: dot_map@100
value: 0.5655109621561234
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSciFact
type: NanoSciFact
metrics:
- type: dot_accuracy@1
value: 0.7
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.72
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.78
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.86
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.7
name: Dot Precision@1
- type: dot_precision@3
value: 0.26666666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.17199999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.09599999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.665
name: Dot Recall@1
- type: dot_recall@3
value: 0.715
name: Dot Recall@3
- type: dot_recall@5
value: 0.765
name: Dot Recall@5
- type: dot_recall@10
value: 0.85
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.7555617268006612
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.7335238095238098
name: Dot Mrr@10
- type: dot_map@100
value: 0.7269493414387032
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoTouche2020
type: NanoTouche2020
metrics:
- type: dot_accuracy@1
value: 0.5306122448979592
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.8571428571428571
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.8979591836734694
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.9591836734693877
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5306122448979592
name: Dot Precision@1
- type: dot_precision@3
value: 0.5510204081632653
name: Dot Precision@3
- type: dot_precision@5
value: 0.4979591836734694
name: Dot Precision@5
- type: dot_precision@10
value: 0.4163265306122449
name: Dot Precision@10
- type: dot_recall@1
value: 0.039127695785450424
name: Dot Recall@1
- type: dot_recall@3
value: 0.1155438931843869
name: Dot Recall@3
- type: dot_recall@5
value: 0.17370824555673137
name: Dot Recall@5
- type: dot_recall@10
value: 0.2788019171170908
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.46657917392520565
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6901603498542274
name: Dot Mrr@10
- type: dot_map@100
value: 0.35374738283707957
name: Dot Map@100
- type: row_non_zero_mean_query
value: 256
name: Row Non Zero Mean Query
- type: row_sparsity_mean_query
value: 0.9375
name: Row Sparsity Mean Query
- type: row_non_zero_mean_corpus
value: 256
name: Row Non Zero Mean Corpus
- type: row_sparsity_mean_corpus
value: 0.9375
name: Row Sparsity Mean Corpus
Sparse CSR model trained on Natural Questions
This is a CSR Sparse Encoder model finetuned from mixedbread-ai/mxbai-embed-large-v1 on the natural-questions dataset using the sentence-transformers library. It maps sentences & paragraphs to a 4096-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
Model Details
Model Description
- Model Type: CSR Sparse Encoder
- Base model: mixedbread-ai/mxbai-embed-large-v1
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 4096 dimensions
- Similarity Function: Dot Product
- Training Dataset:
- Language: en
- License: apache-2.0
Model Sources
Full Model Architecture
SparseEncoder(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): CSRSparsity({'input_dim': 1024, 'hidden_dim': 4096, 'k': 256, 'k_aux': 512, 'normalize': False, 'dead_threshold': 30})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SparseEncoder
model = SparseEncoder("tomaarsen/csr-mxbai-embed-large-v1-nq")
sentences = [
'who is cornelius in the book of acts',
'Cornelius the Centurion Cornelius (Greek: Κορνήλιος) was a Roman centurion who is considered by Christians to be one of the first Gentiles to convert to the faith, as related in Acts of the Apostles.',
"Joe Ranft Ranft reunited with Lasseter when he was hired by Pixar in 1991 as their head of story.[1] There he worked on all of their films produced up to 2006; this included Toy Story (for which he received an Academy Award nomination) and A Bug's Life, as the co-story writer and others as story supervisor. His final film was Cars. He also voiced characters in many of the films, including Heimlich the caterpillar in A Bug's Life, Wheezy the penguin in Toy Story 2, and Jacques the shrimp in Finding Nemo.[1]",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
Evaluation
Metrics
Sparse Information Retrieval
- Datasets:
NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020
- Evaluated with
SparseInformationRetrievalEvaluator
| Metric |
NanoMSMARCO |
NanoNFCorpus |
NanoNQ |
NanoClimateFEVER |
NanoDBPedia |
NanoFEVER |
NanoFiQA2018 |
NanoHotpotQA |
NanoQuoraRetrieval |
NanoSCIDOCS |
NanoArguAna |
NanoSciFact |
NanoTouche2020 |
| dot_accuracy@1 |
0.4 |
0.42 |
0.48 |
0.28 |
0.74 |
0.84 |
0.4 |
0.78 |
0.92 |
0.46 |
0.32 |
0.7 |
0.5306 |
| dot_accuracy@3 |
0.68 |
0.56 |
0.72 |
0.48 |
0.9 |
0.96 |
0.64 |
0.94 |
0.98 |
0.66 |
0.82 |
0.72 |
0.8571 |
| dot_accuracy@5 |
0.76 |
0.6 |
0.76 |
0.56 |
0.92 |
0.96 |
0.7 |
0.98 |
1.0 |
0.74 |
0.88 |
0.78 |
0.898 |
| dot_accuracy@10 |
0.82 |
0.68 |
0.84 |
0.64 |
0.98 |
0.96 |
0.78 |
1.0 |
1.0 |
0.86 |
0.96 |
0.86 |
0.9592 |
| dot_precision@1 |
0.4 |
0.42 |
0.48 |
0.28 |
0.74 |
0.84 |
0.4 |
0.78 |
0.92 |
0.46 |
0.32 |
0.7 |
0.5306 |
| dot_precision@3 |
0.2267 |
0.36 |
0.2467 |
0.18 |
0.64 |
0.3267 |
0.2933 |
0.5267 |
0.4067 |
0.34 |
0.2733 |
0.2667 |
0.551 |
| dot_precision@5 |
0.152 |
0.32 |
0.156 |
0.136 |
0.592 |
0.2 |
0.224 |
0.336 |
0.26 |
0.28 |
0.176 |
0.172 |
0.498 |
| dot_precision@10 |
0.082 |
0.27 |
0.09 |
0.086 |
0.468 |
0.102 |
0.136 |
0.18 |
0.136 |
0.198 |
0.096 |
0.096 |
0.4163 |
| dot_recall@1 |
0.4 |
0.0464 |
0.47 |
0.115 |
0.0898 |
0.7867 |
0.2072 |
0.39 |
0.8073 |
0.0977 |
0.32 |
0.665 |
0.0391 |
| dot_recall@3 |
0.68 |
0.0776 |
0.68 |
0.2117 |
0.1711 |
0.9167 |
0.4125 |
0.79 |
0.942 |
0.2137 |
0.82 |
0.715 |
0.1155 |
| dot_recall@5 |
0.76 |
0.095 |
0.71 |
0.2757 |
0.2382 |
0.9233 |
0.5159 |
0.84 |
0.976 |
0.2897 |
0.88 |
0.765 |
0.1737 |
| dot_recall@10 |
0.82 |
0.1265 |
0.8 |
0.334 |
0.3593 |
0.9333 |
0.6268 |
0.9 |
0.9933 |
0.4057 |
0.96 |
0.85 |
0.2788 |
| dot_ndcg@10 |
0.6233 |
0.3262 |
0.6448 |
0.2809 |
0.6049 |
0.8812 |
0.488 |
0.8241 |
0.9567 |
0.3897 |
0.6618 |
0.7556 |
0.4666 |
| dot_mrr@10 |
0.559 |
0.5004 |
0.6067 |
0.3961 |
0.8312 |
0.89 |
0.5356 |
0.8729 |
0.955 |
0.5808 |
0.563 |
0.7335 |
0.6902 |
| dot_map@100 |
0.5667 |
0.1527 |
0.5961 |
0.2205 |
0.4433 |
0.8538 |
0.4061 |
0.7644 |
0.9393 |
0.3103 |
0.5655 |
0.7269 |
0.3537 |
| row_non_zero_mean_query |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
| row_sparsity_mean_query |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
| row_non_zero_mean_corpus |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
256.0 |
| row_sparsity_mean_corpus |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
0.9375 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.28 |
| dot_accuracy@3 |
0.4133 |
| dot_accuracy@5 |
0.52 |
| dot_accuracy@10 |
0.6733 |
| dot_precision@1 |
0.28 |
| dot_precision@3 |
0.16 |
| dot_precision@5 |
0.132 |
| dot_precision@10 |
0.1007 |
| dot_recall@1 |
0.2085 |
| dot_recall@3 |
0.3087 |
| dot_recall@5 |
0.4064 |
| dot_recall@10 |
0.5136 |
| dot_ndcg@10 |
0.3888 |
| dot_mrr@10 |
0.383 |
| dot_map@100 |
0.3054 |
| row_non_zero_mean_query |
32.0 |
| row_sparsity_mean_query |
0.9922 |
| row_non_zero_mean_corpus |
32.0 |
| row_sparsity_mean_corpus |
0.9922 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.3267 |
| dot_accuracy@3 |
0.5533 |
| dot_accuracy@5 |
0.6333 |
| dot_accuracy@10 |
0.7133 |
| dot_precision@1 |
0.3267 |
| dot_precision@3 |
0.2178 |
| dot_precision@5 |
0.168 |
| dot_precision@10 |
0.1133 |
| dot_recall@1 |
0.2434 |
| dot_recall@3 |
0.4251 |
| dot_recall@5 |
0.4838 |
| dot_recall@10 |
0.5447 |
| dot_ndcg@10 |
0.4526 |
| dot_mrr@10 |
0.457 |
| dot_map@100 |
0.3705 |
| row_non_zero_mean_query |
64.0 |
| row_sparsity_mean_query |
0.9844 |
| row_non_zero_mean_corpus |
64.0 |
| row_sparsity_mean_corpus |
0.9844 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.4 |
| dot_accuracy@3 |
0.6333 |
| dot_accuracy@5 |
0.68 |
| dot_accuracy@10 |
0.72 |
| dot_precision@1 |
0.4 |
| dot_precision@3 |
0.2622 |
| dot_precision@5 |
0.1947 |
| dot_precision@10 |
0.1307 |
| dot_recall@1 |
0.2937 |
| dot_recall@3 |
0.4775 |
| dot_recall@5 |
0.518 |
| dot_recall@10 |
0.5473 |
| dot_ndcg@10 |
0.5036 |
| dot_mrr@10 |
0.5229 |
| dot_map@100 |
0.4213 |
| row_non_zero_mean_query |
128.0 |
| row_sparsity_mean_query |
0.9688 |
| row_non_zero_mean_corpus |
128.0 |
| row_sparsity_mean_corpus |
0.9688 |
Sparse Nano BEIR
| Metric |
Value |
| dot_accuracy@1 |
0.3867 |
| dot_accuracy@3 |
0.6067 |
| dot_accuracy@5 |
0.7 |
| dot_accuracy@10 |
0.8133 |
| dot_precision@1 |
0.3867 |
| dot_precision@3 |
0.2511 |
| dot_precision@5 |
0.2067 |
| dot_precision@10 |
0.154 |
| dot_recall@1 |
0.2867 |
| dot_recall@3 |
0.4548 |
| dot_recall@5 |
0.5026 |
| dot_recall@10 |
0.5979 |
| dot_ndcg@10 |
0.5238 |
| dot_mrr@10 |
0.5265 |
| dot_map@100 |
0.4211 |
| row_non_zero_mean_query |
256.0 |
| row_sparsity_mean_query |
0.9375 |
| row_non_zero_mean_corpus |
256.0 |
| row_sparsity_mean_corpus |
0.9375 |
Sparse Nano BEIR
- Dataset:
NanoBEIR_mean
- Evaluated with
SparseNanoBEIREvaluator with these parameters:{
"dataset_names": [
"climatefever",
"dbpedia",
"fever",
"fiqa2018",
"hotpotqa",
"msmarco",
"nfcorpus",
"nq",
"quoraretrieval",
"scidocs",
"arguana",
"scifact",
"touche2020"
]
}
| Metric |
Value |
| dot_accuracy@1 |
0.5593 |
| dot_accuracy@3 |
0.7629 |
| dot_accuracy@5 |
0.8106 |
| dot_accuracy@10 |
0.8722 |
| dot_precision@1 |
0.5593 |
| dot_precision@3 |
0.3567 |
| dot_precision@5 |
0.2694 |
| dot_precision@10 |
0.1813 |
| dot_recall@1 |
0.3411 |
| dot_recall@3 |
0.5189 |
| dot_recall@5 |
0.5725 |
| dot_recall@10 |
0.6452 |
| dot_ndcg@10 |
0.608 |
| dot_mrr@10 |
0.6703 |
| dot_map@100 |
0.5307 |
| row_non_zero_mean_query |
256.0 |
| row_sparsity_mean_query |
0.9375 |
| row_non_zero_mean_corpus |
256.0 |
| row_sparsity_mean_corpus |
0.9375 |
Training Details
Training Dataset
natural-questions
- Dataset: natural-questions at f9e894e
- Size: 99,000 training samples
- Columns:
query and answer
- Approximate statistics based on the first 1000 samples:
|
query |
answer |
| type |
string |
string |
| details |
- min: 10 tokens
- mean: 11.71 tokens
- max: 26 tokens
|
- min: 4 tokens
- mean: 131.81 tokens
- max: 450 tokens
|
- Samples:
| query |
answer |
who played the father in papa don't preach |
Alex McArthur Alex McArthur (born March 6, 1957) is an American actor. |
where was the location of the battle of hastings |
Battle of Hastings The Battle of Hastings[a] was fought on 14 October 1066 between the Norman-French army of William, the Duke of Normandy, and an English army under the Anglo-Saxon King Harold Godwinson, beginning the Norman conquest of England. It took place approximately 7 miles (11 kilometres) northwest of Hastings, close to the present-day town of Battle, East Sussex, and was a decisive Norman victory. |
how many puppies can a dog give birth to |
Canine reproduction The largest litter size to date was set by a Neapolitan Mastiff in Manea, Cambridgeshire, UK on November 29, 2004; the litter was 24 puppies.[22] |
- Loss:
CSRLoss with these parameters:{
"beta": 0.1,
"gamma": 1.0,
"loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')"
}
Evaluation Dataset
natural-questions
- Dataset: natural-questions at f9e894e
- Size: 1,000 evaluation samples
- Columns:
query and answer
- Approximate statistics based on the first 1000 samples:
|
query |
answer |
| type |
string |
string |
| details |
- min: 10 tokens
- mean: 11.69 tokens
- max: 23 tokens
|
- min: 15 tokens
- mean: 134.01 tokens
- max: 512 tokens
|
- Samples:
| query |
answer |
where is the tiber river located in italy |
Tiber The Tiber (/ˈtaɪbər/, Latin: Tiberis,[1] Italian: Tevere [ˈteːvere])[2] is the third-longest river in Italy, rising in the Apennine Mountains in Emilia-Romagna and flowing 406 kilometres (252 mi) through Tuscany, Umbria and Lazio, where it is joined by the river Aniene, to the Tyrrhenian Sea, between Ostia and Fiumicino.[3] It drains a basin estimated at 17,375 square kilometres (6,709 sq mi). The river has achieved lasting fame as the main watercourse of the city of Rome, founded on its eastern banks. |
what kind of car does jay gatsby drive |
Jay Gatsby At the Buchanan home, Jordan Baker, Nick, Jay, and the Buchanans decide to visit New York City. Tom borrows Gatsby's yellow Rolls Royce to drive up to the city. On the way to New York City, Tom makes a detour at a gas station in "the Valley of Ashes", a run-down part of Long Island. The owner, George Wilson, shares his concern that his wife, Myrtle, may be having an affair. This unnerves Tom, who has been having an affair with Myrtle, and he leaves in a hurry. |
who sings if i can dream about you |
I Can Dream About You "I Can Dream About You" is a song performed by American singer Dan Hartman on the soundtrack album of the film Streets of Fire. Released in 1984 as a single from the soundtrack, and included on Hartman's album I Can Dream About You, it reached number 6 on the Billboard Hot 100.[1] |
- Loss:
CSRLoss with these parameters:{
"beta": 0.1,
"gamma": 1.0,
"loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')"
}
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: stepsper_device_train_batch_size: 64per_device_eval_batch_size: 64learning_rate: 4e-05num_train_epochs: 1bf16: Trueload_best_model_at_end: Truebatch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: stepsprediction_loss_only: Trueper_device_train_batch_size: 64per_device_eval_batch_size: 64per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 4e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 1max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.0warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional
Training Logs
| Epoch |
Step |
Training Loss |
Validation Loss |
NanoMSMARCO_dot_ndcg@10 |
NanoNFCorpus_dot_ndcg@10 |
NanoNQ_dot_ndcg@10 |
NanoBEIR_mean_dot_ndcg@10 |
NanoClimateFEVER_dot_ndcg@10 |
NanoDBPedia_dot_ndcg@10 |
NanoFEVER_dot_ndcg@10 |
NanoFiQA2018_dot_ndcg@10 |
NanoHotpotQA_dot_ndcg@10 |
NanoQuoraRetrieval_dot_ndcg@10 |
NanoSCIDOCS_dot_ndcg@10 |
NanoArguAna_dot_ndcg@10 |
NanoSciFact_dot_ndcg@10 |
NanoTouche2020_dot_ndcg@10 |
| 0.0646 |
100 |
0.3429 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.1293 |
200 |
0.3521 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.1939 |
300 |
0.3399 |
0.3572 |
0.6207 |
0.3281 |
0.6434 |
0.5308 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.2586 |
400 |
0.3458 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.3232 |
500 |
0.3383 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.3878 |
600 |
0.3613 |
0.3705 |
0.5998 |
0.3108 |
0.6044 |
0.5050 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.4525 |
700 |
0.3323 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.5171 |
800 |
0.316 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.5818 |
900 |
0.3336 |
0.3499 |
0.5970 |
0.3092 |
0.6616 |
0.5226 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.6464 |
1000 |
0.3161 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.7111 |
1100 |
0.3329 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.7757 |
1200 |
0.3615 |
0.3609 |
0.6036 |
0.3108 |
0.6372 |
0.5172 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.8403 |
1300 |
0.337 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.9050 |
1400 |
0.3265 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| 0.9696 |
1500 |
0.3246 |
0.3527 |
0.6202 |
0.3109 |
0.6404 |
0.5238 |
- |
- |
- |
- |
- |
- |
- |
- |
- |
- |
| -1 |
-1 |
- |
- |
0.6233 |
0.3262 |
0.6448 |
0.6080 |
0.2809 |
0.6049 |
0.8812 |
0.4880 |
0.8241 |
0.9567 |
0.3897 |
0.6618 |
0.7556 |
0.4666 |
- The bold row denotes the saved checkpoint.
Environmental Impact
Carbon emissions were measured using CodeCarbon.
- Energy Consumed: 0.202 kWh
- Carbon Emitted: 0.079 kg of CO2
- Hours Used: 0.571 hours
Training Hardware
- On Cloud: No
- GPU Model: 1 x NVIDIA GeForce RTX 3090
- CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
- RAM Size: 31.78 GB
Framework Versions
- Python: 3.11.6
- Sentence Transformers: 4.2.0.dev0
- Transformers: 4.49.0
- PyTorch: 2.6.0+cu124
- Accelerate: 1.5.1
- Datasets: 2.21.0
- Tokenizers: 0.21.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
CSRLoss
@misc{wen2025matryoshkarevisitingsparsecoding,
title={Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation},
author={Tiansheng Wen and Yifei Wang and Zequn Zeng and Zhong Peng and Yudi Su and Xinyang Liu and Bo Chen and Hongwei Liu and Stefanie Jegelka and Chenyu You},
year={2025},
eprint={2503.01776},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2503.01776},
}
SparseMultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
