karimouda commited on
Commit
b6f2890
·
1 Parent(s): 18ef3dc

Update speed FAQ

Browse files
Files changed (1) hide show
  1. src/about.py +3 -1
src/about.py CHANGED
@@ -90,7 +90,9 @@ https://huggingface.co/datasets/silma-ai/arabic-broad-benchmark#scoring-rules
90
 
91
  * Speed shows how fast the model was during testing, using the "words per second" metric.
92
  * The score is calculated by dividing the number of words generated by the model during the entire test by the time taken (in seconds) for the model to complete testing.
93
- * Please note that we use the same GPU (A100) and a batch size of 1 for all models (in the same size category) to ensure a fair comparison.
 
 
94
 
95
  ---
96
 
 
90
 
91
  * Speed shows how fast the model was during testing, using the "words per second" metric.
92
  * The score is calculated by dividing the number of words generated by the model during the entire test by the time taken (in seconds) for the model to complete testing.
93
+ * Please note that we use the same GPU (A100) and a batch size of 1 for all Hugging Face models to ensure a fair comparison. Models above 15B are split across multiple GPUs.
94
+ * Each model should only be compared to other models in its size category.
95
+ * API or closed models can't be compared to open models, only to other API models, since they are not hosted on our infrastructure.
96
 
97
  ---
98