Update speed FAQ
Browse files- src/about.py +3 -1
src/about.py
CHANGED
@@ -90,7 +90,9 @@ https://huggingface.co/datasets/silma-ai/arabic-broad-benchmark#scoring-rules
|
|
90 |
|
91 |
* Speed shows how fast the model was during testing, using the "words per second" metric.
|
92 |
* The score is calculated by dividing the number of words generated by the model during the entire test by the time taken (in seconds) for the model to complete testing.
|
93 |
-
* Please note that we use the same GPU (A100) and a batch size of 1 for all
|
|
|
|
|
94 |
|
95 |
---
|
96 |
|
|
|
90 |
|
91 |
* Speed shows how fast the model was during testing, using the "words per second" metric.
|
92 |
* The score is calculated by dividing the number of words generated by the model during the entire test by the time taken (in seconds) for the model to complete testing.
|
93 |
+
* Please note that we use the same GPU (A100) and a batch size of 1 for all Hugging Face models to ensure a fair comparison. Models above 15B are split across multiple GPUs.
|
94 |
+
* Each model should only be compared to other models in its size category.
|
95 |
+
* API or closed models can't be compared to open models, only to other API models, since they are not hosted on our infrastructure.
|
96 |
|
97 |
---
|
98 |
|