Hugging Face's logo

PartAI
/
TookaBERT-Base

Fill-Mask
Transformers
PyTorch
Safetensors
Persian
bert
Model card Files Community
AliMGH commited on
Commit
172b49b
·
verified ·
1 Parent(s): 869fea1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -16
README.md CHANGED
@@ -14,9 +14,7 @@ widget:
14
  ---
15
  # Model Details
16
 
17
- TukaBERT models are a family of encoder models trained on Persian in two sizes of base and large.
18
- These Models pre-trained on over 300GB Persian data including variety of topics such as News, Blogs, Forums,
19
- Books, etc. They were pre-training with the MLM (WWM) objective using two context lengths.
20
 
21
  ## How to use
22
 
@@ -52,18 +50,18 @@ You can use this model to fine-tune it over your dataset and prepare it for your
52
  TukaBERT models are evaluated on a wide range of NLP downstream tasks, such as Sentiment Analysis (SA), Text Classification, Multiple-choice, Question Answering, and Named Entity Recognition (NER).
53
  Here are some key performance results:
54
 
55
- | Model name | DeepSentiPers (f1/acc) | MultiCoNER-v2 (f1/acc) | PQuAD (best_exact/best_f1/HasAns_exact/HasAns_f1) | FarsTail (f1/acc) | ParsiNLU-Multiple-choice (f1/acc) | ParsiNLU-Reading-comprehension (exact/f1) | ParsiNLU-QQP (f1/acc) |
56
- |------------------|------------------------|------------------------|---------------------------------------------------|-------------------|-----------------------------------|-------------------------------------------|-----------------------|
57
- | TukaBERT-large | **85.66/85.78** | **69.69/94.07** | **75.56/88.06/70.24/87.83** | **89.71/89.72** | **36.13/35.97** | **33.6/60.5** | **82.72/82.63** |
58
- | TukaBERT-base | _83.93/83.93_ | _66.23/93.3_ | _73.18_/_85.71_/_68.29_/_85.94_ | _83.26/83.41_ | 33.6/_33.81_ | 20.8/42.52 | _81.33/81.29_ |
59
- | Shiraz | 81.17/81.08 | 59.1/92.83 | 65.96/81.25/59.63/81.31 | 77.76/77.75 | _34.73/34.53_ | 17.6/39.61 | 79.68/79.51 |
60
- | ParsBERT | 80.22/80.23 | 64.91/93.23 | 71.41/84.21/66.29/84.57 | 80.89/80.94 | **35.34/35.25** | 20/39.58 | 80.15/80.07 |
61
- | XLM-V-base | _83.43/83.36_ | 58.83/92.23 | _73.26_/_85.69_/_68.21_/_85.56_ | 81.1/81.2 | **35.28/35.25** | 8/26.66 | 80.1/79.96 |
62
- | XLM-RoBERTa-base | _83.99/84.07_ | 60.38/92.49 | _73.72_/_86.24_/_68.16_/_85.8_ | 82.0/81.98 | 32.4/32.37 | 20.0/40.43 | 79.14/78.95 |
63
- | FaBERT | 82.68/82.65 | 63.89/93.01 | _72.57_/_85.39_/67.16/_85.31_ | _83.69/83.67_ | 32.47/32.37 | _27.2/48.42_ | **82.34/82.29** |
64
- | mBERT | 78.57/78.66 | 60.31/92.54 | 71.79/84.68/65.89/83.99 | _82.69/82.82_ | 33.41/33.09 | _27.2_/42.18 | 79.19/79.29 |
65
- | AriaBERT | 80.51/80.51 | 60.98/92.45 | 68.09/81.23/62.12/80.94 | 74.47/74.43 | 30.75/30.94 | 14.4/35.48 | 79.09/78.84 |
66
-
67
- \*Note because of the randomness in the fine-tuning process, results with less than 1% differences are italic together.
68
 
69
  ## How to Cite
 
14
  ---
15
  # Model Details
16
 
17
+ TukaBERT models are a family of encoder models trained on Persian in two sizes base and large. These Models pre-trained on over 300GB of Persian data including a variety of topics such as News, Blogs, Forums, Books, etc. They pre-trained with the MLM (WWM) objective using two context lengths.
 
 
18
 
19
  ## How to use
20
 
 
50
  TukaBERT models are evaluated on a wide range of NLP downstream tasks, such as Sentiment Analysis (SA), Text Classification, Multiple-choice, Question Answering, and Named Entity Recognition (NER).
51
  Here are some key performance results:
52
 
53
+ | Model name | DeepSentiPers (f1/acc) | MultiCoNER-v2 (f1/acc) | PQuAD (best_exact/best_f1/HasAns_exact/HasAns_f1) | FarsTail (f1/acc) | ParsiNLU-Multiple-choice (f1/acc) | ParsiNLU-Reading-comprehension (exact/f1) | ParsiNLU-QQP (f1/acc) |
54
+ |------------------|------------------------|------------------------|-----------------------------------------------------|--------------------|-----------------------------------|-------------------------------------------|-----------------------|
55
+ | TukaBERT-large | **85.66/85.78** | **69.69/94.07** | **75.56/88.06/70.24/87.83** | **89.71/89.72** | **36.13/35.97** | **33.6/60.5** | **82.72/82.63** |
56
+ | TukaBERT-base | <u>83.93/83.93</u> | <u>66.23/93.3</u> | <u>73.18</u>/<u>85.71</u>/<u>68.29</u>/<u>85.94</u> | <u>83.26/83.41</u> | 33.6/<u>33.81</u> | 20.8/42.52 | <u>81.33/81.29</u> |
57
+ | Shiraz | 81.17/81.08 | 59.1/92.83 | 65.96/81.25/59.63/81.31 | 77.76/77.75 | <u>34.73/34.53</u> | 17.6/39.61 | 79.68/79.51 |
58
+ | ParsBERT | 80.22/80.23 | 64.91/93.23 | 71.41/84.21/66.29/84.57 | 80.89/80.94 | **35.34/35.25** | 20/39.58 | 80.15/80.07 |
59
+ | XLM-V-base | <u>83.43/83.36</u> | 58.83/92.23 | <u>73.26</u>/<u>85.69</u>/<u>68.21</u>/<u>85.56</u> | 81.1/81.2 | **35.28/35.25** | 8/26.66 | 80.1/79.96 |
60
+ | XLM-RoBERTa-base | <u>83.99/84.07</u> | 60.38/92.49 | <u>73.72</u>/<u>86.24</u>/<u>68.16</u>/<u>85.8</u> | 82.0/81.98 | 32.4/32.37 | 20.0/40.43 | 79.14/78.95 |
61
+ | FaBERT | 82.68/82.65 | 63.89/93.01 | <u>72.57</u>/<u>85.39</u>/67.16/<u>85.31</u> | <u>83.69/83.67</u> | 32.47/32.37 | <u>27.2/48.42</u> | **82.34/82.29** |
62
+ | mBERT | 78.57/78.66 | 60.31/92.54 | 71.79/84.68/65.89/83.99 | <u>82.69/82.82</u> | 33.41/33.09 | <u>27.2</u>/42.18 | 79.19/79.29 |
63
+ | AriaBERT | 80.51/80.51 | 60.98/92.45 | 68.09/81.23/62.12/80.94 | 74.47/74.43 | 30.75/30.94 | 14.4/35.48 | 79.09/78.84 |
64
+
65
+ \*Note because of the randomness in the fine-tuning process, results with less than 1% differences are considered together.
66
 
67
  ## How to Cite