Add description for abbreviations
Browse files
README.md
CHANGED
|
@@ -11,8 +11,11 @@ widget:
|
|
| 11 |
## Model description
|
| 12 |
|
| 13 |
**CAMeLBERT** is a collection of BERT models pre-trained on Arabic texts with different sizes and variants.
|
|
|
|
|
|
|
| 14 |
The details are described in the paper *"[The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://arxiv.org/abs/2103.06678)."*
|
| 15 |
-
|
|
|
|
| 16 |
|
| 17 |
||Model|Variant|Size|#Word|
|
| 18 |
|-|-|:-:|-:|-:|
|
|
@@ -25,8 +28,6 @@ We release eight models with different sizes and variants as follows:
|
|
| 25 |
||`bert-base-camelbert-msa-eighth`|MSA|14GB|1.6B|
|
| 26 |
||`bert-base-camelbert-msa-sixteenth`|MSA|6GB|746M|
|
| 27 |
|
| 28 |
-
This model card describes **CAMeLBERT-MSA** (`bert-base-camelbert-msa`), a model pre-trained on the entire MSA dataset.
|
| 29 |
-
|
| 30 |
## Intended uses
|
| 31 |
You can use the released model for either masked language modeling or next sentence prediction.
|
| 32 |
However, it is mostly intended to be fine-tuned on an NLP task, such as NER, POS tagging, sentiment analysis, dialect identification, and poetry classification.
|
|
@@ -80,7 +81,7 @@ output = model(encoded_input)
|
|
| 80 |
```
|
| 81 |
|
| 82 |
## Training data
|
| 83 |
-
- MSA
|
| 84 |
- [The Arabic Gigaword Fifth Edition](https://catalog.ldc.upenn.edu/LDC2011T11)
|
| 85 |
- [Abu El-Khair Corpus](http://www.abuelkhair.net/index.php/en/arabic/abu-el-khair-corpus)
|
| 86 |
- [OSIAN corpus](https://vlo.clarin.eu/search;jsessionid=31066390B2C9E8C6304845BA79869AC1?1&q=osian)
|
|
|
|
| 11 |
## Model description
|
| 12 |
|
| 13 |
**CAMeLBERT** is a collection of BERT models pre-trained on Arabic texts with different sizes and variants.
|
| 14 |
+
We release pre-trained language models for Modern Standard Arabic (MSA), dialectal Arabic (DA), and classical Arabic (CA), in addition to a model pre-trained on a mix of the three.
|
| 15 |
+
We also provide additional models that are pre-trained on a scaled-down set of the MSA variant (half, quarter, eighth, and sixteenth).
|
| 16 |
The details are described in the paper *"[The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://arxiv.org/abs/2103.06678)."*
|
| 17 |
+
|
| 18 |
+
This model card describes **CAMeLBERT-MSA** (`bert-base-camelbert-msa`), a model pre-trained on the entire MSA dataset.
|
| 19 |
|
| 20 |
||Model|Variant|Size|#Word|
|
| 21 |
|-|-|:-:|-:|-:|
|
|
|
|
| 28 |
||`bert-base-camelbert-msa-eighth`|MSA|14GB|1.6B|
|
| 29 |
||`bert-base-camelbert-msa-sixteenth`|MSA|6GB|746M|
|
| 30 |
|
|
|
|
|
|
|
| 31 |
## Intended uses
|
| 32 |
You can use the released model for either masked language modeling or next sentence prediction.
|
| 33 |
However, it is mostly intended to be fine-tuned on an NLP task, such as NER, POS tagging, sentiment analysis, dialect identification, and poetry classification.
|
|
|
|
| 81 |
```
|
| 82 |
|
| 83 |
## Training data
|
| 84 |
+
- MSA (Modern Standard Arabic)
|
| 85 |
- [The Arabic Gigaword Fifth Edition](https://catalog.ldc.upenn.edu/LDC2011T11)
|
| 86 |
- [Abu El-Khair Corpus](http://www.abuelkhair.net/index.php/en/arabic/abu-el-khair-corpus)
|
| 87 |
- [OSIAN corpus](https://vlo.clarin.eu/search;jsessionid=31066390B2C9E8C6304845BA79869AC1?1&q=osian)
|