Update README.md
Browse files
README.md
CHANGED
|
@@ -28,9 +28,11 @@ _Fast Inference with Customization:_ Once trained, the ELM model architecture pe
|
|
| 28 |
## ELM-v0.1 Model Release
|
| 29 |
This repository contains code to run our ELM models. The current ELM model `elm-v0.1` (named _Rambutan_) was pre-trained (an intermediate checkpoint was used) and then instruction fine-tuned for downstream tasks.
|
| 30 |
|
| 31 |
-
|
| 32 |
- toxicity_detection (jigsaw dataset)
|
| 33 |
|
|
|
|
|
|
|
| 34 |
## Setup ELM
|
| 35 |
### Download ELM repo
|
| 36 |
```bash
|
|
|
|
| 28 |
## ELM-v0.1 Model Release
|
| 29 |
This repository contains code to run our ELM models. The current ELM model `elm-v0.1` (named _Rambutan_) was pre-trained (an intermediate checkpoint was used) and then instruction fine-tuned for downstream tasks.
|
| 30 |
|
| 31 |
+
ELM models (in the `models` folder) in this repository come in three sizes (elm-1.0, elm-0.75 and elm-0.25). **All these different slices are derived from the same ELM-1.0 finetuned checkpoint** and supports the following use-case.
|
| 32 |
- toxicity_detection (jigsaw dataset)
|
| 33 |
|
| 34 |
+
**NOTE: ELM-v0.1 release is an early version finetuned from an intermediate pretrained checkpoint & without any KV caching, decoding optimizations, or quantization applied.**
|
| 35 |
+
|
| 36 |
## Setup ELM
|
| 37 |
### Download ELM repo
|
| 38 |
```bash
|