Update README.md
Browse files
README.md
CHANGED
|
@@ -41,18 +41,13 @@ Via pip: `pip install llm-rs`
|
|
| 41 |
|
| 42 |
#### Run inference
|
| 43 |
```python
|
| 44 |
-
from llm_rs import
|
| 45 |
-
from huggingface_hub import hf_hub_download
|
| 46 |
|
| 47 |
-
#
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
#Load the model
|
| 51 |
-
model = Mpt("mpt-7b-q4_0-ggjt.bin")
|
| 52 |
|
| 53 |
#Generate
|
| 54 |
print(model.generate("The meaning of life is"))
|
| 55 |
-
```
|
| 56 |
|
| 57 |
### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
|
| 58 |
|
|
|
|
| 41 |
|
| 42 |
#### Run inference
|
| 43 |
```python
|
| 44 |
+
from llm_rs import AutoModel
|
|
|
|
| 45 |
|
| 46 |
+
#Load the model, define any model you like from the list above as the `model_file`
|
| 47 |
+
model = AutoModel.from_pretrained("Rustformers/mpt-7b-ggml",model_file="mpt-7b-q4_0-ggjt.bin")
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
#Generate
|
| 50 |
print(model.generate("The meaning of life is"))
|
|
|
|
| 51 |
|
| 52 |
### Rust via [Rustformers/llm](https://github.com/rustformers/llm):
|
| 53 |
|