Update README.md
Browse files
README.md
CHANGED
|
@@ -82,6 +82,13 @@ Encoder:
|
|
| 82 |
|
| 83 |
## How to Get Started with the Model
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
```python
|
| 86 |
from transformers import AutoProcessor, VisionEncoderDecoderModel
|
| 87 |
import requests
|
|
@@ -110,6 +117,9 @@ loss = outputs.loss
|
|
| 110 |
generated_ids = model.generate(pixel_values)
|
| 111 |
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 112 |
```
|
|
|
|
|
|
|
|
|
|
| 113 |
|
| 114 |
[More Information Needed]
|
| 115 |
|
|
|
|
| 82 |
|
| 83 |
## How to Get Started with the Model
|
| 84 |
|
| 85 |
+
|
| 86 |
+
### VisionEncoderDecoderModel
|
| 87 |
+
#### As a vision encoder model :
|
| 88 |
+
|
| 89 |
+
the tensors are combined into the original mistral model so it can be accessed by intaciating the correct model which is the VisionEncoderDecoderModel
|
| 90 |
+
|
| 91 |
+
|
| 92 |
```python
|
| 93 |
from transformers import AutoProcessor, VisionEncoderDecoderModel
|
| 94 |
import requests
|
|
|
|
| 117 |
generated_ids = model.generate(pixel_values)
|
| 118 |
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 119 |
```
|
| 120 |
+
### As a standard LLM:
|
| 121 |
+
|
| 122 |
+
it can still also be used as a normal AutoModelForCausalLM or MistralModelForCausalLM !
|
| 123 |
|
| 124 |
[More Information Needed]
|
| 125 |
|