Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,9 @@ Please follow the license of the original model.
|
|
11 |
|
12 |
## How To Use
|
13 |
### INT4 Inference
|
14 |
-
|
|
|
|
|
15 |
```python
|
16 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
17 |
import transformers
|
@@ -121,6 +123,7 @@ Here are the key points about the company:
|
|
121 |
* **Open-Source Contribution:** DeepSeak has made significant contributions to the open-source community. They have released powerful models like **DeepSeek-Coder** (focused on code generation and programming tasks) and the weights for earlier versions of their LLMs, allowing developers and researchers worldwide
|
122 |
--------------------------------------------------
|
123 |
"""
|
|
|
124 |
|
125 |
### Generate the model
|
126 |
Mian branch is required if the model is fp8 and the device supports fp8 https://github.com/intel/auto-round
|
|
|
11 |
|
12 |
## How To Use
|
13 |
### INT4 Inference
|
14 |
+
Potential overflow/underflow issues have been observed on CUDA, primarily due to kernel limitations.
|
15 |
+
For better accuracy, we recommend deploying the model on CPU or using [our INT4 mixed version](https://huggingface.co/Intel/DeepSeek-V3.1-int4-mixed-AutoRound)
|
16 |
+
|
17 |
```python
|
18 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
19 |
import transformers
|
|
|
123 |
* **Open-Source Contribution:** DeepSeak has made significant contributions to the open-source community. They have released powerful models like **DeepSeek-Coder** (focused on code generation and programming tasks) and the weights for earlier versions of their LLMs, allowing developers and researchers worldwide
|
124 |
--------------------------------------------------
|
125 |
"""
|
126 |
+
```
|
127 |
|
128 |
### Generate the model
|
129 |
Mian branch is required if the model is fp8 and the device supports fp8 https://github.com/intel/auto-round
|