wenhuach commited on
Commit
760a4e0
·
verified ·
1 Parent(s): 606d084

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -11,7 +11,9 @@ Please follow the license of the original model.
11
 
12
  ## How To Use
13
  ### INT4 Inference
14
- Due to kernel issues, potential overflow/underflow has been observed on CUDA. For accuracy, it is recommended to use CPU.
 
 
15
  ```python
16
  from transformers import AutoModelForCausalLM, AutoTokenizer
17
  import transformers
@@ -121,6 +123,7 @@ Here are the key points about the company:
121
  * **Open-Source Contribution:** DeepSeak has made significant contributions to the open-source community. They have released powerful models like **DeepSeek-Coder** (focused on code generation and programming tasks) and the weights for earlier versions of their LLMs, allowing developers and researchers worldwide
122
  --------------------------------------------------
123
  """
 
124
 
125
  ### Generate the model
126
  Mian branch is required if the model is fp8 and the device supports fp8 https://github.com/intel/auto-round
 
11
 
12
  ## How To Use
13
  ### INT4 Inference
14
+ Potential overflow/underflow issues have been observed on CUDA, primarily due to kernel limitations.
15
+ For better accuracy, we recommend deploying the model on CPU or using [our INT4 mixed version](https://huggingface.co/Intel/DeepSeek-V3.1-int4-mixed-AutoRound)
16
+
17
  ```python
18
  from transformers import AutoModelForCausalLM, AutoTokenizer
19
  import transformers
 
123
  * **Open-Source Contribution:** DeepSeak has made significant contributions to the open-source community. They have released powerful models like **DeepSeek-Coder** (focused on code generation and programming tasks) and the weights for earlier versions of their LLMs, allowing developers and researchers worldwide
124
  --------------------------------------------------
125
  """
126
+ ```
127
 
128
  ### Generate the model
129
  Mian branch is required if the model is fp8 and the device supports fp8 https://github.com/intel/auto-round