satreysa commited on
Commit
81af4c3
·
verified ·
1 Parent(s): f8e1493

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -29
README.md CHANGED
@@ -7,39 +7,14 @@ base_model:
7
 
8
  # mistralai/Mistral-7B-Instruct-v0.3
9
  - ## Introduction
10
- - Quantization Tool: Quark 0.6.0
11
- - OGA Model Builder: v0.5.1
12
- - Postprocess
13
  - ## Quantization Strategy
14
  - AWQ / Group 128 / Asymmetric / UINT4 Weights / FP16 activations
15
  - Excluded Layers: None
16
- ```
17
- python3 quantize_quark.py \
18
- --model_dir "$model" \
19
- --output_dir "$output_dir" \
20
- --quant_scheme w_uint4_per_group_asym \
21
- --num_calib_data 128 \
22
- --quant_algo awq \
23
- --dataset pileval_for_awq_benchmark \
24
- --seq_len 512 \
25
- --model_export quark_safetensors \
26
- --data_type float16 \
27
- --exclude_layers [] \
28
- --custom_mode awq
29
- ```
30
- - ## OGA Model Builder
31
- ```
32
- python builder.py \
33
- -i <quantized safetensor model dir> \
34
- -o <oga model output dir> \
35
- -p int4 \
36
- -e dml
37
- ```
38
- - PostProcessed to generate Hybrid Model
39
- -
40
  - ## Quick Start
41
- For quickstart, refer to hybrid-llm-artifacts_1.3.0.zip available in [RyzenAI-SW-EA](https://account.amd.com/en/member/ryzenai-sw-ea.html)
42
-
43
 
44
  #### Evaluation scores
45
  The perplexity measurement is run on the wikitext-2-raw-v1 (raw data) dataset provided by Hugging Face. Perplexity score measured for prompt length 2k is 5.7053.
 
7
 
8
  # mistralai/Mistral-7B-Instruct-v0.3
9
  - ## Introduction
10
+ This model was prepared using the AMD Quark Quantization tool, followed by necessary post-processing.
11
+
 
12
  - ## Quantization Strategy
13
  - AWQ / Group 128 / Asymmetric / UINT4 Weights / FP16 activations
14
  - Excluded Layers: None
15
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  - ## Quick Start
17
+ For quickstart, refer to [Ryzen AI doucmentation](https://ryzenai.docs.amd.com/en/latest/hybrid_oga.html)
 
18
 
19
  #### Evaluation scores
20
  The perplexity measurement is run on the wikitext-2-raw-v1 (raw data) dataset provided by Hugging Face. Perplexity score measured for prompt length 2k is 5.7053.