Update README.md

README.md

@@ -222,6 +222,8 @@ lm_eval --model hf --model_args pretrained=$MODEL --tasks hellaswag --device cud
 ```
 </details>
 
+
+
 # Peak Memory Usage
 
 ## Results
@@ -232,6 +234,7 @@ lm_eval --model hf --model_args pretrained=$MODEL --tasks hellaswag --device cud
 | Peak Memory (GB) | 16.47          | 6.27 (62% reduction)           |
 
 
+
 <details>
 <summary> Reproduce Peak Memory Usage Results </summary>
 
@@ -279,6 +282,8 @@ print(f"Peak Memory Usage: {mem:.02f} GB")
 
 </details>
 
+
+
 # Model Performance
 
 Our int4wo is only optimized for batch size 1, so expect some slowdown with larger batch sizes, we expect this to be used in local server deployment for single or a few users where the decode tokens per second will matters more than the time to first token.
@@ -293,6 +298,8 @@ Our int4wo is only optimized for batch size 1, so expect some slowdown with larg
 Note the result of latency (benchmark_latency) is in seconds, and serving (benchmark_serving) is in number of requests per second.
 Int4 weight only is optimized for batch size 1 and short input and output token length, please stay tuned for models optimized for larger batch sizes or longer token length.
 
+
+
 <details>
 <summary> Reproduce Model Performance Results </summary>
 
@@ -367,6 +374,8 @@ python benchmarks/benchmark_serving.py --backend vllm --dataset-name sharegpt --
 ```
 </details>
 
+
+
 # Disclaimer
 PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.