GPTQ quants
#5
by dazipe - opened
Hello,
Would you consider uploading GPTQ 4bit and 8bit quants for the Qwen/QwQ-32B?
The AWQ is supported on newer GPUs only. Support is particularly poor on AMD cards.
You have provided it previously. For example your GPTQ quants have the most downloads for the Qwen2.5-32B-Instruct:
https://huggingface.co/models?other=base_model:quantized:Qwen%2FQwen2.5-32B-Instruct&sort=downloads