How can I do quantization on my own?

by rusauron - opened Jul 1

Jul 1

Sorry to bother you, but how do you quantify new types of models? Using llama.cpp ? I would be very grateful for the link to the instructions on how to do it yourself under windows.

YarvixPA

QuantStack org Jul 15

•

edited Jul 15

Here you have a guide https://github.com/city96/ComfyUI-GGUF/tree/main/tools

YarvixPA changed discussion status to closed Jul 16

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment