oobabooga text-generation-webui
#11
by
Lyralei
- opened
Hello, has anyone got this GGUF version working in oobabooga text-generation-webui? I got it loaded but it is buggy at the moment, sometimes repeating itself
Maybe it's not the correct chat template, Idk.
Try the ggml-model-f16.gguf version, because gpt-oss-20b itself is 4-bit quantized, and quantizing it to 4-bit again will slightly degrade the quality.