Quantized version?

#2
by muratowski - opened

It would be great if we had some quantized version of it like GGUF or even fp8

AWQ would be awesome !!!!!

can be done? or compressed tensors in W4A16_ASYM

Sign up or log in to comment