AIDC-AI
/

Ovis1.6-Gemma2-9B-GPTQ-Int4

Image-Text-to-Text

text-generation

4-bit precision

Model card Files Files and versions

TryingHard commited on Nov 11, 2024

Commit

1210ec3

·

verified ·

1 Parent(s): da09c52

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -198,6 +198,7 @@ model = OvisGemma2GPTQForCausalLM.from_pretrained(
     multimodal_max_length=8192,
     trust_remote_code=True
 ).cuda()
 print(f"Model Loaded!")

     multimodal_max_length=8192,
     trust_remote_code=True
 ).cuda()
+model.model.llm.model.config.use_cache = False
 print(f"Model Loaded!")