mlx-community/medgemma-27b-text-it-8bit does not stop generating output
#1
by
sjug
- opened
On the plus side the output of the 8bit quant seems plausibly correct.
The downside is that gets to <end_of_turn>thought and continues generating over and over with no end.
Once again I don't think that mlx-lm supports MedGemma yet.
Any idea about this. Trying the 4bit version without much success.