KV Cached not working in MLX

by statisticalplumber - opened about 24 hours ago

about 24 hours ago

Model takes all messages to process everytime, ignoring cache.
Other models are working as expected.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment