GGUF quantized version of Medical Gemma-3N for ultra-fast local inference with Ollama. Optimized for offline emergency medical assistance on consumer hardware without requiring Python dependencies. Base Model: unsloth/gemma-3n-E4B-it
| File | Size | Description | Quality |
|---|---|---|---|
medical-gemma-3n-4b.gguf |
13.7GB | Full precision GGUF | Maximum |
Note: This is a high-precision GGUF conversion. For smaller sizes, you can quantize further using llama.cpp tools.
Download it:
wget https://huggingface.co/ericrisco/medical-gemma-3n-4b-gguf/resolve/main/medical-gemma-3n-4b.gguf
Create a new directory:
mkdir -p ~/ollama/models/medical-gemma-3n-4b
cd ~/ollama/models/medical-gemma-3n-4b
And add a file called Modelfile with this content:
FROM ./medical-gemma-3n-4b.gguf
# You can adjust these parameters if needed
PARAMETER temperature 0.7
PARAMETER top_k 40
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
Now build the model:
ollama create medical-gemma-3n-4b -f Modelfile
Once created, simply run:
ollama run medical-gemma-3n-4b
This will give you an interactive interface to talk to your model.
# Start interactive session
ollama run medical-gemma-3n-4b
# Example conversation:
>>> What are the signs of a stroke?
The main signs of a stroke can be ...
@misc{medical_gemma_gguf,
title={Medical Gemma-3N GGUF: Quantized Emergency Medical Assistant},
author={Eric Risco},
year={2025},
url={https://huggingface.co/ericrisco/medical-gemma-3n-lora-gguf},
note={GGUF quantized model for offline medical assistance}
}
This model is released under the Gemma License. See LICENSE for details.
🚀 Ultra-fast offline medical assistance - Deploy anywhere with zero dependencies