Gemma-3-4b-Darija

A fine-tuned version of Google's Gemma-3-4b model specialized for Moroccan Darija language translation. This model can translate between English and Darija, providing both Arabic script and Latin transliteration.

Model Description

Gemma-3-4b-Darija is a fine-tuned version of the Google Gemma-3-4b instruction-tuned model, specifically trained to handle translations and language tasks in Moroccan Darija. The model maintains the reasoning capabilities of the base model while adding specialized knowledge of the Darija dialect.

• Developed by: Yamemaru
• Base model: unsloth/gemma-3-4b-it
• Language: English and Moroccan Darija (Arabic script and Latin transliteration)
• License: Apache 2.0
• Training Dataset: atlasia/Atlaset

Intended Uses

• Translation between English and Moroccan Darija
• Generating responses in both Arabic script and Latin transliteration
• Language learning assistance for Darija
• Cultural exchange and communication support

Usage Examples

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "Yamemaru/gemma-3-4b-Darija"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,  # Use float16 for efficiency
    device_map="auto"
)

# Create messages
messages = [{
    "role": "user",
    "content": [{"type": "text", "text": "How do you say 'Hello, how are you?' in Moroccan Darija?"}]
}]

# Apply chat template for generation
text = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True
)

# Generate response
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=128,
    temperature=1.0,
    top_p=0.95,
    top_k=64
)

# Print the response
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hardware Requirements

• Minimum GPU: NVIDIA T4 (16GB VRAM)
• This model requires approximately 15GB of GPU memory for inference
• For quantized inference (4-bit), 8GB VRAM may be sufficient

Limitations

• The model is specialized for Darija and may not perform as well on general tasks as the base model
• Coverage of dialectal variations within Darija may be limited to the examples in the training data
• The model may sometimes mix Modern Standard Arabic with Darija in its responses
• Quality of translations may vary based on complexity and context
• Latin transliterations follow common conventions but may not be standardized

Training Details

This model was fine-tuned using the Unsloth library with QLoRA techniques on the atlasia/Atlaset dataset. Training specifications:

License

This model is subject to the license of the original Gemma-3-4B model.

• Finetuned from model : unsloth/gemma-3-4b-it-unsloth-bnb-4bit