README.md

metadata

license: gemma
datasets:
  - hks350d/commit-message-generation
  - Maxscha/commitbench
language:
  - en
base_model:
  - google/gemma-3-270m-it

Git Diff -> Commit Message (Gemma 3 270M IT + LoRA)

A small, fast model specialized to turn a git diff into a concise, English commit message. Built on top of google/gemma-3-270m-it and fine-tuned with LoRA using MLX on macOS.

What this model expects (most important)

• Input type: a unified git diff as plain text.
• Wrap the diff in a Markdown code fence labeled diff for best results:

  ```diff
  <your unified git diff here>
  

• The diff should look like the output of git diff --no-color (hunk headers like @@, +/- line prefixes, file headers, etc.).
• Keep diffs reasonably sized. The training/CLI path truncates diffs to ~3,000 characters and trains/infers with a context window of ~2,048 tokens. Extremely large diffs should be summarized or sampled.
• Language of response: English only. The system prompt enforces English output.

Chat template (Gemma 3)

The model was trained and inferred using Gemma’s chat template. Conceptually:

• system: "You are a helpful assistant that generates git commit messages. Always respond in English only. Do not use any other language."
• user: "Generate a concise and descriptive commit message for this git diff:" + the diff wrapped in ```diff fences
• assistant: single-line commit message (target)

Training data (chat format) examples were stored like:

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant that generates git commit messages. Always respond in English only. Do not use any other language."},
    {"role": "user", "content": "Generate a concise and descriptive commit message for this git diff:\n\n```diff\n<diff text>\n```"},
    {"role": "assistant", "content": "<single-line commit message>"}
  ]
}

Output

• A single-line commit subject, in English.
• The CLI post-processes the generation and returns the first non-empty line.
• Keep it concise and descriptive; optionally target ~50–72 characters where possible.

Quick usage

CLI (included in this repo)

• From a staged diff in your current repo:

python commit_msg_cli.py run --from-git --staged --adapter \
  --model google/gemma-3-270m-it \
  --adapter-path ./adapters

• From a diff file:

python commit_msg_cli.py run --diff path/to/diff.txt --adapter \
  --model google/gemma-3-270m-it \
  --adapter-path ./adapters

The CLI will wrap your diff with the expected prompt/template and return a single-line message.

Programmatic (MLX)

from mlx_lm.utils import load as mlx_load
from mlx_lm.generate import generate
from chat_template_utils import get_gemma_tokenizer, format_commit_message_prompt
from mlx_lm import sample_utils

model_name = "google/gemma-3-270m-it"
adapter_path = "./adapters"  # or a specific run dir

diff_text = """diff --git a/app.py b/app.py
index e69de29..f4c3b4a 100644
--- a/app.py
+++ b/app.py
@@ -0,0 +1,3 @@
+def add(a, b):
+    return a + b
+"""

# Load with adapter if available
model, tok = mlx_load(model_name, adapter_path=adapter_path)

# Use Gemma chat template for the prompt
tokenizer = get_gemma_tokenizer(model_name)
prompt = format_commit_message_prompt(diff_text, tokenizer, include_generation_prompt=True)

sampler = sample_utils.make_sampler(temp=0.7, top_p=0.9, top_k=64)
out = generate(model, tok, prompt=prompt, max_tokens=100, verbose=False, sampler=sampler)
print(out)

Examples

Input (user message content):

diff --git a/app.py b/app.py
index e69de29..f4c3b4a 100644
--- a/app.py
+++ b/app.py
@@ -0,0 +1,3 @@
+def add(a, b):
+    return a + b
+

Possible outputs:

• Add simple add() helper
• Implement add function
• Introduce add utility for two-number sum

Training summary

• Base model: google/gemma-3-270m-it (Gemma 3, 270M, instruction-tuned).
• Method: LoRA fine-tuning with MLX (mlx_lm lora). Prompt masking was enabled so the model learns from the assistant response.
• Data: Local JSONL converted to chat format with diffs fenced as ```diff and English, single-line commit messages as targets. In this repo, the dataset used (data/train_gpt-oss-20b.jsonl) was parsed and converted to a chat messages format. This particular set is Python-focused.
• Context/config highlights: max sequence length ~2048 tokens; diffs truncated to ~3,000 characters during preprocessing/inference to be model-friendly.

Evaluation

• The repo includes a lightweight evaluation that compares generated messages to a reference using a simple string similarity (SequenceMatcher) across multiple runs (varying the RNG seed). Results and artifacts are saved under evaluation_results/.

Limitations and risks

• Diff size sensitivity: Very large diffs may be truncated; consider summarizing large changes.
• Domain bias: Training set emphasized Python diffs; behavior may be better for Python-heavy repos.
• Hallucinations: As with any LLM, may produce generic or mismatched messages if the diff is ambiguous.
• Security: Do not feed secrets; generated text may inadvertently paraphrase sensitive context.
• Language: System prompt enforces English responses.

Intended use

• Assist developers by proposing a concise commit subject from a given git diff.
• Not a replacement for human judgment; review messages before committing.

How to format inputs yourself

If you’re not using the CLI helpers, follow this structure with the Gemma chat template:

• system: English-only instruction for commit message generation (see above)
• user: instruction + the diff in ```diff code fences
• assistant: the target single-line subject (for training) or left empty (for inference)

The repository’s format_commit_message_prompt builds the correct prompt for Gemma 3.

License and credits

• Base model: Google Gemma 3 (google/gemma-3-270m-it). Use subject to the Gemma license terms.
• Fine-tuning code: MLX and utilities in this repository. See repository license for details.