dogmatix v0.1

v0.1 — first public release. v0.2 is in progress. Known priorities for the next version: end-to-end app building (currently the weakest area), a rebuilt benchmark that is not saturated, and Django coverage. The limitations listed below are what v0.2 is aimed at.

A QLoRA fine-tune of google/gemma-4-E2B-it (~2.3B effective) into a local, Python-specialised web-application coding agent. Trained on executed, test-verified trajectories distilled from a GLM-5.2 teacher running live inside the real harness. Runs on a single 16GB GPU.

What it is for

Small, well-specified Python web work — FastAPI, Flask, SQLAlchemy — driven through an agent loop: read the files, make the edit, run the tests, fix what broke. It is meant to be fast, local and private, not to compete with frontier models on open-ended engineering.

Measured results

All figures from the current harness on a held-out suite of 156 tasks (26 families x 6 nouns), greedy decoding, executed and graded by hidden tests.

benchmark	dogmatix	untuned base
Web-app agentic pass@1	0.955	0.756
Django pass@1 (60 tasks)	0.650	not measured
Test-authoring pass@1	0.682	not measured
MBPP (general Python)	0.710	0.760

The web-app gain over the untuned base is statistically significant under a family-level permutation test (delta +0.199, p = 0.0038, 26 families) — the suite groups correlated tasks by family, so a per-task test would overstate it.

Where the fine-tuning helped, and where it did not

group	dogmatix	untuned base
fastapi	0.905	0.905
flask	1.000	0.762
multifile	1.000	0.667
sqlalchemy	0.929	0.667

Fine-tuning buys large gains on Flask (+24), multi-file editing (+33), SQLAlchemy (+26) and test-authoring (which the base cannot do at all). FastAPI is unchanged. The one regression is general Python: MBPP 0.710 vs 0.760 untuned — the usual cost of specialisation, and worth knowing if you also want a general coder.

It has also built a working Flask CRUD application end to end through the OpenAI-compatible proxy — spec in, working app out, verified by a hidden acceptance suite it never saw.

Honest limitations

Read these before using it.

It cannot reliably build a whole application in one pass. Measured 0/3 on building apps from an empty directory in a single agent loop. It succeeds materially more often when the work is decomposed into small sub-tasks, which is how the bundled harness drives it.
It cannot be trusted to judge its own work. In the end-to-end Pi build it produced a correct application alongside broken tests for it — it would have reported its own success as a failure. Use external verification.
The benchmark is saturated and partly self-similar. Two of four task groups score 1.000, and several "held-out" families closely resemble training families under noun substitution, so the effective number of independent tasks is nearer 26 than 156. Treat the headline as an upper bound.
Django is the weakest area, and the corpus contains almost no Django agentic trajectories.
Fine-tuning regressed general Python (MBPP 0.710 vs 0.760 untuned). This is a specialisation trade, not a free win.
Python only. No other languages, no JS frontends, no deployment.
Long horizons degrade. The design target is 5-10 tool-steps per loop; longer work needs decomposition.

Serving

Ships with an OpenAI-compatible and an Anthropic-compatible proxy (dogmatix/serving/). The translation layer is part of the model: it enforces a turn boundary the base chat template does not provide, without which the model emits long bursts of tool calls and never observes their results.

Training

QLoRA (4-bit NF4, r=32) on the language-model projections only, loss masked to assistant tokens, merged to bf16 on CPU. Single adapter, single run, curriculum expressed as data-sampling weights.

Provenance and caveats

Distilled from GLM-5.2 executed trajectories, filtered to test-verified successes.
The released checkpoint's selection history is recorded in docs/checkpoint-selection-release.md.
Numbers above are reproducible with scripts/benchmark.py in the repository.
The released checkpoint is step200, chosen on a held-out validation suite. Later checkpoints scored worse (step800: 0.600, step1000: 0.650), so the run over-fits after roughly 200 steps.