reneeice/editlens-qwen3-4b-repro

EditLens-Qwen3-4B — Continuous AI-Edit Detection

Score how much a piece of text was edited by AI — on a smooth 0-to-1 scale, not just yes/no.

Most AI-text detectors give you a binary verdict: human or machine. Real writing isn't binary. People run their own drafts through an LLM to fix grammar, tighten phrasing, change tone, or rewrite whole paragraphs — and a single yes/no label can't tell light proofreading apart from a full rewrite. EditLens-Qwen3-4B predicts a continuous AI-edit score in [0, 1], so you can measure how much of a text is AI's work, not just whether AI touched it.

• 0.0 — clean human writing, untouched or trivially edited
• ~0.3-0.6 — partially AI-edited (rephrasing, tone changes, added detail)
• 1.0 — fully AI-generated

Why use it

• Graded, not binary. A real number you can threshold however your policy needs — strict for high-stakes review, lenient where light AI help is fine.
• Built for the common case. Tuned on human text that was edited by AI, the messy middle that binary detectors handle worst.
• Self-hostable. Standard 🤗 Transformers model, single-GPU inference, no API — your text never leaves your machine.
• Tunable false positives. Because the output is continuous, you pick the operating point — raise the threshold to avoid wrongly flagging human writers.
• A size for every budget. Part of a family — 0.6B for speed, 1.7B for balance, 4B for top accuracy.

Useful for editorial and academic-integrity workflows, content-provenance auditing, dataset curation (filtering AI-contaminated corpora), and platform moderation where you need a degree of AI involvement rather than a label.

How it works

The model is a 4-bucket sequence classifier over increasing levels of AI editing (bucket 0 = human, bucket 3 = fully AI). At inference the four bucket probabilities are collapsed into one continuous score by taking their expected value, normalized to [0, 1]:

score = sum(p_i * i) / (n_buckets - 1)

This expected-bucket decoding turns a discrete classifier into a smooth, monotone "extent-of-editing" meter: text the model is sure is human scores near 0, fully-AI text scores near 1, and genuinely mixed text lands in between in proportion to how much was rewritten.

How it was trained

• Backbone: Qwen/Qwen3-4B-Base, fine-tuned with QLoRA (4-bit NF4 base, LoRA rank 8 on all attention and MLP projections; the classification head trained in full).
• Objective: cross-entropy over 4 edit-extent buckets, 5 epochs.
• Supervision: targets come from an embedding-based edit-magnitude metric (cosine distance between a text and its human source), bucketed at thresholds 0.03 / 0.15 — so the labels encode how far an edit moved the text, not just that it changed.
• Training data: 2,400 examples of human-written texts paired with AI-edited and fully-AI-generated versions across reviews, web articles, news, and creative writing.
• Compute: a single H100, minutes of fine-tuning.

Training loss over the 5 epochs (logged every 0.2 epoch; sqrt-scaled y-axis to expose the tail) converges smoothly to near zero, confirming the model was genuinely fit on this training split:

 4.03 |o
      | oo
      |   ooooooo
      |          ooooo o o
      |               o o oooooo
0.003 |
      +-------------------------
       epoch 0.2             5.0

The approach is inspired by recent work on quantifying the extent of AI editing (EditLens, Thai et al., 2025) rather than treating detection as binary classification.

Performance

Evaluated on a held-out test set of 6,115 texts spanning human, AI-edited, and AI-generated writing.

Three-way classification (human / AI-edited / AI-generated):

Metric	Score
Accuracy	0.892
Macro-F1	0.890
F1 — human	0.888
F1 — AI-generated	0.958
F1 — AI-edited	0.824

Binary (human vs. any AI involvement): accuracy 0.920, macro-F1 0.913.

The score tracks the actual amount of editing (Pearson correlation with independent edit-magnitude metrics on the test set):

Reference edit metric	Correlation
Embedding cosine distance	+0.885
Soft n-gram novelty	+0.956

A correlation near +0.95 with n-gram novelty means the score rises smoothly and reliably as more of the text is rewritten — exactly the graded behavior a binary detector can't give you.

Usage

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tok = AutoTokenizer.from_pretrained("reneeice/editlens-qwen3-4b-repro")
model = AutoModelForSequenceClassification.from_pretrained("reneeice/editlens-qwen3-4b-repro").eval()

def ai_edit_score(text: str) -> float:
    inputs = tok(text.lower(), truncation=True, max_length=1024, return_tensors="pt")
    with torch.no_grad():
        probs = model(**inputs).logits.softmax(-1)[0]
    buckets = torch.arange(probs.numel(), dtype=probs.dtype)
    return (probs @ buckets / (probs.numel() - 1)).item()

print(ai_edit_score("Your text here."))  # 0.0 = human, 1.0 = fully AI

Pick a threshold that matches your tolerance (e.g. flag > 0.5 for "substantially AI-edited"), or use the raw score directly as a feature.

Limitations

• English text; best on inputs of roughly a paragraph or more (very short snippets are noisier).
• The score reflects degree of AI editing, not authorship intent or quality — treat it as a signal, not a verdict, and keep a human in the loop for consequential decisions.
• Like all detectors it can be affected by domain shift and adversarial paraphrasing; calibrate the threshold on data resembling your own.

License

Apache-2.0.