Reza2kn/Shenava-Koochik-v1.5-RNNT-sherpa-onnx

🦻 Shenava · Koochik v1.5 — RNNT for sherpa-onnx

Persian (فارسی) on-device speech recognition. This is the RNNT / transducer branch of Reza2kn/Shenava-Koochik-v1.5 — a 114M FastConformer hybrid — exported as an offline transducer for sherpa-onnx (encoder + decoder + joiner).

It runs with the same binary everywhere sherpa-onnx does: Linux · macOS · Windows · Android · iOS · WebAssembly · Raspberry Pi, from Python, C++, C#, Go, Swift, Kotlin, Dart, Rust, and more.

Part of VisualEars / Shenava — open, offline, real-time Persian captioning for the Deaf & Hard-of-Hearing.

---

✨ Why this model exists

v1.0's RNNT head was broken — a tokenizer/blank misalignment sent greedy decoding into an infinite loop (~4398 % WER, unusable). v1.5 revives it: the prediction network and joint are surgically re-initialised under the ve_tok_v4 tokenizer and trained up with a two-phase curriculum → a genuinely working transducer head. Crucially, the record-setting CTC head is frozen and preserved bit-for-bit during the corrective fine-tune, so there is zero CTC regression.

This repository packages the transducer branch only for sherpa-onnx. If you want the lower-WER CTC head, see the related models below.

📊 Benchmarks

golden-6669 — a held-out Persian test set (strict fair_text normalizer, att_context [70,13]):

head	WER ↓	CER ↓	status
CTC (v1.0 = v1.5)	8.12 %	~2.3 %	record-holder · recommended for lowest WER
RNNT (this repo)	9.50 %	~2.6 %	newly functional in v1.5

The RNNT head trails the CTC head slightly on WER but is a fully independent decoding path (useful for fusion, confidence, or streaming experiments).

📦 What's inside

Both precisions are included — pick per your size/latency budget:

file set	precision	encoder	decoder	joiner
encoder.onnx · decoder.onnx · joiner.onnx	fp32	457 MB	16 MB	5.6 MB
encoder.int8.onnx · decoder.int8.onnx · joiner.int8.onnx	int8 (dynamic)	131 MB	4 MB	1.4 MB
tokens.txt	—	SentencePiece BPE-1024 + <blk>
export-rnnt-sherpa.py	—	reproducible export script

int8 is ~3.5× smaller and decoded byte-identically to fp32 on our Persian test clips — it's a free win for most deployments.

🚀 Quick start

Python

pip install sherpa-onnx soundfile

import sherpa_onnx, soundfile as sf

recognizer = sherpa_onnx.OfflineRecognizer.from_transducer(
    encoder="encoder.onnx",          # or encoder.int8.onnx
    decoder="decoder.onnx",          # or decoder.int8.onnx
    joiner="joiner.onnx",            # or joiner.int8.onnx
    tokens="tokens.txt",
    model_type="nemo_transducer",    # ⚠️ REQUIRED — see notes below
    num_threads=2,
    sample_rate=16000,
    feature_dim=80,
    decoding_method="greedy_search",
)

audio, sr = sf.read("audio.wav", dtype="float32")   # 16 kHz mono
if audio.ndim > 1:
    audio = audio[:, 0]

stream = recognizer.create_stream()
stream.accept_waveform(sr, audio)
recognizer.decode_stream(stream)
print(stream.result.text)

Command line

sherpa-onnx-offline \
  --encoder=encoder.int8.onnx \
  --decoder=decoder.int8.onnx \
  --joiner=joiner.int8.onnx \
  --tokens=tokens.txt \
  --model-type=nemo_transducer \
  audio.wav

Example output (colloquial, see notes): سلام حال شما خوبه نه همچنان

⚠️ Important notes

• model_type="nemo_transducer" is required. Without it, sherpa-onnx routes to the icefall stateless-decoder path and fails with 'context_size' does not exist in the metadata. NeMo uses an LSTM prediction network, not a stateless context-N decoder.
• Un-normalized features. These models train with normalize=NA (they consume raw log-mel). The encoder metadata sets normalize_type="" so sherpa-onnx does not re-normalize — otherwise the output is garbage. This is baked into the model; you don't need to do anything.
• Full-context / offline. The encoder is exported at att_context [-1,-1] for offline (whole-utterance) decoding, matching the offline CTC package. For real-time streaming, see the streaming export recipe linked below.
• Colloquial output & numbers. The RNNT head is trained on audio-faithful labels, so it leans colloquial and drops sentence punctuation (؟ / ،). Numbers are emitted in spoken form (هشت, not ۸) — apply Persian inverse text normalization at display time if you need digits.

🧠 Model details

Architecture	FastConformer hybrid RNNT-CTC (transducer branch exported here)
Parameters	114 M · 17 encoder layers · d_model 512 · ×8 subsampling (80 ms/frame)
Prediction network	1-layer LSTM · pred_hidden 640
Vocabulary	1024 SentencePiece BPE + blank (ve_tok_v4)
Sample rate / features	16 kHz · 80-dim log-mel
Language	Persian / Farsi (fa)
Base checkpoint	Reza2kn/Shenava-Koochik-v1.5

🔁 Reproduce

export-rnnt-sherpa.py (included) loads the v1.5 .nemo, sets full context, and exports the transducer branch following the official sherpa-onnx NeMo recipe:

python export-rnnt-sherpa.py shenava-koochik-v1.5.nemo

🔗 Related models

model	head	runtime
Shenava-Koochik-v1.5	RNNT + CTC	NeMo (source)
Shenava-v1-sherpa-onnx	CTC	sherpa-onnx
Shenava-Koochik-v1.0-tract-streaming	CTC	tract (pure Rust, streaming)
Shenava-Koochik-1.0-CoreML-iOS15-fp16	CTC	CoreML (Apple ANE)

📄 License & citation

Released under the Apache License 2.0. If you use it:

@misc{shenava_koochik_v15_rnnt_sherpa,
  title  = {Shenava Koochik v1.5 — RNNT for sherpa-onnx},
  author = {Sayar, Reza},
  year   = {2026},
  url    = {https://huggingface.co/Reza2kn/Shenava-Koochik-v1.5-RNNT-sherpa-onnx}
}

Built with ❤️ for accessibility · shenava.app