piyushptiwari/insureos-models

INSUREOS Models — Complete Insurance AI Training Pipeline

Created by Bytical AI — AI agents that run insurance operations.

Start Here

If you are trying to understand the Hugging Face repos, start with LEARN.md.

That guide explains, in plain English, why the project is split across model, dataset, code, and search repos, and it walks through the concepts, data, and training code behind each model in this repository.

Overview

INSUREOS is a complete AI/ML training and inference pipeline for UK insurance operations. This repository contains all source code for data generation, model training, evaluation, data collection, and a hybrid search engine.

Model Suite

Model	HuggingFace	Task	Key Metric
InsureLLM-4B	piyushptiwari/InsureLLM-4B	Insurance domain LLM	ROUGE-1: 0.384
InsureDocClassifier	piyushptiwari/InsureDocClassifier	12-class document classification	F1: 1.0
InsureNER	piyushptiwari/InsureNER	13-entity NER	F1: 1.0
InsureFraudNet	piyushptiwari/InsureFraudNet	Fraud detection (3 LoB)	AUC-ROC: 1.0
InsurePricing	piyushptiwari/InsurePricing	Premium pricing (GLM + EBM)	MAE: £11,132
InsureSearch	piyushptiwari/insureos-search-engine	Hybrid search engine	33K docs indexed

Training Dataset

piyushptiwari/insureos-training-data — 10K SFT, 5K DPO, 50K tabular, 10K docs, 8K NER

Repository Structure

insureos-models/
├── data/                        # Synthetic data generation
│   ├── constants.py             # UK insurance constants (regions, perils, regulators)
│   ├── gen_sft.py               # Generate SFT instruction-response pairs
│   ├── gen_dpo.py               # Generate DPO preference pairs
│   ├── gen_documents.py         # Generate insurance documents (12 classes)
│   ├── gen_ner.py               # Generate NER-annotated text
│   ├── gen_tabular.py           # Generate claims tabular data
│   └── generate_all.py          # Run all generators
│
├── collect/                     # Real-world data collection
│   ├── config.py                # Scraping targets and configuration
│   ├── scraper_base.py          # Base HTTP scraper with caching
│   ├── convert_sft.py           # Convert raw docs → SFT/DPO format
│   ├── run_fast.py              # Fast collection orchestrator
│   └── sources/                 # Per-source scrapers
│       ├── wikipedia.py         # Wikipedia insurance articles
│       ├── legislation.py       # UK legislation (legislation.gov.uk)
│       ├── fca.py               # FCA Handbook
│       ├── hf_datasets.py       # HuggingFace insurance datasets
│       ├── rss_news.py          # Insurance news RSS feeds
│       └── education.py         # Insurance education resources
│
├── training/                    # Model training scripts
│   ├── qlora_finetune.py        # QLoRA fine-tuning (Qwen3-4B)
│   ├── dpo_train.py             # DPO alignment training
│   ├── retrain_realworld.py     # Real-world data retraining
│   ├── doc_classifier.py        # ModernBERT document classifier
│   ├── ner_model.py             # ModernBERT NER model
│   ├── fraud_model.py           # XGBoost + Isolation Forest fraud
│   ├── pricing_glm.py           # Tweedie GLM + EBM pricing
│   └── distill.py               # Model distillation (experimental)
│
├── evaluation/                  # Evaluation suite
│   ├── run_eval.py              # Full multi-model evaluation
│   └── results/                 # Evaluation results (JSON)
│
├── search/                      # Hybrid search engine
│   ├── config.py                # Search configuration
│   ├── embedder.py              # BGE-small-en-v1.5 embedding service
│   ├── bm25.py                  # Custom Okapi BM25 implementation
│   ├── vector_store.py          # Qdrant vector store
│   ├── reranker.py              # Cross-encoder reranker
│   ├── hybrid_engine.py         # RRF fusion (vector + BM25 + reranker)
│   ├── indexer.py               # Document ingestion pipeline
│   ├── models.py                # Pydantic data models
│   └── api.py                   # FastAPI REST API
│
├── serve/                       # Model serving
│   └── api.py                   # FastAPI inference endpoints
│
└── scripts/                     # Automation
    ├── setup.sh                 # Environment setup (NVIDIA, Python, deps)
    └── train_all.sh             # Full training pipeline script

Quick Start

1. Environment Setup

# Create virtual environment
python3 -m venv .venv && source .venv/bin/activate

# Install dependencies
pip install torch transformers trl peft bitsandbytes
pip install xgboost scikit-learn interpret
pip install sentence-transformers qdrant-client fastapi uvicorn

2. Generate Training Data

python -m data.generate_all
# Outputs: data/output/ (SFT, DPO, docs, NER, tabular)

3. Train Models

# Train all models sequentially
bash scripts/train_all.sh

# Or individually:
python training/qlora_finetune.py          # InsureLLM QLoRA
python training/dpo_train.py               # InsureLLM DPO
python training/doc_classifier.py          # Document classifier
python training/ner_model.py               # NER model
python training/fraud_model.py             # Fraud detection
python training/pricing_glm.py             # Pricing models

4. Evaluate

python evaluation/run_eval.py
# Results saved to evaluation/results/

5. Run Search Engine

# Index documents
python search/indexer.py

# Start API
python search/api.py
# API at http://localhost:8900
# Endpoints: /search, /search/vector, /search/keyword, /suggest, /facets, /stats

Search Engine — InsureSearch

A hybrid search engine rivaling Azure AI Search, built entirely on open-source components:

Component	Technology	Details
Vector Search	BGE-small-en-v1.5 (384-dim) + Qdrant	Semantic similarity
Keyword Search	Custom Okapi BM25	Insurance-aware tokenization
Reranking	cross-encoder/ms-marco-MiniLM-L-6-v2	Cross-encoder reranking
Fusion	Reciprocal Rank Fusion (RRF)	Vector 60% + BM25 40%
API	FastAPI	REST API with facets, suggestions

Index stats: 33,034 chunks from 31,679 documents, 51,640 BM25 terms.

Training Pipeline

Stage 1: Synthetic Data Generation
├── 10K SFT instruction-response pairs
├── 5K DPO preference pairs
├── 50K tabular claims (Motor/Property/Liability)
├── 10K insurance documents (12 classes)
└── 8K NER-annotated texts (13 entity types)

Stage 2: QLoRA Fine-Tuning → Qwen3-4B
├── rank=64, alpha=128, all-linear targets
├── 2 epochs, batch=2, grad_accum=4
├── Final: train_loss=0.012, eval_loss=0.118
└── Token accuracy: 95.88%

Stage 3: DPO Alignment
├── 5K preference pairs
├── 149 steps, reward_accuracy=1.0
└── Reward margin: 26.76

Stage 4: Real-World Data Collection
├── Wikipedia (150 docs), UK Legislation (692)
├── HuggingFace datasets (31,060), RSS (50), Education (88)
├── Converted to 3,685 SFT + 776 DPO pairs
└── Quality filtered (English-only, no echo responses)

Stage 5: Real-World Retraining
├── 876 steps on real-world SFT data
└── Claims process score improved 0.40 → 0.60

Stage 6: Specialized Models (parallel)
├── FraudNet: XGBoost + Isolation Forest → AUC-ROC 1.0
├── PricingGLM: Tweedie GLM + EBM → MAE £11,132
├── DocClassifier: ModernBERT → F1 1.0
└── InsureNER: ModernBERT → F1 1.0

Tech Stack

• LLM: Qwen3-4B + QLoRA + DPO (PyTorch, Transformers, TRL, PEFT, bitsandbytes)
• Classification & NER: ModernBERT-base (Transformers)
• Fraud Detection: XGBoost + Isolation Forest (scikit-learn)
• Pricing: Tweedie GLM (scikit-learn) + EBM (InterpretML)
• Search: BGE-small-en-v1.5 + Qdrant + BM25 + cross-encoder
• Training GPU: NVIDIA Tesla T4 16GB

Citation

@misc{bytical2026insureos,
  title={INSUREOS: A Complete AI/ML Suite for UK Insurance Operations},
  author={Bytical AI},
  year={2026},
  url={https://huggingface.co/piyushptiwari/insureos-models}
}

About Bytical AI

Bytical builds AI agents that run insurance operations — claims automation, underwriting intelligence, digital sales, and core system modernization for insurers across the UK and Europe. Microsoft AI Partner | NVIDIA | Salesforce.