Jan-v1-4B / README.md
jan-hq's picture
Update README.md
9935c46 verified
|
raw
history blame
3.49 kB
metadata
license: apache-2.0
language:
  - en
base_model:
  - Qwen/Qwen3-4B-Thinking-2507
pipeline_tag: text-generation

Jan-v1: Advanced Agentic Language Model

GitHub License Jan App

Overview

Introducing Jan-v1, the first release in the Jan Family – specifically designed for advanced agentic reasoning and complex problem-solving within the Jan App. Building on the innovative agentic capabilities of our earlier Lucy model, Jan-v1 represents a significant leap forward through strategic model scaling.

Jan-v1 leverages the newly released Qwen3-4B-thinking model to deliver significantly enhanced reasoning capabilities and effective tool utilizatio. This architectural evolution is designed to deliver superior performance on complex agentic tasks, setting a new benchmark for accessible, high-performance AI.

Evaluation

Question Answering (SimpleQA)

For question-answering, Jan-v1 shows a significant performance gain from model scaling, achieving 91.2% accuracy.

Model SimpleQA Accuracy
Jan-v1 (Ours) 91.1%
Qwen3-4B-thinking-2507 86.5%
Jan-nano-128k-MCP (YaRN 130k) 83.2%
Jan-nano-MCP 80.7%
Jan-nano-MCP (YaRN 130k) 79.7%
Lucy (YaRN 130k) 78.3%
DeepSeek-V3-MCP 78.2%
ChatGPT-4.5 62.5%
Baseline-MCP 59.2%
Gemini-2.5-Pro 52.9%
Claude-3.7-Sonnet 50%
o3 49.4%
Grok-3 44.6%
o1 42.6%

The 91.2% SimpleQA accuracy represents a significant milestone in factual question answering for models of this scale, demonstrating the effectiveness of our scaling and fine-tuning approach.

Report Generation & Factuality

Evaluated on a benchmark testing factual report generation from web sources, using an LLM-as-judge. The benchmark includes our proprietary Jan Exam - Longform and the DeepResearchBench.

Model Average Overall Score
o4-mini 7.30
Jan-v1-4B (Ours) 7.17
gpt-4.1 6.90
Qwen3-4B-Thinking-2507 6.84
4o-mini 6.60
Jan-nano-128k 5.63

Quick Start

Integration with Jan App

Jan-v1 is optimized for direct integration with the Jan App. Simply select the model from the Jan App interface for immediate access to its full capabilities.

Local Deployment

Using vLLM:

vllm serve Menlo/Jan-v1 \
    --host 0.0.0.0 \
    --port 1234 \
    --enable-auto-tool-choice \
    --tool-call-parser hermes

Using llama.cpp:

llama-server --model jan-v1.gguf \
    --host 0.0.0.0 \
    --port 1234

Recommended Parameters

temperature: 0.7
top_p: 0.9
top_k: 20
min_p: 0.0
max_tokens: 2048

🤝 Community & Support

📄 Citation

Updated Soon