jan-hq commited on
Commit
de24021
·
verified ·
1 Parent(s): d3f4d90

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -16
README.md CHANGED
@@ -26,22 +26,7 @@ Jan-v1 leverages the newly released [Qwen3-4B-thinking](https://huggingface.co/Q
26
  ### Question Answering (SimpleQA)
27
  For question-answering, Jan-v1 shows a significant performance gain from model scaling, achieving 91.2% accuracy.
28
 
29
- | Model | SimpleQA Accuracy |
30
- | :--- | :--- |
31
- | **Jan-v1 (Ours)** | **91.1%** |
32
- | Qwen3-4B-thinking-2507 | 86.5% |
33
- | Jan-nano-128k-MCP (YaRN 130k) | 83.2% |
34
- | Jan-nano-MCP | 80.7% |
35
- | Jan-nano-MCP (YaRN 130k) | 79.7% |
36
- | Lucy (YaRN 130k) | 78.3% |
37
- | DeepSeek-V3-MCP | 78.2% |
38
- | ChatGPT-4.5 | 62.5% |
39
- | Baseline-MCP | 59.2% |
40
- | Gemini-2.5-Pro | 52.9% |
41
- | Claude-3.7-Sonnet | 50% |
42
- | o3 | 49.4% |
43
- | Grok-3 | 44.6% |
44
- | o1 | 42.6% |
45
 
46
  *The 91.2% SimpleQA accuracy represents a significant milestone in factual question answering for models of this scale, demonstrating the effectiveness of our scaling and fine-tuning approach.*
47
 
 
26
  ### Question Answering (SimpleQA)
27
  For question-answering, Jan-v1 shows a significant performance gain from model scaling, achieving 91.2% accuracy.
28
 
29
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/65713d70f56f9538679e5a56/iWBkeFvE9jAvB9VN_5JeT.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
  *The 91.2% SimpleQA accuracy represents a significant milestone in factual question answering for models of this scale, demonstrating the effectiveness of our scaling and fine-tuning approach.*
32