jan-hq commited on
Commit
c29df97
·
verified ·
1 Parent(s): d359ca3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -7
README.md CHANGED
@@ -25,13 +25,23 @@ Jan-v1 leverages the newly released [Qwen3-4B-thinking](https://huggingface.co/Q
25
 
26
  Jan-v1's strategic scaling has resulted in a notable performance uplift, particularly evident in its "thinking" and reasoning prowess. Following the established MCP benchmark methodology, Jan-v1 sets a new standard for models in its class.
27
 
28
- | Model | SimpleQA Accuracy |
29
- | :---------------------------------- | :---------------- |
30
- | **Jan-v1 (Qwen3-4B)** | **91.2%** |
31
- | Lucy (Qwen3-1.7B) | [Lucy's Score] |
32
- | DeepSeek-v3 (Comparison from Lucy) | [DeepSeek's Score]|
33
-
34
- The **91.2% accuracy on SimpleQA** underscores Jan-v1's advanced ability to precisely retrieve and synthesize information, showcasing the effectiveness of our model scaling approach for agentic intelligence.
 
 
 
 
 
 
 
 
 
 
35
 
36
  ## Quick Start
37
 
 
25
 
26
  Jan-v1's strategic scaling has resulted in a notable performance uplift, particularly evident in its "thinking" and reasoning prowess. Following the established MCP benchmark methodology, Jan-v1 sets a new standard for models in its class.
27
 
28
+ | Model | SimpleQA Accuracy |
29
+ | :--- | :--- |
30
+ | **Jan-v1 (Ours, Qwen3-4B)** | **91.1%** |
31
+ | Jan-nano-128k-MCP (YaRN 130k) | 83.2% |
32
+ | Jan-nano-MCP | 80.7% |
33
+ | Jan-nano-MCP (YaRN 130k) | 79.7% |
34
+ | Lucy (YaRN 130k) | 78.3% |
35
+ | DeepSeek-V3-MCP | 78.2% |
36
+ | ChatGPT-4.5 | 62.5% |
37
+ | Baseline-MCP | 59.2% |
38
+ | Gemini-2.5-Pro | 52.9% |
39
+ | Claude-3.7-Sonnet | 50% |
40
+ | o3 | 49.4% |
41
+ | Grok-3 | 44.6% |
42
+ | o1 | 42.6% |
43
+
44
+ The **91.1% accuracy on SimpleQA** underscores Jan-v1's advanced ability to precisely retrieve and synthesize information, showcasing the effectiveness of our model scaling approach for agentic intelligence.
45
 
46
  ## Quick Start
47