Update README.md
Browse files
README.md
CHANGED
@@ -25,13 +25,23 @@ Jan-v1 leverages the newly released [Qwen3-4B-thinking](https://huggingface.co/Q
|
|
25 |
|
26 |
Jan-v1's strategic scaling has resulted in a notable performance uplift, particularly evident in its "thinking" and reasoning prowess. Following the established MCP benchmark methodology, Jan-v1 sets a new standard for models in its class.
|
27 |
|
28 |
-
| Model
|
29 |
-
|
|
30 |
-
| **Jan-v1 (Qwen3-4B)**
|
31 |
-
|
|
32 |
-
|
|
33 |
-
|
34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
35 |
|
36 |
## Quick Start
|
37 |
|
|
|
25 |
|
26 |
Jan-v1's strategic scaling has resulted in a notable performance uplift, particularly evident in its "thinking" and reasoning prowess. Following the established MCP benchmark methodology, Jan-v1 sets a new standard for models in its class.
|
27 |
|
28 |
+
| Model | SimpleQA Accuracy |
|
29 |
+
| :--- | :--- |
|
30 |
+
| **Jan-v1 (Ours, Qwen3-4B)** | **91.1%** |
|
31 |
+
| Jan-nano-128k-MCP (YaRN 130k) | 83.2% |
|
32 |
+
| Jan-nano-MCP | 80.7% |
|
33 |
+
| Jan-nano-MCP (YaRN 130k) | 79.7% |
|
34 |
+
| Lucy (YaRN 130k) | 78.3% |
|
35 |
+
| DeepSeek-V3-MCP | 78.2% |
|
36 |
+
| ChatGPT-4.5 | 62.5% |
|
37 |
+
| Baseline-MCP | 59.2% |
|
38 |
+
| Gemini-2.5-Pro | 52.9% |
|
39 |
+
| Claude-3.7-Sonnet | 50% |
|
40 |
+
| o3 | 49.4% |
|
41 |
+
| Grok-3 | 44.6% |
|
42 |
+
| o1 | 42.6% |
|
43 |
+
|
44 |
+
The **91.1% accuracy on SimpleQA** underscores Jan-v1's advanced ability to precisely retrieve and synthesize information, showcasing the effectiveness of our model scaling approach for agentic intelligence.
|
45 |
|
46 |
## Quick Start
|
47 |
|