hoanganhpham commited on
Commit
f619c10
·
verified ·
1 Parent(s): ef2c1d6

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +106 -0
README.md ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/63466107f7bd6326925fc770/b6xfld0bUDDAQIFvMCapD.png)
2
+
3
+ # II-Search-4B
4
+
5
+
6
+ <aside>
7
+
8
+ A 4B parameter language model specialized in information seeking, multi-hop reasoning, and web-integrated search, achieving state-of-the-art performance among models of similar size.
9
+
10
+ </aside>
11
+
12
+ ## Model Description
13
+
14
+ II-Search-4B is a 4B parameter language model based on Qwen3-4B, fine-tuned specifically for information seeking tasks and web-integrated reasoning. It excels at complex multi-hop information retrieval, fact verification, and comprehensive report generation.
15
+
16
+ ### Key Features
17
+
18
+ - Enhanced tool usage for web search and webpage visits
19
+ - Multi-hop reasoning capabilities with sophisticated planning
20
+ - Verified information retrieval with cross-checking
21
+ - Strong performance on factual QA benchmarks
22
+ - Comprehensive report generation for research queries
23
+
24
+ ## Training Methodology
25
+
26
+ Our training process consisted of three key phases:
27
+
28
+ ### Phase 1: Tool Call Ability Stimulation
29
+
30
+ We used a distillation approach from larger models (Qwen3-235B) to generate reasoning paths with function calling on multi-hop datasets. This established the base capabilities for tool use.
31
+
32
+ ### Phase 2: Reasoning Improvement
33
+
34
+ We addressed initial limitations by:
35
+
36
+ - Creating synthetic problems requiring more reasoning turns, inspired by Random Walk algorithm
37
+ - Improving reasoning thought patterns for more efficient and cleaner reasoning paths
38
+
39
+ ### Phase 3: Rejection Sampling & Report Generation
40
+
41
+ We applied:
42
+
43
+ - Filtering to keep only high-quality reasoning traces (correct answers with proper reasoning)
44
+ - STORM-inspired techniques to enhance comprehensive report generation
45
+
46
+ ## Performance
47
+
48
+ | **Benchmark** | **Qwen3-4B** | **Jan-4B** | **WebSailor-3B** | **II-Search-4B** |
49
+ | --- | --- | --- | --- | --- |
50
+ | OpenAI/SimpleQA | 76.8 | 80.1 | 81.8 | 91.8 |
51
+ | Google/Frames | 30.7 | 24.8 | 34.0 | 67.5 |
52
+ | Seal_0 | 6.31 | 2.7 | 1.8 | 22.5 |
53
+
54
+ ### Tool Usage Comparison
55
+
56
+ **Simple QA (SerpDev)**
57
+
58
+ | | **Qwen3-4B** | **Jan-4B** | **WebSailor-3B** | **II-Search-4B** |
59
+ | --- | --- | --- | --- | --- |
60
+ | # Search | 1.0 | 0.9 | 2.1 | 2.2 |
61
+ | # Visit | 0.1 | 1.9 | 6.4 | 3.5 |
62
+ | # Total Tools | 1.1 | 2.8 | 8.5 | 5.7 |
63
+
64
+ All benchmark traces from models can be found at: https://huggingface.co/datasets/II-Vietnam/Inspect-Search-Models-Benchmarking-Result
65
+
66
+ ## Intended Use
67
+
68
+ II-Search-4B is designed for:
69
+
70
+ - Information seeking and factual question answering
71
+ - Research assistance and comprehensive report generation
72
+ - Fact verification and evidence-based reasoning
73
+ - Educational and research applications requiring factual accuracy
74
+
75
+ ## Usage
76
+
77
+ - LMStudio
78
+
79
+ ### Recommended Generation Parameters
80
+
81
+ ```python
82
+ generate_cfg = {
83
+ 'thought_in_content': True,
84
+ 'top_p': 0.95,
85
+ 'temperature': 0.6,
86
+ 'repetition_penalty': 1.1,
87
+ 'max_tokens': 2048
88
+ }
89
+
90
+ ```
91
+
92
+ - For a query that you need to find a short and accurate answer. Add the following phrase: "\n\nPlease reason step-by-step and put the final answer within \\boxed{}."
93
+
94
+ ## Citation
95
+
96
+ ```
97
+ @misc{II-Search-4B,
98
+ author = {Intelligent Internet},
99
+ title = {II-Search-4B: Information Seeking and Web-Integrated Reasoning LLM},
100
+ year = {2025},
101
+ publisher = {Hugging Face},
102
+ journal = {Hugging Face Hub},
103
+ howpublished = {\url{https://huggingface.co/II-Vietnam/II-Search-4B}},
104
+ }
105
+
106
+ ```