DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment
Abstract
Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-horizon interactions. We present DeepSearch-Evolve, a self-distillation framework for web agents built on DeepSearch-World, a deterministic and verifiable environment with reproducible search and page-reading tools. DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks and supports key agentic cognitive behaviors useful for self-evolving, including progress verification, grounded reflection, and failure recovery. DeepSearch-Evolve iteratively performs trajectory generation, filtering, data mixing, and fine-tuning to train stronger agents. Without distillation from more capable models, DeepSearch-World-9B achieves competitive performance compared with open-source agents, reaching 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA, showing that verifiable environments enable scalable self-evolution for long-horizon web agents. We will release the environment, 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.
Community
Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-horizon interactions. We present DeepSearch-Evolve, a self-distillation framework for web agents built on DeepSearch-World, a deterministic and verifiable environment with reproducible search and page-reading tools. DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks and supports key agentic cognitive behaviors useful for self-evolving, including progress verification, grounded reflection, and failure recovery. DeepSearch-Evolve iteratively performs trajectory generation, filtering, data mixing, and fine-tuning to train stronger agents. Without distillation from more capable models, DeepSearch-World-9B achieves competitive performance compared with open-source agents, reaching 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA, showing that verifiable environments enable scalable self-evolution for long-horizon web agents. We will release the environment, 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.
Great, help me a lot. Thanks for your work.
How many of us have watched an agent burn 14 tool calls on the same dead-end page because the reward signal was too sparse to tell it to try something else? That's the gap this paper actually addresses — self-distillation in a verifiable environment where you can trust the reward. The 420K multi-hop QA tasks built from entity-level random walks is a smart design choice: it gives you ground-truth trajectories without hand-labeling, which is the bottleneck that kills most self-improvement attempts in practice. What I'd want to see is how the self-distilled policies transfer when the search environment changes — different page structures, different APIs. The deterministic environment is a strength for training but it's also the thing that usually breaks first in prod. If the self-distillation holds up under distribution shift, that's the real result worth watching.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search (2026)
- Learning Agent-Compatible Context Management for Long-Horizon Tasks (2026)
- SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation (2026)
- S1-DeepResearch: Beyond Search, Toward Real-World Long-Horizon Research Agents (2026)
- Why Does Feedback-Augmented Self-Distillation Fail to Improve Retrieval-Interleaved Search Agents? (2026)
- STAMP: Provenance-Guided Credit Assignment for Deep Search Agents (2026)
- DEEPRUBRIC: Evidence-Tree Rubric Supervision for Efficient Reinforcement Learning of Deep Research Agents (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper