AgenticRAGTracer
AgenticRAGTracer is a hop-aware benchmark for diagnosing multi-step retrieval reasoning in Agentic RAG systems, accepted at ACL 2026 Findings.
Tool overview
AgenticRAGTracer addresses the lack of systematic diagnostic tools for multi-step retrieval reasoning in Agentic RAG. Existing RAG benchmarks primarily evaluate single-hop retrieval accuracy and fail to capture the step-by-step reasoning behavior of agents performing multiple retrievals.
The project provides a multi-hop evaluation pipeline: multihop_pipeline.py for core generation, multihop_run.py for parallel execution, and multihop_prompt.yaml for prompt templates. Typical use cases include evaluating multi-step retrieval reasoning in RAG systems, regression testing of agent retrieval strategies, and paper reproduction or comparison experiments.
Its strengths include ACL 2026 Findings paper backing and a clean structure that is ready to run. The main caveats are its early stage (14 stars, 1 fork), limited community adoption, and evolving documentation. Teams with strict evaluation coverage or stability requirements should monitor update frequency.
It best fits academic teams and engineers researching RAG or agent retrieval. It is less suitable as a sole production quality gate.