Agentic RAG in Practice: Self-RAG, Adaptive Routing, and Corrective Query Loops
Transforming passive retrieval into an active cognitive loop: building Self-RAG and Corrective RAG (CRAG) pipelines that grade document relevance, rewrite queries, and fall back to web search.
Standard RAG is a passive linear pipeline:
$$\text{User Query} \longrightarrow \text{Retrieve Chunks} \longrightarrow \text{Generate Answer}$$
If the retrieval query is poorly phrased, the database contains stale data, or the retrieved chunks lack the answer, traditional RAG blindly generates a completion anyway—often resulting in confident, fabricated hallucinations.
Agentic RAG flips this paradigm from passive ingestion to active investigation. The agent:
1. Decides if retrieval is necessary.
2. Evaluates the semantic relevance of retrieved snippets.
3. If the snippets are deficient, rewires the query and re-executes.
4. If internal corpora fail completely, routes the query to real-time external web search.
In this capstone post of Module 2, we architect an end-to-end Corrective RAG (CRAG) state machine in Python.
1. The Corrective RAG (CRAG) State Topology
[User Inquiry]
│
▼
[Is Retrieval Required?]
│ │
(No) (Yes)
│ │
▼ ▼
[Direct LLM Resp] [Execute Hybrid Search]
│
▼
[Document Relevance Grader]
│
┌───────────────┼───────────────┐
▼ ▼ ▼
(CORRECT) (AMBIGUOUS) (INCORRECT)
│ │ │
▼ ▼ ▼
[Synthesize] [Query Rewriter] [Fallback to Web Search]
│ │ │
│ └───────┬───────┘
│ ▼
│ [Secondary Retrieval]
│ │
└───────────────┬───────┘
▼
[Hallucination Guard]
│
▼
[Final Response]
2. Component 1: The Document Relevance Grader
Rather than feeding raw retrieved chunks straight to the answer generator, an agentic grader evaluates each candidate snippet independently:
from pydantic import BaseModel, Field
from typing import Literal
class DocumentGrade(BaseModel):
binary_score: Literal["yes", "no"] = Field(
...,
description="Whether the document contains facts directly relevant to answering the user question."
)
explanation: str = Field(..., description="Brief reasoning behind the evaluation decision.")
def grade_retrieval_relevance(client, query: str, document_text: str) -> bool:
"""Evaluates whether a single retrieved chunk is relevant to the query."""
prompt = f"""
You are a strict technical grading judge. Evaluate if the following document contains information
relevant to resolving the user inquiry.
[USER INQUIRY]: {query}
[RETRIEVED DOCUMENT]: {document_text}
"""
response = client.beta.chat.completions.parse(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}],
response_format=DocumentGrade,
temperature=0.0
)
return response.choices[0].message.parsed.binary_score == "yes"
3. Component 2: The Corrective Query Rewriter
When the grader flags that retrieved documents are irrelevant or insufficient, the agent diagnoses the initial query and reformulates it for improved vector affinity:
def rewrite_query_for_search(client, original_query: str, failed_context_summary: str) -> str:
"""Transforms a conversational question into an optimized semantic keyword query."""
prompt = f"""
The initial retrieval for the following query returned poor or irrelevant results.
[ORIGINAL QUERY]: {original_query}
[FAILED CONTEXT ATTEMPT]: {failed_context_summary}
Re-formulate this query to maximize semantic similarity and keyword density in an enterprise database.
Return only the optimized query string.
"""
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}],
temperature=0.2
)
return response.choices[0].message.content.strip()
4. End-to-End Corrective RAG Controller
Here is the master control loop binding retrieval, grading, query re-writing, and web fallback:
class CorrectiveRAGController:
def __init__(self, client, vector_engine, web_search_tool):
self.client = client
self.vector_engine = vector_engine
self.web_search = web_search_tool
def answer_inquiry(self, query: str) -> str:
print(f"\n🔍 [QUERY RECEIVED]: {query}")
# Step 1: Initial Hybrid Retrieval
documents = self.vector_engine.search(query, limit=4)
# Step 2: Relevance Grading Filter
valid_docs = []
for doc in documents:
if grade_retrieval_relevance(self.client, query, doc["text"]):
valid_docs.append(doc["text"])
print(f"📊 [GRADING RESULT]: {len(valid_docs)} of {len(documents)} documents deemed relevant.")
# Step 3: Decision Branching
if len(valid_docs) >= 2:
# Sufficient internal context -> Direct synthesis
return self._generate_grounded_answer(query, "\n\n".join(valid_docs))
elif len(valid_docs) == 1:
# Marginal context -> Rewrite query and try secondary vector pass
print("⚠️ [LOW RELEVANCE]: Attempting query reformulation...")
new_query = rewrite_query_for_search(self.client, query, valid_docs[0])
secondary_docs = self.vector_engine.search(new_query, limit=3)
combined_context = valid_docs + [d["text"] for d in secondary_docs]
return self._generate_grounded_answer(query, "\n\n".join(combined_context))
else:
# Zero relevant internal documents -> Fallback to external web search
print("🌐 [ZERO RELEVANCE]: Falling back to external web search tool...")
web_results = self.web_search.execute(query)
return self._generate_grounded_answer(query, f"WEB SEARCH RESULTS:\n{web_results}")
def _generate_grounded_answer(self, query: str, context: str) -> str:
prompt = f"""
Answer the user inquiry using strictly the provided context. If the context does not contain
the answer, state that you do not have sufficient information.
[CONTEXT]:
{context}
[QUERY]:
{query}
"""
response = self.client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0.0
)
return response.choices[0].message.content
Summary of Module 2: The Retrieval & Context Layer
In this module, we elevated RAG from a brittle static index to a dynamic, self-correcting agentic subsystem:
- Post 05: Hierarchical parent-document chunking for precision vector matching without context loss.
- Post 06: Hybrid search combining BM25 keyword matching, dense vectors, and cross-encoder re-ranking.
- Post 07: Self-RAG and Corrective RAG (CRAG) loops that grade relevance, rewrite queries, and fall back to web tools.
Now that our agent can reliably perceive the world and interrogate vast knowledge corpora, we need a robust framework to orchestrate complex, multi-step state machines.
Welcome to Module 3: State Machines & Orchestration [LangGraph], starting with [Post 08: The Limits of Linear Chains]!