raglab

raglab — the agentic-search / RAG orchestration layer on top of ir.

raglab turns a retrieval substrate into a Composable Search Agent (the ir_09 architecture): a small set of injected roles — Planner, Formulator, Retriever, Evaluator, Reranker, Citer — wired by a control loop whose back-edge (evaluator → reformulate) is what makes it an agent rather than a DAG. Concrete tools live at the leaves: an ir corpus becomes one Retriever via ir.as_retriever.

Quick start (the no-LLM thin slice — runs offline):

import ir
import raglab

# register ir corpora as the agent's sources, then search across them:
sources = raglab.ir_sources("skills", "reports", mode="hybrid")
agent = raglab.make_search_agent(sources)
results = agent("how do I deploy the app")      # ranked ir.SearchHits

Inject an LLM formulator (query rewrite/HyDE) and evaluator (sufficiency + refinement) to turn on query understanding and the back-edge. Dependency direction is one-way: raglab imports ir; ir never imports raglab.

> Fresh start (v0.2.0+). This repo took over the raglab PyPI name; the older > backend now lives at raglab_bak. Development is just beginning.

class raglab.Budget(max_rounds: int = 3, max_sources_per_task: int = 4, max_results_per_task: int = 50)

Loop bounds: the safety net under the (harder) sufficiency decision.

class raglab.Citer(*args, **kwargs)

Confirm/annotate that each result supports its use (identity by default).

class raglab.Evaluator(*args, **kwargs)

Judge relevance + sufficiency; optionally emit a refinement (back-edge).

class raglab.Formulator(*args, **kwargs)

Turn a sub-task + one source into concrete low-level queries.

class raglab.Judgement(relevant: Sequence[SearchHit], sufficient: bool, refinement: SubTask | None = None)

An evaluator’s verdict over a round’s results.

relevant is the kept subset; sufficient says whether to stop; a non-None refinement is the back-edge — the next sub-task to re-query with. The pass-through default returns sufficient=True and no refinement (no loop).

class raglab.LowLevelQuery(source: str, query: str, params: Mapping[str, ~typing.Any]=<factory>)

One concrete query against one source.

query is the text handed to the source’s Retriever; params are per-call retriever overrides (e.g. mode / filter / k for an ir corpus). A formulator turns a SubTask into these.

class raglab.Planner(*args, **kwargs)

Decompose a query into sub-tasks and select sources for each.

class raglab.Query(text: str, constraints: Mapping[str, ~typing.Any]=<factory>)

A user intent: free text plus optional structured constraints.

class raglab.Reranker(*args, **kwargs)

Produce the final ordering over the (cross-source) merged results.

raglab.Result

alias of SearchHit

class raglab.SingleContextAgent(sources: ~collections.abc.Mapping[str, ~typing.Callable[[...], list[~ir.base.SearchHit]]], planner: ~raglab.agent.Planner = <function single_subtask_planner>, formulator: ~raglab.agent.Formulator = <function identity_formulator>, evaluator: ~raglab.agent.Evaluator = <function passthrough_evaluator>, reranker: ~raglab.agent.Reranker = <function rrf_reranker>, citer: ~raglab.agent.Citer = <function identity_citer>, budget: ~raglab.agent.Budget = <factory>)

One ReAct-style loop, sequential sub-tasks (ir_09 §7 — the cheap default).

The loop is fixed; every decision is an injected role. Promotion to a multi-agent orchestrator (ir_09 §7) swaps this class while keeping the same role contracts. The back-edge is the single line current = judged.refinement in _run_task() — that is what makes this an agent.

citer() → Sequence[SearchHit]

No-op citer (verification needs a generated claim — that lives in srag).

evaluator(results: Sequence[SearchHit]) → Judgement

Pass-through critic: keep everything, declare sufficient, never re-query.

formulator(source: str) → list[LowLevelQuery]

Identity formulator: the sub-goal verbatim as one query (no rewrite/HyDE).

planner(sources: Mapping[str, Callable[[...], list[SearchHit]]]) → list[SubTask]

Trivial planner: one sub-task over all registered sources, no decomposition.

reranker() → Sequence[SearchHit]

Cross-source merge (the default fan-in): fuse by rank, never by raw score.

Groups the accumulated pool by hit.source and delegates the merge to ir.fuse_hits() (ir is the SSOT for hit operations): within each source raw scores order and dedup that source’s hits — one scale, sound — and across sources only ranks interact (Reciprocal Rank Fusion), so heterogeneous embedders / modes can never mis-order the merge, and colliding artifact_ids from different sources stay distinct results (identity is (source, artifact_id)).

A single-source pool (or an untagged one — hits with no source) keeps its raw scores and exactly score_reranker()’s ordering; fused, rank-derived scores only appear when there is genuinely something to fuse. Each fused hit keeps its pre-fusion magnitude as metadata["source_score"]. For per-source weights, another rrf_k, or opt-in cross-source duplicate merging, use make_rrf_reranker().

class raglab.SubTask(goal: str, sources: tuple[str, ...])

A planner’s unit of work: a sub-goal bound to a set of registered sources.

raglab.identity_citer(results: Sequence[SearchHit]) → Sequence[SearchHit]

No-op citer (verification needs a generated claim — that lives in srag).

raglab.identity_formulator(task: SubTask, source: str) → list[LowLevelQuery]

Identity formulator: the sub-goal verbatim as one query (no rewrite/HyDE).

raglab.ir_sources(*names: str, **search_defaults: Any) → dict[str, Callable[[...], list[SearchHit]]]

A source registry {name: Retriever} backed by named ir corpora.

Each name is bound to ir.as_retriever(name, **search_defaults), opened eagerly. For the lazy live view over everything registered (a corpus opens only when its key is first used), use ir.retrievers() instead — the agent accepts either, or any Mapping[name, Retriever]. search_defaults (e.g. mode="hybrid") apply to every source.

raglab.make_llm_evaluator(*, judge: Callable[[...], tuple[bool, str | None] | str] | None = None, select_strategy: str = 'conservative', select_kwargs: Mapping[str, Any] | None = None, prerank: Reranker | None = None, prompt: str = 'A search agent is pursuing this goal:\n\n{goal}\n\nA calibrated selector reviewed the retrieved candidates and committed to the\nresults below (it abstained if this list is empty):\n\n{results}\n\nDecide whether these results are SUFFICIENT to satisfy the goal.\n- If they are, reply with exactly: SUFFICIENT\n- If they are not, reply with: INSUFFICIENT\n  then, on the next line, a single improved search query that would retrieve what\n  is still missing. Keep it a terse search phrase — no prose, no numbering.\n', max_result_chars: int = 500, **prompt_function_kwargs: Any) → Evaluator

An LLM-backed raglab Evaluator (turns on the back-edge).

Relevance is ir’s: each round the accumulated results are passed through ir.select() and the committed subset becomes the Judgement.relevant (so the LLM stays in its lane — LLM relevance is known-fragile, ir_01 §3). Sufficiency is the LLM’s: it reads the committed subset (informed by ir’s model-free sufficient hint via abstention) and decides whether the goal is satisfied. When it is not, the judge’s improved query becomes a refinement SubTask over the same sources — the back-edge.

Parameters:

• judge – an injectable (goal, results) -> (sufficient, refinement) callable (a test double, or your own router); a raw text reply is also accepted and parsed. When omitted it is built lazily on oa.

• select_strategy – ir selection strategy for the relevance decision (default "conservative" — distractor-robust).

• select_kwargs – extra args forwarded to ir.select() (e.g. max_k, rel). min_score is allowed only while the round’s pool keeps raw scores (single-source, or prerank=score_reranker): an absolute floor is per-(corpus, mode, embedder), so a round whose pool was rank-fused across sources raises rather than silently mass-abstaining on ordinal RRF scores.

• prerank – the per-round merge that puts the accumulated pool best-first (and deduped) before ir.select. Defaults to rrf_reranker() — a round’s pool can already mix sources (a SubTask spans several), so the same rank-based, scale-safe merge as the fan-in applies; single-source rounds keep raw scores. Inject score_reranker() for sources known to share one score scale.

• max_result_chars – per-result text truncation in the judge prompt.

Safe fallback: any judge error returns refinement=None — the control loop’s break condition — so a failing judge can never fabricate an endless loop. Sufficiency without a refinement query is likewise treated as a stop.

raglab.make_llm_formulator(*, formulate: Callable[[str], str | Sequence[str]] | None = None, params: Mapping[str, Any] | None = None, **make_kwargs: Any) → Formulator

An LLM-backed raglab Formulator.

Adapts an ir-style query formulator (str -> str | [str]) to the role contract (SubTask, source) -> [LowLevelQuery]: the sub-goal text is rewritten / expanded into one or more search queries, each wrapped as a LowLevelQuery against source.

Parameters:

• formulate – an injectable ir-style formulator (a test double, or one you built). When omitted it is built once via ir.make_llm_formulator() (**make_kwargs are forwarded — e.g. n, prompt, rewriter); that call is offline (oa is imported lazily only when the formulator is invoked), so import raglab stays offline.

• params – per-call retriever overrides (e.g. {"mode": "hybrid", "k": 5}) attached to every emitted LowLevelQuery.

The boundary: this returns queries, never SubTasks. ir’s formulator already falls back to identity on any failure, so the emitted list is never empty — a formulator must never make retrieval worse than the raw sub-goal.

raglab.make_rrf_reranker(*, rrf_k: int = 60, weights: Mapping[str, float] | None = None, identity: Callable[[SearchHit], Any] | str | None = None) → Reranker

A parametrized rrf_reranker() (per-source trust weights, rrf_k).

Parameters:

• rrf_k – the RRF rank constant (standard default 60).

• weights – optional per-source trust dial, by source name (default 1.0 each) — biases the merge without ever comparing raw scores.

• identity – opt-in cross-source duplicate detection (e.g. "pointer") — see ir.retrieve.Identity. Default: never merge across sources.

raglab.make_search_agent(sources: Mapping[str, Callable[[...], list[SearchHit]]], *, planner: Planner | None = None, formulator: Formulator | None = None, evaluator: Evaluator | None = None, reranker: Reranker | None = None, citer: Citer | None = None, budget: Budget | None = None) → SingleContextAgent

Build a SingleContextAgent over sources with smart defaults.

sources is a Mapping[name, Retriever] — e.g. {"skills": ir.as_retriever("skills")}, ir_sources(), or the lazy ir.retrievers() view. Every role defaults to its no-LLM thin-slice implementation, so make_search_agent(sources)("query") just works; inject an LLM formulator / evaluator to turn on rewriting and the back-edge.

A custom Retriever must return ir.SearchHit instances (the Result alias): the loop stamps provenance (hit.source) on its output, so duck-typed hit objects raise at the tagging step.

raglab.passthrough_evaluator(task: SubTask, results: Sequence[SearchHit]) → Judgement

Pass-through critic: keep everything, declare sufficient, never re-query.

raglab.rrf_reranker(results: Sequence[SearchHit]) → Sequence[SearchHit]

Cross-source merge (the default fan-in): fuse by rank, never by raw score.

Groups the accumulated pool by hit.source and delegates the merge to ir.fuse_hits() (ir is the SSOT for hit operations): within each source raw scores order and dedup that source’s hits — one scale, sound — and across sources only ranks interact (Reciprocal Rank Fusion), so heterogeneous embedders / modes can never mis-order the merge, and colliding artifact_ids from different sources stay distinct results (identity is (source, artifact_id)).

A single-source pool (or an untagged one — hits with no source) keeps its raw scores and exactly score_reranker()’s ordering; fused, rank-derived scores only appear when there is genuinely something to fuse. Each fused hit keeps its pre-fusion magnitude as metadata["source_score"]. For per-source weights, another rrf_k, or opt-in cross-source duplicate merging, use make_rrf_reranker().

raglab.score_reranker(results: Sequence[SearchHit]) → Sequence[SearchHit]

Magnitude merge: one surface per artifact, ordered by descending raw score.

Delegates to ir.base.best_per_artifact() (ir is the SSOT for hit operations): an artifact retrieved by several queries / rounds — common once the back-edge re-queries — survives once, at its highest score. Identity is (source, artifact_id), so two sources’ same-id artifacts never collapse.

A plain score sort compares raw scores across sources, which is only sound when every source shares one score scale (same embedder + mode) — it is the explicit homogeneous-sources opt-in. The default fan-in is rrf_reranker(), which never compares raw scores across sources.

raglab.single_subtask_planner(query: Query, sources: Mapping[str, Callable[[...], list[SearchHit]]]) → list[SubTask]

Trivial planner: one sub-task over all registered sources, no decomposition.