ef.service

The HTTP-service bridge — EfService, a stateless-friendly facade.

ef’s query surface is stateful: ingest() returns a live SearchableCorpus, and a SourceManager holds a live corpus, an ArtifactGraph and vd collections. HTTP is stateless. To serve ef over HTTP — so a frontend (app_ef) can reach it through qh.mk_app() — something must map a JSON-friendly corpus_id back to the live indexed object across requests.

EfService is that bridge: a facade holding a handle registry {corpus_id: SourceManager} on the instance. A single EfService() is constructed once at server start-up, and its seven bound methods — create_corpus(), search(), retrieve(), explore_corpus(), corpus_info(), list_corpora(), delete_corpus() — are handed to qh.mk_app() as the HTTP surface. The registry lives on the instance, never as a module global: a process-wide mutable registry would be the ServiceContext singleton anti-pattern ef rejects (.claude/CLAUDE.md §6) — inject an EfService explicitly instead.

Every method is JSON-friendly and fully type-annotated, because qh derives the HTTP schema from the type hints. ef’s stateful objects never cross the boundary: a corpus is addressed by its string corpus_id, an embedder / segmenter by a string the DI seam resolves ("hashing", "openai:...", "cohere:...", …), and results are plain SearchHit / Segment data.

This is only a transport bridge — it adds no orchestration. ef stays a facade (.claude/CLAUDE.md §6); answer synthesis, agents and UI live outside.

Example — create a corpus, search it, then drop it, all offline:

>>> from ef.service import EfService
>>> service = EfService()
>>> info = service.create_corpus(
...     ['the cat sat on the mat', 'dogs are loyal', 'felines and canines'],
...     corpus_id='animals',
... )
>>> info['corpus_id'], info['n_sources'], info['n_segments']
('animals', 3, 3)
>>> info['embedder'], info['dim']
('hashing:v1@512', 512)
>>> hits = service.search('animals', 'cat', limit=2)
>>> len(hits)
2
>>> isinstance(hits[0].score, float)
True
>>> segments = service.retrieve('animals', 'cat', limit=1)
>>> isinstance(segments[0]['text'], str)
True
>>> [c['corpus_id'] for c in service.list_corpora()]
['animals']
>>> service.delete_corpus('animals')
>>> service.list_corpora()
[]

class ef.service.CorpusInfo

A JSON-friendly summary of one registered corpus.

The return shape shared by EfService.create_corpus(), EfService.corpus_info() and (as a list) EfService.list_corpora() — a plain dict an HTTP client and qh’s schema both understand.

Keys:

corpus_id: the registry handle — how every other method addresses it. n_sources: the number of source documents in the corpus. n_segments: the number of indexed segments (sources cut by the segmenter). embedder: the embedder’s model_id (e.g. "hashing:v1@512"). dim: the embedding dimensionality. config_id: the content hash of the segmenter+embedder pipeline.

class ef.service.EfService(*, default_embedder: str | None = None)

A registry-backed facade bridging stateful ef to a stateless transport.

Construct one EfService per process, register corpora with create_corpus(), then query them by corpus_id. Hand the seven bound methods to qh.mk_app() to expose them over HTTP — see the module docstring.

The handle registry is private instance state — never a module global. Corpora are isolated: each create_corpus() builds its own SourceManager over its own in-memory vd backend, so one corpus’s vectors never leak into another’s search.

default_embedder sets the embedder create_corpus() resolves when its own embedder argument is None — the per-instance hook a host (e.g. app_ef) uses to pick a policy default without passing embedder= on every call. It defaults to DEFAULT_EMBEDDER ("hashing"), so a bare EfService() stays dependency-free and offline.

>>> service = EfService()
>>> len(service)
0
>>> _ = service.create_corpus(['hello world'], corpus_id='greetings')
>>> 'greetings' in service
True
>>> len(service)
1

A host can swap the default embedder at construction (the string is resolved lazily, per corpus, by the DI seam — constructing the service touches no network):

>>> EfService(default_embedder='openai:text-embedding-3-small')
<EfService: 0 corpus(es) registered>

corpus_info(corpus_id: str) → CorpusInfo

Return the CorpusInfo of a registered corpus.

Raises:

KeyError – if corpus_id is not registered.

create_corpus(sources: list[str], *, embedder: str | None = None, segmenter: str | None = None, corpus_id: str | None = None, embedder_api_key: str | None = None) → CorpusInfo

Index sources into a new corpus, register it, return its CorpusInfo.

Parameters:

• sources – the corpus — a list of text documents.

• embedder – the embedder, as a string the DI seam resolves (as_embedder()) — "hashing", "openai:text-embedding-3-small", "cohere:...", an http(s):// URL, …. None → the service’s default_embedder (chosen at construction; itself DEFAULT_EMBEDDER, the dependency-free HashingEmbedder, unless a host overrode it).

• segmenter – the segmenter, as a string (as_segmenter()) — None → the recursive-character default.

• corpus_id – the handle to register the corpus under; None → a fresh random id. Reusing a live id is an error.

• embedder_api_key – an optional API key for the embedder — the bring-your-own-key seam. When given and embedder is a hosted-API spec ("openai:…", "cohere:…", …), the key is forwarded to that adapter, so the service itself never needs to hold one. None → the adapter’s usual environment-variable resolution. Ignored for key-less embedders (e.g. "hashing"). The corpus’s embedder keeps the key for the life of the corpus, so search() / retrieve() / explore_corpus() need no key of their own.

Returns:

the CorpusInfo of the freshly indexed corpus.

Raises:

ValueError – if corpus_id is already registered.

delete_corpus(corpus_id: str) → None

Drop a corpus from the registry, releasing its index.

Raises:

KeyError – if corpus_id is not registered.

explore_corpus(corpus_id: str, *, dims: int = 2, projection_method: str = 'auto', cluster_method: str = 'kmeans', n_clusters: int = 8, label: bool = False) → ExploreResult

Project & cluster a registered corpus — the corpus-map surface.

Runs ef.exploration.explore() over the corpus: every indexed segment is projected to dims coordinates and assigned a cluster, returned as a row-aligned ExploreResult (ids / coords / labels / cluster_titles) — the JSON-friendly shape an app_ef corpus map consumes.

Parameters:

• corpus_id – the corpus to explore.

• dims – projection target dimensionality — 2 or 3.

• projection_method – "auto" / "umap" / "pca".

• cluster_method – "kmeans" / "hdbscan".

• n_clusters – number of k-means clusters.

• label – when True, also name each cluster with an LLM (needs the ef[imbed] extra and a key); default False.

Raises:

• KeyError – if corpus_id is not registered.

• ValueError – if the corpus has fewer than 2 indexed segments.

list_corpora() → list[CorpusInfo]

Return the CorpusInfo of every registered corpus.

retrieve(corpus_id: str, query: str, *, limit: int = 10) → list[Segment]

Retrieve ranked Segments — the RAG-plug-in shape.

Like search(), but returns plain segments in rank order (the score dropped, the source_id folded into metadata["source"]) — clean context to hand to an external RAG/agent framework. ef returns context; it does not synthesize answers.

Raises:

KeyError – if corpus_id is not registered.

search(corpus_id: str, query: str, *, limit: int = 10) → list[SearchHit]

Search a registered corpus — up to limit ranked SearchHits.

Each hit carries the matched Segment, its similarity score (higher = closer) and the source_id it was cut from.

Raises:

KeyError – if corpus_id is not registered.