ek.agents.cost

The cost model and the price SSOT: dollars per successfully completed task.

The unit that matters for an agent is cost per successfully completed task, not cost per token. Two facts force it (misc/docs/ek_11):

  1. Tokens are not dollars. Input, output, cached-input and reasoning tokens are priced asymmetrically (output typically ~5x input; cached input bills at a fraction). A “tokens per success” figure is at best a rough proxy.

  2. Failure is not free. Tokens spent on an episode that failed are pure waste, so the denominator must be successes, not attempts. This is Cost-of-Pass: the expected monetary cost of obtaining a correct solution, E[cost] / P(success) – which diverges to infinity when the agent never succeeds (a model that cannot solve a task at any price is not cheap, it is unusable; a per-token metric that reports it as “$0.003/call” is actively misleading).

No hardcoded prices. Rates go stale monthly, so this module ships no built-in price table: a ModelPrice is either passed directly or resolved from an injected catalog (load_prices() reads LiteLLM’s MIT-licensed model_prices_and_context_window.json – the data file, never the SDK, whose enterprise/ subtree is a proprietary carve-out). Missing rates raise an actionable error rather than silently guessing.

Example

>>> from ek.agents.base import Cost
>>> price = ModelPrice(input=1e-6, output=5e-6)      # $1 / $5 per 1M tokens
>>> c = Cost(input_tokens=1_000_000, output_tokens=200_000)
>>> round(dollars(c, price), 4)
2.0
ek.agents.cost.DEFAULT_BATCH_DISCOUNT = 0.5

Batch/deferred APIs are ~50% off. A keyword multiplier, never assumed.

ek.agents.cost.DEFAULT_CACHE_DISCOUNT = 0.1

Cached input is billed at a fraction of the normal input rate (~10% across the major providers). Used only when a model’s price entry declares no explicit cached rate.

class ek.agents.cost.ModelPrice(input: float = 0.0, output: float = 0.0, cached_input: float | None = None, reasoning: float | None = None)[source]

Per-token rates for one model, in USD per token.

Parameters:

  • input – USD per input (prompt) token.

  • output – USD per output (completion) token.

  • cached_input – USD per cached-input token; defaults to DEFAULT_CACHE_DISCOUNT * input when not declared.

  • reasoning – USD per reasoning token; defaults to the output rate (reasoning tokens are billed as output by the major providers).

property cached_rate: float

The cached-input rate (explicit, else the discounted input rate).

property reasoning_rate: float

The reasoning-token rate (explicit, else the output rate).

exception ek.agents.cost.UnknownModelPrice[source]

Raised when a model’s rates are not in the injected price catalog.

ek.agents.cost.cost_of_pass(total_cost: float, n_success: int) → float[source]

Expected monetary cost of one successful task: total_cost / n_success.

Returns inf when nothing succeeded – infeasibility is the honest answer, not a cheap-looking zero (this divergence is the point of the metric).

Example

>>> cost_of_pass(10.0, 4)
2.5
>>> cost_of_pass(10.0, 0)
inf
ek.agents.cost.cost_report(episodes: Iterable[Episode], *, prices: Mapping[str, ModelPrice] | None = None, price: ModelPrice | None = None) → dict[source]

The quality x cost x latency triple, reported together.

Scoring accuracy without cost lets an agent chase tiny gains with unbounded API calls, so a report that omits cost is not a report. Returns n, n_success, success_rate, total_dollars, cost_per_success (Cost-of-Pass), mean_latency_s and total_tokens.

With no price/prices, dollar figures are None rather than a fabricated zero: ek will not invent rates, and a silent 0.0 would make a costly agent look free. Tokens and latency are still reported.

Example

>>> from ek.agents.base import Cost, Episode
>>> p = per_million(1.0, 1.0)
>>> eps = [Episode(task_id="a", cost=Cost(input_tokens=1_000_000), success=True),
...        Episode(task_id="b", cost=Cost(input_tokens=1_000_000), success=False)]
>>> r = cost_report(eps, price=p)
>>> r["success_rate"], round(r["cost_per_success"], 2)
(0.5, 2.0)
>>> cost_report(eps)["cost_per_success"] is None      # no rates -> no dollars invented
True
ek.agents.cost.dollars(cost: Cost, price: ModelPrice | str, *, prices: Mapping[str, ModelPrice] | None = None, batch: bool = False) → float[source]

Monetary cost of one Cost tally, in USD.

Parameters:

  • cost – The token/retry tally.

  • price – A ModelPrice, or a model name to resolve against prices.

  • prices – The injected price catalog (required when price is a name).

  • batch – Apply the batch-API discount (DEFAULT_BATCH_DISCOUNT).

Raises:

UnknownModelPrice – if a model name is given with no rates for it.

ek.agents.cost.episode_dollars(episode: Episode, *, prices: Mapping[str, ModelPrice] | None = None, price: ModelPrice | None = None, batch: bool = False) → float[source]

Monetary cost of one episode (0.0 if it carries no Cost).

ek.agents.cost.load_prices(source: Any) → dict[source]

Parse a LiteLLM-format price catalog into a {model: ModelPrice} table.

source is a path to (or an already-parsed mapping of) LiteLLM’s MIT-licensed model_prices_and_context_window.json. We read the data file only and never import the litellm SDK – its enterprise/ subtree is a proprietary carve-out, and the SDK drags a large transitive tree in just to look up a rate.

Entries lacking token rates (embeddings, unpriced models) are skipped.

Example

>>> table = load_prices({"m": {"input_cost_per_token": 1e-6,
...                            "output_cost_per_token": 2e-6}})
>>> table["m"].output
2e-06
ek.agents.cost.per_million(input: float, output: float, *, cached_input=None, reasoning=None) → ModelPrice[source]

Build a ModelPrice from the human-facing USD per 1M tokens figures.

Example

>>> p = per_million(3.0, 15.0)          # $3 in / $15 out per 1M tokens
>>> round(p.input, 9), round(p.output, 9)
(3e-06, 1.5e-05)
ek.agents.cost.price_of(model: str, *, prices: Mapping[str, ModelPrice] | None = None) → ModelPrice[source]

Resolve a model’s rates from an injected catalog, or fail actionably.

ek deliberately ships no built-in price table (rates go stale monthly and would be magic numbers). Supply one with prices=, or load the LiteLLM catalog with load_prices().