Skip to main content

The spend you can act on: agent turns, budgets and one rate table

Maintainers

Reading what your models cost is not the same as being able to act on it. This release closes that gap — and it opens by fixing a way of reading those spans that turned out to be wrong.

  • Tool calls are no longer counted as model calls. The AI module decided what counted as a call to a model by testing that gen_ai.operation.name was present, never looking at its value — but chat is only one of the values the convention defines. An agent instrumentation emits execute_tool for every tool it runs, so on an agent workload every tool execution was counted as a model call, and four numbers went wrong together: call counts inflated (one turn consulting two tools reported as five calls), the latency percentiles ranked a database lookup against a completion, the model resolved to nothing and that empty row then counted as a model of its own, and the "reported no usage" bucket — which exists to name an instrumentation gap honestly — filled with spans that were never model calls at all.

    Calls are now split by operation, and every existing view reads the inference population it was designed for. Embeddings stay counted as model calls, because an embeddings call spends real tokens on a real model. An estate whose applications make plain chat completions was never affected and will see no change; one running agents will see its call counts fall to the truth.

  • The tools your agents actually run. Once tool executions were told apart they became worth reporting, and the model table could never have shown them: a new Tools table names each tool an agent ran, how often, how often it failed, how long it took at p95, and which services invoked it. A tool a turn hit four times is one row with a count, because the loop is the thing worth seeing.

    No tokens and no cost columns, deliberately — a tool execution spends neither, and a zero there would read as "this tool is free" rather than "tokens are not the unit". A tool whose instrumentation set no name is reported under its span name and labelled as such rather than dropped.

  • An agent turn, drawn as the shape it is. A turn is not a list — it is a model call that decides, a fan-out to tools, results coming back, often another model call after — and the questions worth asking about it are graph questions: which tool is slow, which fails, how many hops before it converges, which one a retry loop is stuck on. A new Turn view on the trace draws exactly that, and appears only on traces that hold one.

    Time is weighted by what was spent inside each call rather than by span duration, because a model-call span contains the tool spans it triggered and duration would report the model as responsible for time the tools spent.

  • A threshold on spend. Nobody watches a screen. Monthly budgets — in tokens or in money, for one calling service or across the estate — now fire through the alerting channels you already have: a warning at 80% of the ceiling by default, an alert at the ceiling, each resolving on its own when the month rolls over or spend drops back.

    A budget in money is refused at startup when no prices are declared: spend measured without prices is zero, so such a budget would sit under every threshold forever and never fire. Where only some models in scope are priced, the alert says so — an operator reading "at 82%" is entitled to know the real figure is higher.

  • One rate table, written down once. Declaring what your estate costs used to mean doing it twice, in two formats, one of which needed a pod restart: model prices as a mounted ConfigMap, compute rates as environment variables read once at startup. The two had separate currency fields, so an install could render one screen in EUR and another in USD with nothing noticing.

    Both now live in one table, editable in Settings → Rates and applied without a redeploy. Values declared in your chart keep working exactly as they did and are shown read-only — offering an edit would promise a change a helm upgrade silently reverts — with UI-authored entries overlaying them and every row saying which it is. There is one currency now, and an install that had set the two differently gets a startup warning naming both rather than a silent pick.