Reconcile the LLM cost ledger with the bill, and count cache tokens by response format

In OwlAI, administrators can set a usage limit for each user, and OwlAI SaaS draws down a member's credits on every call. Both read the LLM costs the product records itself. If that record comes in under the provider's bill, limits stop only after real spending has passed them, and credits shrink by less than was used. If it comes in over, users are blocked while budget is still left. The record usually drifts in three places. With prompt caching, the token fields in a response mean different things depending on the provider. The processing tier you request can differ from the tier you are billed at. And the price table itself can differ from the price actually billed.

This post describes the design that handles all three. Every billable call goes into a usage ledger (a table that records the tokens and amount of each call as one row), and the amount is fixed when the row is written. A single function counts tokens by looking at the format of the response, and the amount is computed at the price of the processing tier the response reports. Whether the ledger is right is checked against the amount the provider actually billed, not against other numbers inside the ledger.

#Every billable call becomes one ledger row

There are two ledgers. LLM calls go into the LLM usage ledger, and separately billed tool calls such as embedding, reranking, speech recognition and web search go into the tool usage ledger, one row per call. Amounts are recorded in USD. Per-user limit checks, the administrator's limit screen, a member's own usage gauge and per-user organization cost all read one aggregation that combines the two ledgers. That aggregation counts only calls made with the shared key. The Total cost tab under System status in the admin console reads the same two ledgers and shows the installation's total split into models and tools.

A separate cost table for each feature (chat, playbooks, meetings) would get any single screen built faster. But each feature would drift toward its own counting rule, and once a filter is added in only one place, the limit check and the screen see different numbers. So the shared-key condition and the period condition are written in one place only, the aggregation.

An LLM ledger row holds the time, model, provider, user, conversation, input and output tokens, cache read and write tokens, the amount, and whether the call used the shared key or a personal key. It also stores the processing tier that was requested and the tier the response reported. Each row records which agent made the call, inside which unit of work and how many levels down, so the cost of work where a lead and researchers run together, such as deep research, can be broken down by agent.

Ledger writes follow these rules.

• Failed and stopped calls are recorded too. Providers bill for what they processed even when a call fails partway. “A turn's input tokens grow with the square of its rounds” covers how the tokens of a call the user stopped mid-answer are filled in.
• Embedding and rerank adapters keep their return type unchanged and hand over usage separately when a call ends. A call path in product code that leaves out the place to receive this usage fails the type check. A call on such a path would never reach the ledger and would become a cost the limits cannot see.
• A failed ledger write does not fail the call. The failure is logged. Raising it as a call failure would make the job retry, and the retry bills the same call to the provider a second time.
• The tool usage ledger stores the usage the provider reported as is, and draws down budget by the share computed directly from the request. For a tool that fills its billing unit from usage accumulated across calls, the call that crosses the unit boundary is reported the earlier calls' share as well.

#Cache tokens are counted by response format, not provider name

With prompt caching, one response carries fields for input tokens, tokens read from the cache and tokens newly written to the cache. How these fields overlap differs by provider. “One system prompt edit zeroes cache reads, so changing blocks go last” covers what the cache saves; this section covers only how its tokens are counted in the ledger.

Count both formats with one rule and the error runs to orders of magnitude. Assume the inclusive format and clip cache reads with the rule "cache reads cannot exceed input tokens", and a response in the separate format loses most of its cache reads. The better a long conversation uses the cache, the smaller its new input and the larger its cache reads, so the ledger amount shrinks to a fraction of the real bill.

So token normalization happens in exactly one function. If cache read and write fields sit separately at the top level of the response usage, the function adds the three fields into "total input, including cache reads and writes"; otherwise it takes input tokens as total input. The provider name plays no part in the decision. The ledger's provider column holds the name an administrator gave the provider in that installation, and that name says nothing about the response format.

The ledger, the structured log written for each round (“On-premises OwlAI is observed through one JSON log, joined to the worker by a request ID”), a turn's token total and the token estimate for stopped calls all go through this function. If logs and ledger counted by different rules, the two numbers would diverge with no basis for deciding which one is right.

#Amounts use four prices and the processing tier the response reports

Prices are set per model row in the installation config, under pricing. A chat model has four prices, matching the provider's invoice line items one to one: uncached input (input), cache read (cachedInput), cache write (cacheWrite) and output (output). Cache write gets its own price because some models bill tokens newly written to the cache at more than the list input price. If the cache read or write price is left empty, the list input price applies. Guessing low on an unknown price would put the ledger under the real bill, and the limits would stop late by that difference.

Two more conditions change the amount.

• Long-context pricing applies per request. For a model with a long-context band (longContext), once a single request's total input crosses the threshold, the whole request is priced at the higher rate. It is not the case that only the part above the threshold costs more.
• The processing tier comes from the response. A request sent at the fast tier may be processed and billed at the standard tier when traffic is heavy. So the amount uses the price of the tier the response reports, and a mismatch with the requested tier is logged. If the config has no price for the reported tier, the ledger does not guess with a multiplier: it records the amount at that model row's price and logs an error, a sign that amounts are drifting from the bill.

To keep the ledger from quietly recording $0 for a model without a price, the config is checked when it is read. A model row for an external provider without a price is rejected by config validation. A self-hosted model with no price is recorded at $0, and once an operator enters GPU cost as its price, it starts drawing down budget (“Measure the vLLM concurrency limit with production input lengths, not short prompts”). OwlAI SaaS, which bills in credits, refuses to start if any model lacks a price, self-hosted models included. A model is judged to use no budget only when its declared price is 0, and a failed price lookup counts as a model that uses budget.

Calls made with a personal key record only tokens and call counts, with the amount at 0. The key owner is billed for those calls, and the product cannot know that account's discounts or pricing tier. The admin console shows personal-key tokens and personal-key calls separately.

The amount is fixed at recording time. After a price change, calls from servers started with the new config are recorded at the new price, and rows already written are not recomputed. A row that matched the bill at the time still holds the right value.

#Only the provider's billed amount shows whether the ledger is right

A test that compares numbers inside the ledger passes even when the counting rule is wrong. Count with a rule that clips cache reads, and the ledger, the screens and the limits are all wrong together while agreeing with each other. The error only shows when you compare against what the provider actually billed.

Major providers offer a usage API and a cost API, called with an organization admin key. The usage API returns tokens and request counts per model per day. The cost API splits each day's amount into line items such as input, cached input, cache write and output, with long-context charges shown separately. With these two APIs, the operations procedure checks two things.

1. Amount reconciliation: compare the ledger's daily, per-model totals with the same cells in the cost API.
2. Back-calculating actual prices: divide each line item's amount in the cost API by the token count of the same item in the usage API to get that day's actual price, and compare it with the price in the config.

The cached input line item separates cases best in this back-calculation. Usage of the same key outside the ledger raises the amount and the tokens together, so the quotient holds, and on the day a price changes the value breaks cleanly. The input and output items drift by a few percent once usage outside the ledger mixes in.

These conditions come up during reconciliation.

• If one organization has several projects, filter by project.
• If several installations share a key, reconcile on request counts and tokens rather than amounts, because the other installations' usage mixes into the amounts.
• Costs are bucketed by UTC day, and the current day's costs arrive late. If a price changes on a boundary day, that day's back-calculated value mixes the two prices.

In an internal production installation, reconciling one provider's usage, long-context charges included, over about 40 days in September and October 2026, 39 of 41 daily per-model cells matched to the fourth decimal place, and the difference in the period total was under 0.05%.

When a price needs changing, the back-calculated value comes before the pricing page. A pricing page's footnote or listed price sometimes differs from the price actually billed, and the day a price cut is announced can be several days off from the day the bill starts using it. The price written into the config is the value confirmed by back-calculation, and the evidence kept with it is the pricing page's address and the date it was checked.

#Design choices

• We chose an operations procedure over automatic reconciliation inside the product. Having the product call the cost API would save operators work. But the admin key behind that API reads usage and cost for the whole organization, so it is kept out of the product and used only for reconciliation. On-premises customers can match the total on the Total cost tab against the bill in their own provider console the same way.
• We fixed amounts at recording time instead of recomputing at current prices. Recomputation is simpler, since only the price table needs fixing. But limits already applied and last month's screens would then disagree with what was actually billed at the time.
• For personal-key usage we keep tokens and call counts instead of an estimated amount. Multiplying by product prices would let you compare everyone's usage as a single dollar figure. But that figure would not match what the key owner is billed, so the amount is left to each provider's console.
• Model prices stay in the config and the ledger and do not appear on any screen. The same model's cost per call changes from response to response with the processing tier and long-context charges, so a fixed price on screen would not match the real amount. Screens show totals of the amounts fixed in the ledger.

If your system mixes models from several providers and sets a cost limit per person, start by checking that a single function counts the tokens. Then back-calculate the actual cached input price for one day from the provider's cost API; it shows right away whether the ledger is counting cache tokens correctly.