~1 minIn draft
The real cost of an LLM in production: a working ledger
Token pricing is the smallest line item. A cost model for agentic workloads — context windows, retries, caching, evaluation — built the way finance would build it.
Everyone prices the happy path: tokens in, tokens out. Production agentic systems spend differently — context that grows with every hop, retries on malformed tool calls, evaluation runs, and the caching decisions that make or break the unit economics.
What this essay will cover
- The line items nobody budgets: retries, evaluation, growing context
- Why caching strategy is a finance decision wearing an engineering costume
- A working cost model you can hold to account, line by line
This essay is in draft. The full write-up lands here when it's ready.
LLM · Cost · Agentic Systems