The LLM bill nobody modelled

Most AI features are approved on a cost estimate produced during a pilot. That estimate is close to worthless, and not because someone got the forecast wrong. A pilot measures the price of a model call. What sets the bill is how many calls the design needs, and at pilot scale a good design and a bad one cost the same.

I know this because I built both. PureSignal is a job-matching system I designed, built and ran during 2026 as research within DSC. It reads new job adverts and checks each one against a person’s requirements: role, location, salary, seniority and trade-offs. About twenty candidates used it during a trial. Every figure below comes from its own logs and database.

Contents

1. The obvious build
2. The same output, built differently
3. Where the saving came from
4. The pilot bill tells you nothing
5. Cost follows sign-ups, value follows use
6. A handful is not a corpus
7. What to ask your CTO

The obvious build

The first version ran on my own server. For every new advert it found, it sent the advert and my requirements to a model and asked whether they matched. About thirty calls a day. The answers were good and the cost didn’t register.

That isn’t a foolish design. It’s what the documentation demonstrates, and for one person it’s the right design. The problem is the unit. Each call is one person’s requirements against one advert, so the volume is people multiplied by adverts.

Take a modest service: 100 people and 500 new adverts a day. That’s 50,000 comparisons a day, and every new person adds another 500, every day, for as long as they stay registered.

The trial showed the same thing with real numbers. Thirty calls a day was one set of requirements against the thirty or so adverts found each day. By the end of June the trial had 29 sets of requirements, and that month it collected nearly a thousand new adverts a day. The same design would have needed about 26,000 calls a day. Both numbers had risen at once, which is exactly what the multiplication predicts.

The same output, built differently

The version the candidates used produces the same result, a score with the reasons for and against, but gets there in layers. Each layer is cheaper than the one after it and removes work before the expensive step.

1. Read each advert once. When an advert arrives, a model extracts the facts that matter: title, salary, location, seniority, working pattern, industry. That’s one call per advert, however many people are registered. Requirements are stored in the same structured form.
2. Filter with ordinary database rules. Salary below the person’s floor, wrong country, excluded industry, too junior: out, with no model involved. This costs next to nothing at any scale.
3. Rank by meaning. Surviving adverts are compared with the person’s requirements using embeddings, a numerical fingerprint of the text, so that similar roles sit close together without anyone writing a rule for each. Weak matches drop out, and at most fifty go forward per run.
4. Only then ask the model. The remaining pairs get the full judgement: a score, an explanation and the trade-offs.

Between March and mid-September the system processed 60,224 adverts. Checking every advert against every set of requirements would have meant 1,396,690 model calls. The layered build made about 70,000 scoring calls, one in twenty. Add the one call per advert to read it, and the total is still under a tenth of the obvious build.

Where the saving came from

The usual advice is to choose a cheaper model, cache responses and batch requests. Here is what actually did the work, using the month from 15 August to 14 September, for which the system recorded every stage.

• 144,652 possible comparisons between new adverts and existing requirements.
• 9,914 survived the database filter. That one step removed 93%.
• 8,478 survived the similarity ranking, removing a further 1% of the original total.
• 8,478 went to the model, and 2,989 scored well enough to keep.

The database filter did almost all of the work, and it involves no AI at all. The similarity ranking’s real job is putting what’s left in order.

Model choice mattered, but differently. I started on a top-tier model and moved to a fast, low-cost one after testing showed the judgements held up. That cut the price of each call. It did nothing to the number of calls, and the number is where the large multiples are.

The system doesn’t cache model responses or batch requests. Both can help. Neither changes how many comparisons you decide to make.

One more detail matters later. Roughly two-thirds of model spend went on reading adverts rather than judging matches, and how many adverts get read depends on how many searches are running.

The pilot bill tells you nothing

Thirty calls a day produces a bill nobody reads. There’s nothing to alert on and no reason to look. That’s the trap: the design gets fixed during the pilot, when the cost is invisible.

I didn’t find the problem in a bill. I found it by doing the multiplication before there was one. The requirements document I wrote for the system included this line:

LLM spend per user must remain flat as scale grows.

A requirement like that only means something if it was inconvenient, and this one was. The obvious build was already working and giving good answers. The layered design needed a structured data model, an extraction step, embeddings, a vector index and a scheduler. I built and hardened them before anyone else had used the system, with no evidence yet that I’d need any of it. The only argument for it was arithmetic.

That trade is why this trap catches competent teams, not just careless ones. Noticing early costs weeks of design and build, spent at the moment the pilot is working, everyone wants to move on, and the bill is too small to justify any delay. Noticing late costs a rebuild of a system that by then has users, integrations and a production history. Those are not comparable amounts, but only the first is visible when the decision is made. A team that skips the early cost is responding sensibly to the numbers in front of it. They are the wrong numbers.

The fix isn’t clever. Most pilot estimates take the price per call, multiply by pilot volume and apply a growth assumption. The real driver is people multiplied by items multiplied by the share of pairs that reach a model. The first grows in line with usage. The second grows with the product of two numbers that both rise when the business is doing well.

Cost follows sign-ups, value follows use

In a pilot, everyone is keen. They volunteered, they log in, they read what you send them. So nobody designs for the person who signs up, sets their requirements and drifts away. The system doesn’t notice they’ve gone. It carries on searching and matching for them.

The trial data shows what that looks like:

• Eight people who used the system on five days or fewer set up the searches that collected 80% of the adverts found for candidates. Between them they opened fewer than one in ten of the jobs anyone read.
• The people who used it on more than five days opened 88% of them.
• Once new sign-ups tailed off, the pattern sharpened. In the month to 14 September, 86% of scoring calls served people who hadn’t been back for a month.

None of this is unusual. Someone finds a job and stops opening the emails. Someone else pauses their search. The point is that the cost doesn’t know. Spend follows how broad a person’s requirements are and how long they stay registered. Value follows whether they’re still reading.

The answer is a product rule rather than an engineering fix: slow down or pause the work for people who stop engaging, and resume when they come back. Pilots never reveal the need for that rule, because pilots are made of enthusiasts.

Most businesses running AI features will have a version of this: reports generated for users who no longer open them, documents summarised for lapsed accounts, agents working on behalf of customers who have left. It’s a quick thing to check.

A handful is not a corpus

The obvious build and the layered build aren’t two settings on the same system. They’re different systems.

A pilot proves the judgement is useful, which is worth knowing. It says nothing about the cost of making that judgement across a corpus, because the pilot never had to decide which comparisons to skip. A portfolio company that scales a pilot by adding capacity and a bigger budget line gets the obvious build at volume. Its estimate will be out by roughly the gap between the two designs, and nobody will have modelled it.

What to ask your CTO

You don’t need to know what an embedding is to ask these.

1. What does it cost per unit of value? Per useful result or per active user, not per call.
2. What share of requests reach a model at all, and what share reach the most expensive one? And why that share.
3. Show me the multiplication. Which two or three numbers drive call volume, and what happens to the bill when each goes up tenfold? The answer should be arithmetic, not a growth percentage applied to the pilot bill.
4. What removes work before the model sees it? And how much does each step remove?
5. How much of the spend serves people who haven’t used the product in the last month?
6. Who watches the spend line, and what makes an alert fire?

The third question matters most, because a pilot invoice can’t answer it. Model prices will keep falling. The multiplication won’t.