What a model writes, before it costs anything

One model was asked for architectures in plain English, 4 tasks, no human in the loop. Every graph they returned went through the same pre-flight gate /api/v1/check applies, offline, about ten milliseconds each.

The author changed

Every distinct structure a model proposed was one our own deterministic generator cannot produce for the same sentence, at any scale. That share is not noisy and does not depend on which model: it is 100% in every run recorded here. Whatever else is true, the thing writing the architecture is no longer the thing we can enumerate.

How often a machine-written architecture will not run

This one is noisy, and it is quoted with its width for that reason. It ranges from 2.8% to 26.4% across the models below, and a single model moves by more than ten points between runs on the same four sentences. Anyone quoting one row of this table as "the refusal rate" is quoting a different claim than the data supports.

Refused means pre-flight blockers only: the graph does not forward-pass. It is not a judgement about whether a legal design is any good. We publish how bad we are at that separately, at /calibration.html.

Every refusal, and the run it would have started

13 of the 29 refusals below are the same check, structure-missing-params. A model that writes a residual or a dense block and does not make the branches agree is one specific, nameable habit, not a random defect rate, which is why it can be checked in ten milliseconds and why the same check keeps earning its place.

Quoted, this run (2026-09-07): $53.48 (1 refusals too broken to price, counted as zero)

Never add two rows of that table together. The sentences changed between runs, and a total across both is a figure about a task set that never existed.

The quote is pre-flight's own figure for each refused run at list GPU prices, the same figure the product shows for the same graph. It is what would have been dispatched, not a saving we assert, and a refusal that cannot be priced counts as zero rather than as a guess.

What this is not

• Not out-of-sample. 4 sentences, three models, three repeats each. The range in the table is the honest width of it.
• Not a measured bill. Every dollar here is pre-flight's quote for a run that did not happen, at list GPU prices, on the workload the sentence states. The refusal is measured; the price is arithmetic, and it scales with the run being refused rather than with the refusal, which is why the same check is worth 47 cents on a tutorial and four figures on a production workload.
• Not prevented spend in the product's sense. These graphs were refused when a model proposed them, not when a person pressed Train, and the counting rule we sell on deliberately excludes proposal-time refusals: six proposed and one trained means refusing five prevented nothing. What this page measures is what a machine author would have dispatched if nothing had checked it.
• Not a claim about accuracy. Nothing here says a design that passes will train well.
• Not a claim that a human could not have caught these. A dummy forward pass catches most of them too. The point is that no one runs a dummy forward pass when the author is a machine and the next step is a dispatch.

Machine-readable: /loop.json. Run npm run bench:proposer to reproduce. Newest run 2026-09-07.