Checks / performance
Full multi-head attention at LLM scale
Check R31. Runs in the editor as you build, in CI through the
GitHub Action, and over the wire at
POST /api/v1/check. Milliseconds, before any GPU is billed.
info
performance
R31
| Trigger | A transformer stack (≥6 attention layers) at LLM width (embedDim ≥ 2048) uses full multi-head attention everywhere, with no grouped-query or latent attention present. |
|---|---|
| Why | At real model scale the KV cache, not the weights, dominates serving memory. Full per-head K/V caching is what production LLMs move away from: grouped-query attention (e.g. 8:1) cuts the cache ~8×, and multi-head latent attention (MLA) shrinks it ~10× or more, without changing the parameter count. Advice, not a bug: set numKVHeads below numHeads, or switch to mla. |
| Source | Serving-cost roofline; the same KV-cache math the app and the public serving calculator run. |
Why it is not a lint you can ignore
A structural mistake does not fail at review time and it does not fail at import
time. It fails when the module is constructed on the training node, after the job was queued and
the dataset was downloaded. That is why this runs before the spend and not after it.
Run this check on your own model
Free, no account needed
Every check
41 structural checks: 6 guardrail gates and 35 architecture advisor rules. See the full catalogue.
← R30 Very large Linear layer · R32 Default init assumes ReLU, saturating activation follows →