Five calculations that decide what an LLM deployment costs and how fast it feels. Everything below runs in this page — no server, no GPU, no model download. The same arithmetic is in the Python package, and its test suite asserts these figures, so if the two ever disagree one of them is wrong.
Peak arithmetic rate divided by memory bandwidth gives the ridge point: the operations-per-byte below which you are paying to move bytes rather than to multiply them. In decode, your arithmetic intensity is roughly your batch size.
Intensity is batch × (2 ÷ bytes-per-weight), so it equals the batch only at 2-byte weights. Peak figures are dense: vendors often publish the with-sparsity number, which is double.
Two, times the layers whose cache grows, times the key-value heads, times the
head dimension, times the bytes per number. Every value is in the model's
config.json.
Read layer_types before you trust the layer count.
Models that interleave sliding-window attention only grow the cache on their full-attention
layers. gpt-oss-120b has 36 layers and 18 of them grow, so using 36 doubles your answer.
Total memory, minus the weights, divided by what one conversation costs. This belongs in every capacity plan and is almost never in one.
Uses the cache size from calculator 2 above, so change the model there and
this follows. Utilisation defaults to vLLM's gpu_memory_utilization of 0.92, and
5 GiB is held back for activations and CUDA graphs.
A kernel that runs twice as fast only helps in proportion to how much of your time was spent in it. The speedup you were quoted was measured on somebody else's workload.
Convert the percentages back into items before you believe anything. A dozen items is not a finding, however large the percentage gap looks.
Overlapping intervals prove nothing. Non-overlap implies significance; overlap does not imply its absence. If both models were scored on the same items the correct test is McNemar's, which needs to know which items each one got wrong — and a published percentage never carries that.