How these numbers are calculated
Every figure on this site is arithmetic on published specifications. None of it is generated by a language model, and none of it is a benchmark.
Weights
Weight memory is parameter count times bits per parameter. The bits per parameter for each quantisation are the effective GGUF rates, which are slightly above the nominal value because embedding and output tensors are usually kept at higher precision:
| Quantisation | Bits/param |
|---|---|
| FP16 | 16 |
| Q8_0 | 8.5 |
| Q6_K | 6.6 |
| Q5_K_M | 5.7 |
| Q4_K_M | 4.9 |
| Q3_K_M | 3.9 |
| Q2_K | 3 |
So Llama 3.1 8B at Q4_K_M is 8.03 × 10⁹ × 4.9 ÷ 8 = 4.6 GiB, which matches the published file size.
KV cache
The KV cache stores one key and one value vector per layer per token. Its size is 2 × layers × kv_heads × head_dim × 2 bytes per token, which is a fixed property of the architecture. Grouped-query attention is why a 7B Qwen has a far smaller cache than a 9B Gemma.
Multiply that per-token figure by your context length. At 128K context the cache can be larger than the weights.
Runtime overhead
Compute buffers, activations and the inference process itself. Modelled as 0.5 GiB + 5% of weights + 0.3 GiB per 8K of context — a reasonable approximation of what llama.cpp and MLX actually allocate.
Usable memory
macOS, the window server and your open apps need memory too, and Metal applies a default wired-memory limit. We reserve 15% of unified memory, floored at 3 GiB and capped at 16 GiB. On a 16GB Mac that leaves 13.0 GiB; on a 64GB Mac, 54.4 GiB.
You can push past this with sudo sysctl iogpu.wired_limit_mb=N, which is why pages marked "tight" are often still worth trying.
Token speed
Generating a token requires reading every active weight from memory once. That makes generation memory-bandwidth bound, not compute bound — which is why an M4 Max at 546 GB/s is roughly four times faster than an M4 at 120 GB/s on the same model, despite a much smaller gap in GPU cores.
The estimate is bandwidth ÷ active weight size × efficiency, with efficiency at 0.80 for dense models and 0.50 for mixture-of-experts models, whose sparse gather patterns use bandwidth less effectively.
What this does not model
- Prompt processing speed, which is compute bound and depends on GPU core count.
- Thermal throttling. A MacBook Air under sustained load will fall below these numbers; a Mac Studio will not.
- Quality. Fitting is not the same as being good. A model that barely fits at Q2 is usually worse than a smaller one at Q4.
- Speculative decoding, batching or draft models, all of which can beat these figures.
Treat everything here as a well-founded estimate that tells you what to try, not a benchmark result. The calculation lives in lib/calc.js in the repository and is about a hundred lines — read it if you want to check the assumptions.