Can an M3 Ultra 256GB Mac run Llama 3.1 405B?
M3 Ultra · 256 GB unified memory · 819 GB/s · Mac Studio (M3 Ultra)
Llama 3.1 405B needs about 247.3 GiB but an M3 Ultra 256GB Mac has only ~240.0 GiB available for a model — you are 7.3 GiB short. macOS will swap to SSD and generation will crawl. There is a smaller quantisation that fits: see below.
Where the memory goes
Three things occupy memory when a model is loaded: the weights themselves, the KV cache that holds the conversation, and the runtime's own working buffers. Only the first is fixed — the KV cache grows with every token in your context.
| Quantisation | Weights | Total | Tok/s | Fits? |
|---|---|---|---|---|
| FP16 | 754.4 GiB | 796.8 GiB | 1 | No |
| Q8_0 | 400.8 GiB | 425.5 GiB | 2 | No |
| Q6_K | 311.2 GiB | 331.5 GiB | 2 | No |
| Q5_K_M | 268.7 GiB | 286.9 GiB | 2 | No |
| Q4_K_M | 231.0 GiB | 247.3 GiB | 3 | No |
| Q3_K_M | 183.9 GiB | 197.8 GiB | 3 | Yes, comfortably |
| Q2_K | 141.4 GiB | 153.3 GiB | 4 | Yes, comfortably |
The default. Best quality-per-gigabyte for most people.
How long a context fits
The KV cache is often what breaks a setup that looked fine at load time. Quantising it to 8-bit roughly halves its footprint with little measurable quality cost.
| Context | KV cache | Total | FP16 KV | Q8 KV |
|---|---|---|---|---|
| 2K | 1.0 GiB | 244.1 GiB | No | No |
| 4K | 2.0 GiB | 245.2 GiB | No | No |
| 8K | 3.9 GiB | 247.3 GiB | No | No |
| 16K | 7.9 GiB | 251.6 GiB | No | No |
| 32K | 15.8 GiB | 260.0 GiB | No | No |
| 128K | 63.0 GiB | 310.9 GiB | No | No |
What to run instead on an M3 Ultra 256GB Mac
These are the largest models that fit comfortably on this machine at Q4_K_M with an 8K context.
| Model | Size | Total | Tok/s | |
|---|---|---|---|---|
| DeepSeek V4 Flash Vision Exp | 304.65B | 183.9 GiB | 4 | Yes, comfortably |
| DeepSeek V4 Flash 0731 | 304.18B | 183.7 GiB | 4 | Yes, comfortably |
| DeepSeek V4 Flash | 290.94B | 175.7 GiB | 4 | Yes, comfortably |
| gpt-oss-120b | 116.8B | 71.3 GiB | 131 | Yes, comfortably |
| Qwen2.5 72B | 72.7B | 46.8 GiB | 15 | Yes, comfortably |
| DeepSeek-R1-Distill-Llama-70B | 70.6B | 45.6 GiB | 15 | Yes, comfortably |
Macs that run Llama 3.1 405B comfortably
The smallest configuration that handles it is the M3 Ultra 512GB (Mac Studio (M3 Ultra)).
| Mac | Memory | Bandwidth | Tok/s |
|---|---|---|---|
| M3 Ultra 512GB | 512 GB | 819 GB/s | 3 |
Common questions
- Can an M3 Ultra 256GB Mac run Llama 3.1 405B?
- No. At Q4_K_M it needs about 247.3 GiB, and an M3 Ultra 256GB Mac has roughly 240.0 GiB available for a model after macOS takes its share.
- How fast is Llama 3.1 405B on an Mac?
- Around 3 tokens per second. Token generation on Apple Silicon is limited by memory bandwidth, and this machine has 819 GB/s.
- How much memory does Llama 3.1 405B need?
- 231.0 GiB for the weights at Q4_K_M, plus 3.9 GiB for an 8K-token KV cache and about 12.4 GiB of runtime overhead — 247.3 GiB in total.