Can an M1 Pro 16GB Mac run Mixtral 8x7B?
M1 Pro · 16 GB unified memory · 200 GB/s · MacBook Pro 14" (M1 Pro), MacBook Pro 16" (M1 Pro)
Mixtral 8x7B needs about 29.8 GiB but an M1 Pro 16GB Mac has only ~13.0 GiB available for a model — you are 16.8 GiB short. macOS will swap to SSD and generation will crawl.
Where the memory goes
Three things occupy memory when a model is loaded: the weights themselves, the KV cache that holds the conversation, and the runtime's own working buffers. Only the first is fixed — the KV cache grows with every token in your context.
| Quantisation | Weights | Total | Tok/s | Fits? |
|---|---|---|---|---|
| FP16 | 87.0 GiB | 93.1 GiB | 4 | No |
| Q8_0 | 46.2 GiB | 50.3 GiB | 7 | No |
| Q6_K | 35.9 GiB | 39.5 GiB | 9 | No |
| Q5_K_M | 31.0 GiB | 34.3 GiB | 11 | No |
| Q4_K_M | 26.6 GiB | 29.8 GiB | 13 | No |
| Q3_K_M | 21.2 GiB | 24.1 GiB | 16 | No |
| Q2_K | 16.3 GiB | 18.9 GiB | 21 | No |
The default. Best quality-per-gigabyte for most people.
How long a context fits
The KV cache is often what breaks a setup that looked fine at load time. Quantising it to 8-bit roughly halves its footprint with little measurable quality cost.
| Context | KV cache | Total | FP16 KV | Q8 KV |
|---|---|---|---|---|
| 2K | 0.3 GiB | 28.8 GiB | No | No |
| 4K | 0.5 GiB | 29.1 GiB | No | No |
| 8K | 1.0 GiB | 29.8 GiB | No | No |
| 16K | 2.0 GiB | 31.1 GiB | No | No |
| 32K | 4.0 GiB | 33.7 GiB | No | No |
What to run instead on an M1 Pro 16GB Mac
These are the largest models that fit comfortably on this machine at Q4_K_M with an 8K context.
| Model | Size | Total | Tok/s | |
|---|---|---|---|---|
| Qwen3 14B | 14.8B | 10.9 GiB | 18 | Yes, comfortably |
| Gemma 2 9B | 9.24B | 9.0 GiB | 28 | Yes, comfortably |
| Qwen3 8B | 8.2B | 6.8 GiB | 32 | Yes, comfortably |
| Llama 3.1 8B | 8.03B | 6.6 GiB | 33 | Yes, comfortably |
| DeepSeek-R1-Distill-Qwen-7B | 7.62B | 5.8 GiB | 34 | Yes, comfortably |
| Qwen2.5 7B | 7.62B | 5.8 GiB | 34 | Yes, comfortably |
Macs that run Mixtral 8x7B comfortably
The smallest configuration that handles it is the M4 Pro 48GB (Mac mini (M4 Pro)).
| Mac | Memory | Bandwidth | Tok/s |
|---|---|---|---|
| M4 Pro 48GB | 48 GB | 273 GB/s | 17 |
| M4 Max 48GB | 48 GB | 546 GB/s | 35 |
| M1 Max 64GB | 64 GB | 400 GB/s | 25 |
| M2 Ultra 64GB | 64 GB | 800 GB/s | 51 |
| M4 Pro 64GB | 64 GB | 273 GB/s | 17 |
| M4 Max 64GB | 64 GB | 546 GB/s | 35 |
| M2 Max 96GB | 96 GB | 400 GB/s | 25 |
| M3 Ultra 96GB | 96 GB | 819 GB/s | 52 |
| M3 Max 128GB | 128 GB | 400 GB/s | 25 |
| M4 Max 128GB | 128 GB | 546 GB/s | 35 |
| M2 Ultra 192GB | 192 GB | 800 GB/s | 51 |
| M3 Ultra 256GB | 256 GB | 819 GB/s | 52 |
Common questions
- Can an M1 Pro 16GB Mac run Mixtral 8x7B?
- No. At Q4_K_M it needs about 29.8 GiB, and an M1 Pro 16GB Mac has roughly 13.0 GiB available for a model after macOS takes its share.
- How fast is Mixtral 8x7B on an Mac?
- Around 13 tokens per second. Token generation on Apple Silicon is limited by memory bandwidth, and this machine has 200 GB/s. Because this is a mixture-of-experts model only 12.9B of its 46.7B parameters are read per token, which is why it is faster than its size suggests.
- How much memory does Mixtral 8x7B need?
- 26.6 GiB for the weights at Q4_K_M, plus 1.0 GiB for an 8K-token KV cache and about 2.1 GiB of runtime overhead — 29.8 GiB in total.