Can an M1 8GB Mac run Mistral 7B v0.3?
M1 · 8 GB unified memory · 68 GB/s · MacBook Air (M1), MacBook Pro 13" (M1)
Mistral 7B v0.3 needs about 6.1 GiB but an M1 8GB Mac has only ~5.0 GiB available for a model — you are 1.1 GiB short. macOS will swap to SSD and generation will crawl. There is a smaller quantisation that fits: see below.
Where the memory goes
Three things occupy memory when a model is loaded: the weights themselves, the KV cache that holds the conversation, and the runtime's own working buffers. Only the first is fixed — the KV cache grows with every token in your context.
| Quantisation | Weights | Total | Tok/s | Fits? |
|---|---|---|---|---|
| FP16 | 13.5 GiB | 16.0 GiB | 4 | No |
| Q8_0 | 7.2 GiB | 9.3 GiB | 7 | No |
| Q6_K | 5.6 GiB | 7.6 GiB | 9 | No |
| Q5_K_M | 4.8 GiB | 6.9 GiB | 11 | No |
| Q4_K_M | 4.1 GiB | 6.1 GiB | 12 | No |
| Q3_K_M | 3.3 GiB | 5.3 GiB | 15 | No |
| Q2_K | 2.5 GiB | 4.5 GiB | 20 | Yes, but tight |
The default. Best quality-per-gigabyte for most people.
How long a context fits
The KV cache is often what breaks a setup that looked fine at load time. Quantising it to 8-bit roughly halves its footprint with little measurable quality cost.
| Context | KV cache | Total | FP16 KV | Q8 KV |
|---|---|---|---|---|
| 2K | 0.3 GiB | 5.2 GiB | No | No |
| 4K | 0.5 GiB | 5.5 GiB | No | No |
| 8K | 1.0 GiB | 6.1 GiB | No | No |
| 16K | 2.0 GiB | 7.4 GiB | No | No |
| 32K | 4.0 GiB | 10.0 GiB | No | No |
What to run instead on an M1 8GB Mac
These are the largest models that fit comfortably on this machine at Q4_K_M with an 8K context.
| Model | Size | Total | Tok/s | |
|---|---|---|---|---|
| Llama 3.2 3B | 3.21B | 3.6 GiB | 28 | Yes, comfortably |
| Qwen2.5 3B | 3.09B | 2.9 GiB | 29 | Yes, comfortably |
| Gemma 2 2B | 2.61B | 3.0 GiB | 34 | Yes, comfortably |
| Qwen2.5 1.5B | 1.54B | 1.9 GiB | 58 | Yes, comfortably |
| Llama 3.2 1B | 1.24B | 1.8 GiB | 72 | Yes, comfortably |
Macs that run Mistral 7B v0.3 comfortably
The smallest configuration that handles it is the M1 16GB (MacBook Air (M1)).
| Mac | Memory | Bandwidth | Tok/s |
|---|---|---|---|
| M1 16GB | 16 GB | 68 GB/s | 12 |
| M1 Pro 16GB | 16 GB | 200 GB/s | 36 |
| M2 16GB | 16 GB | 100 GB/s | 18 |
| M4 16GB | 16 GB | 120 GB/s | 22 |
| M3 Pro 18GB | 18 GB | 150 GB/s | 27 |
| M2 24GB | 24 GB | 100 GB/s | 18 |
| M4 24GB | 24 GB | 120 GB/s | 22 |
| M4 Pro 24GB | 24 GB | 273 GB/s | 49 |
| M1 Max 32GB | 32 GB | 400 GB/s | 72 |
| M2 Max 32GB | 32 GB | 400 GB/s | 72 |
| M4 32GB | 32 GB | 120 GB/s | 22 |
| M3 Max 36GB | 36 GB | 300 GB/s | 54 |
Common questions
- Can an M1 8GB Mac run Mistral 7B v0.3?
- No. At Q4_K_M it needs about 6.1 GiB, and an M1 8GB Mac has roughly 5.0 GiB available for a model after macOS takes its share.
- How fast is Mistral 7B v0.3 on an Mac?
- Around 12 tokens per second. Token generation on Apple Silicon is limited by memory bandwidth, and this machine has 68 GB/s.
- How much memory does Mistral 7B v0.3 need?
- 4.1 GiB for the weights at Q4_K_M, plus 1.0 GiB for an 8K-token KV cache and about 1.0 GiB of runtime overhead — 6.1 GiB in total.