Qwen2.5 1.5B on a Mac
Alibaba · 1.54B parameters · Apache 2.0 · released 2024-09
Weights @ Q4_K_M
0.9 GiB
the usual download size
KV cache
28 KB
per token, FP16
Max context
32K
tokens
Smallest Mac
8 GB
M1
Short answerA M1 8GB is the smallest Apple Silicon machine that loads Qwen2.5 1.5B at Q4_K_M with an 8K context, at roughly 58 tokens/sec. MacBook Air (M1) is the cheapest way to get that configuration.
Every Mac, ranked
| Mac | Memory | Needs | Tok/s | Verdict |
|---|---|---|---|---|
| M1 8GB | 8 GB | 1.9 GiB | 58 | Yes, comfortably |
| M1 16GB | 16 GB | 1.9 GiB | 58 | Yes, comfortably |
| M2 16GB | 16 GB | 1.9 GiB | 85 | Yes, comfortably |
| M4 16GB | 16 GB | 1.9 GiB | 102 | Yes, comfortably |
| M1 Pro 16GB | 16 GB | 1.9 GiB | 170 | Yes, comfortably |
| M3 Pro 18GB | 18 GB | 1.9 GiB | 127 | Yes, comfortably |
| M2 24GB | 24 GB | 1.9 GiB | 85 | Yes, comfortably |
| M4 24GB | 24 GB | 1.9 GiB | 102 | Yes, comfortably |
| M4 Pro 24GB | 24 GB | 1.9 GiB | 232 | Yes, comfortably |
| M4 32GB | 32 GB | 1.9 GiB | 102 | Yes, comfortably |
| M1 Max 32GB | 32 GB | 1.9 GiB | 339 | Yes, comfortably |
| M2 Max 32GB | 32 GB | 1.9 GiB | 339 | Yes, comfortably |
| M3 Max 36GB | 36 GB | 1.9 GiB | 254 | Yes, comfortably |
| M4 Max 36GB | 36 GB | 1.9 GiB | 348 | Yes, comfortably |
| M4 Pro 48GB | 48 GB | 1.9 GiB | 232 | Yes, comfortably |
| M4 Max 48GB | 48 GB | 1.9 GiB | 463 | Yes, comfortably |
| M4 Pro 64GB | 64 GB | 1.9 GiB | 232 | Yes, comfortably |
| M1 Max 64GB | 64 GB | 1.9 GiB | 339 | Yes, comfortably |
| M4 Max 64GB | 64 GB | 1.9 GiB | 463 | Yes, comfortably |
| M2 Ultra 64GB | 64 GB | 1.9 GiB | 679 | Yes, comfortably |
| M2 Max 96GB | 96 GB | 1.9 GiB | 339 | Yes, comfortably |
| M3 Ultra 96GB | 96 GB | 1.9 GiB | 695 | Yes, comfortably |
| M3 Max 128GB | 128 GB | 1.9 GiB | 339 | Yes, comfortably |
| M4 Max 128GB | 128 GB | 1.9 GiB | 463 | Yes, comfortably |
| M2 Ultra 192GB | 192 GB | 1.9 GiB | 679 | Yes, comfortably |
| M3 Ultra 256GB | 256 GB | 1.9 GiB | 695 | Yes, comfortably |
| M3 Ultra 512GB | 512 GB | 1.9 GiB | 695 | Yes, comfortably |
Download size by quantisation
Weight memory scales linearly with bits per parameter. Everything below assumes an 8K context on top.
| Quantisation | Weights | Total @ 8K | Notes |
|---|---|---|---|
| FP16 | 2.9 GiB | 4.0 GiB | Full precision. Reference quality, twice the memory of Q8. |
| Q8_0 | 1.5 GiB | 2.6 GiB | Indistinguishable from FP16 in practice, at half the size. |
| Q6_K | 1.2 GiB | 2.3 GiB | Near-lossless. A good stop when you have memory to spare. |
| Q5_K_M | 1.0 GiB | 2.1 GiB | Slightly better than Q4_K_M, noticeably bigger. |
| Q4_K_M | 0.9 GiB | 1.9 GiB | The default. Best quality-per-gigabyte for most people. |
| Q3_K_M | 0.7 GiB | 1.8 GiB | Visible quality loss. Use to squeeze one size class up. |
| Q2_K | 0.5 GiB | 1.6 GiB | Last resort. Often worse than a smaller model at Q4. |