Can an M1 16GB Mac run Gemma 3 12B?
M1 · 16 GB unified memory · 68 GB/s · MacBook Air (M1), Mac mini (M1)
Gemma 3 12B at Q4_K_M needs about 11.1 GiB against roughly 13.0 GiB usable — that is 85% of your budget. It will load, but close your browser first, keep the context short, and expect memory pressure. Around 7 tokens/sec.
Where the memory goes
Three things occupy memory when a model is loaded: the weights themselves, the KV cache that holds the conversation, and the runtime's own working buffers. Only the first is fixed — the KV cache grows with every token in your context.
| Quantisation | Weights | Total | Tok/s | Fits? |
|---|---|---|---|---|
| FP16 | 22.7 GiB | 27.7 GiB | 2 | No |
| Q8_0 | 12.1 GiB | 16.5 GiB | 4 | No |
| Q6_K | 9.4 GiB | 13.6 GiB | 5 | No |
| Q5_K_M | 8.1 GiB | 12.3 GiB | 6 | Yes, but tight |
| Q4_K_M | 7.0 GiB | 11.1 GiB | 7 | Yes, but tight |
| Q3_K_M | 5.5 GiB | 9.6 GiB | 9 | Yes, comfortably |
| Q2_K | 4.3 GiB | 8.3 GiB | 12 | Yes, comfortably |
The default. Best quality-per-gigabyte for most people.
How long a context fits
The KV cache is often what breaks a setup that looked fine at load time. Quantising it to 8-bit roughly halves its footprint with little measurable quality cost.
| Context | KV cache | Total | FP16 KV | Q8 KV |
|---|---|---|---|---|
| 2K | 0.8 GiB | 8.6 GiB | Yes, comfortably | Yes, comfortably |
| 4K | 1.5 GiB | 9.5 GiB | Yes, comfortably | Yes, comfortably |
| 8K | 3.0 GiB | 11.1 GiB | Yes, but tight | Yes, comfortably |
| 16K | 6.0 GiB | 14.4 GiB | No | Yes, but tight |
| 32K | 12.0 GiB | 21.0 GiB | No | No |
| 128K | 48.0 GiB | 60.6 GiB | No | No |
What to run instead on an M1 16GB Mac
These are the largest models that fit comfortably on this machine at Q4_K_M with an 8K context.
| Model | Size | Total | Tok/s | |
|---|---|---|---|---|
| Qwen3 14B | 14.8B | 10.9 GiB | 6 | Yes, comfortably |
| Gemma 2 9B | 9.24B | 9.0 GiB | 10 | Yes, comfortably |
| Qwen3 8B | 8.2B | 6.8 GiB | 11 | Yes, comfortably |
| Llama 3.1 8B | 8.03B | 6.6 GiB | 11 | Yes, comfortably |
| DeepSeek-R1-Distill-Qwen-7B | 7.62B | 5.8 GiB | 12 | Yes, comfortably |
| Qwen2.5 7B | 7.62B | 5.8 GiB | 12 | Yes, comfortably |
How to run it
Ollama is the shortest path. It picks a quantisation automatically — usually Q4_K_M, which is what the numbers above assume.
brew install ollama
ollama serve &
ollama run gemma3:12bFor MLX — Apple's own array framework, typically a little faster than llama.cpp on Apple Silicon:
pip install mlx-lm
mlx_lm.generate --model mlx-community/Gemma-3-12B-4bit --prompt "Hello"sudo sysctl iogpu.wired_limit_mb=13926. This resets on reboot. Quantising the KV cache (--kv-cache-type q8_0 in llama.cpp) buys back 1.4 GiB.Macs that run Gemma 3 12B comfortably
The smallest configuration that handles it is the M3 Pro 18GB (MacBook Pro 14" (M3 Pro)).
| Mac | Memory | Bandwidth | Tok/s |
|---|---|---|---|
| M3 Pro 18GB | 18 GB | 150 GB/s | 16 |
| M2 24GB | 24 GB | 100 GB/s | 11 |
| M4 24GB | 24 GB | 120 GB/s | 13 |
| M4 Pro 24GB | 24 GB | 273 GB/s | 29 |
| M1 Max 32GB | 32 GB | 400 GB/s | 43 |
| M2 Max 32GB | 32 GB | 400 GB/s | 43 |
| M4 32GB | 32 GB | 120 GB/s | 13 |
| M3 Max 36GB | 36 GB | 300 GB/s | 32 |
| M4 Max 36GB | 36 GB | 410 GB/s | 44 |
| M4 Pro 48GB | 48 GB | 273 GB/s | 29 |
| M4 Max 48GB | 48 GB | 546 GB/s | 58 |
| M1 Max 64GB | 64 GB | 400 GB/s | 43 |
Common questions
- Can an M1 16GB Mac run Gemma 3 12B?
- Yes. At Q4_K_M it needs about 11.1 GiB of the roughly 13.0 GiB available, generating around 7 tokens per second.
- How fast is Gemma 3 12B on an Mac?
- Around 7 tokens per second. Token generation on Apple Silicon is limited by memory bandwidth, and this machine has 68 GB/s.
- How much memory does Gemma 3 12B need?
- 7.0 GiB for the weights at Q4_K_M, plus 3.0 GiB for an 8K-token KV cache and about 1.1 GiB of runtime overhead — 11.1 GiB in total.