8 GB VRAM guide
Best Local AI Models for 8 GB VRAM
Eight gigabytes is a strict local-AI tier: small models fit with useful headroom, while most 20B-class packages do not. This guide favors honest compatibility over aspirational model names.
Calculation baseline: 8,192 tokens, published weights + KV cache + 1.5 GB overhead. Comfortable means at least 20% headroom; tight means the estimate fits with less than 20% headroom. See every threshold and source.
Comfortable fit
Models with usable headroom at 8 GB.
| Model | Recommended quantization | Estimated requirement | Headroom | Ollama command |
|---|---|---|---|---|
| Gemma 3 4B 4B dense | Q4_K_M | 6.0 GB | +2.0 GB · 132% fit | ollama run gemma3:4b-it-q4_K_Mlms get "Gemma 3 4B"@q4_k_m |
| Phi-4 Mini 3.8B dense | Q4_K_M | 5.1 GB | +2.9 GB · 158% fit | ollama run phi4-mini:3.8b-q4_K_Mlms get "Phi-4 Mini"@q4_k_m |
| Llama 3.2 3B 3.2B dense | Q8_0 | 5.8 GB | +2.2 GB · 137% fit | ollama run llama3.2:3b-instruct-q8_0lms get "Llama 3.2 3B"@q8_0 |
Tight fit
Models that fit only with less headroom.
| Model | Recommended quantization | Estimated requirement | Headroom | Ollama command |
|---|---|---|---|---|
| Qwen3 8B 8.2B dense | Q4_K_M | 7.9 GB | +0.1 GB · 101% fit | ollama run qwen3:8b-q4_K_Mlms get "Qwen3 8B"@q4_k_m |
| DeepSeek R1 8B 8B dense | Q4_K_M | 7.5 GB | +0.5 GB · 107% fit | ollama run deepseek-r1:8b-llama-distill-q4_K_Mlms get "DeepSeek R1 8B"@q4_k_m |
| Gemma 4 E4B 4.5B effective / 8B stored | Q4_K_M | 7.7 GB | +0.3 GB · 103% fit | ollama run gemma4:e4b-it-q4_K_Mlms get "Gemma 4 E4B"@q4_k_m |
| Mistral 7B 7.2B dense | Q4_K_M | 7.0 GB | +1.0 GB · 115% fit | ollama run mistral:7b-instruct-v0.3-q4_K_Mlms get "Mistral 7B"@q4_k_m |
Quantization sensitivity
How lower-bit files change the fit at 8 GB.
This table keeps the same 8,192 token baseline and shows each published quantization for the five featured models. “Does not fit” is calculated, not a quality judgment.
| Model | Quantization | Requirement | Fit | Headroom | Context ceiling | Speed estimate | Ollama / LM Studio |
|---|---|---|---|---|---|---|---|
| Qwen3.6 35B-A3B | Q8_0 | 39.5 GB | too-large | -31.5 GB | 0 tokens | Estimate · Fast relative class | ollama run qwen3.6:35b-a3b-q8_0lms get "Qwen3.6 35B-A3B"@q8_0 |
| Qwen3.6 35B-A3B | Q4_K_M | 24.7 GB | too-large | -16.7 GB | 0 tokens | Estimate · Fast relative class | ollama run qwen3.6:35b-a3b-q4_K_Mlms get "Qwen3.6 35B-A3B"@q4_k_m |
| Qwen3.6 35B-A3B | MXFP4 | 24.3 GB | too-large | -16.3 GB | 0 tokens | Estimate · Fast relative class | ollama run qwen3.6:35b-a3blms get "Qwen3.6 35B-A3B"@mxfp4 |
| Qwen3.8 27B | Q8_0 | 32.0 GB | too-large | -24.0 GB | 0 tokens | Estimate · Heavy relative class | ollama run qwen3.8:27b-q8_0lms get "Qwen3.8 27B"@q8_0 |
| Qwen3.8 27B | Q4_K_M | 19.4 GB | too-large | -11.4 GB | 0 tokens | Estimate · Heavy relative class | ollama run qwen3.8:27b-q4_K_Mlms get "Qwen3.8 27B"@q4_k_m |
| gpt-oss-20b | MXFP4 | 13.8 GB | too-large | -5.8 GB | 0 tokens | Estimate · Fast relative class | ollama run gpt-oss:20blms get "gpt-oss-20b"@mxfp4 |
| Gemma 4 12B Unified | Q8_0 | 14.7 GB | too-large | -6.7 GB | 0 tokens | Estimate · Moderate relative class | ollama run gemma4:12b-it-q8_0lms get "Gemma 4 12B Unified"@q8_0 |
| Gemma 4 12B Unified | Q4_K_M | 9.2 GB | too-large | -1.2 GB | 0 tokens | Estimate · Moderate relative class | ollama run gemma4:12b-it-q4_K_Mlms get "Gemma 4 12B Unified"@q4_k_m |
| Gemma 4 12B Unified | Q4_0 | 9.1 GB | too-large | -1.1 GB | 0 tokens | Estimate · Moderate relative class | ollama run gemma4:12b-it-qatlms get "Gemma 4 12B Unified"@q4_0 |
| Gemma 4 E4B | Q8_0 | 11.0 GB | too-large | -3.0 GB | 0 tokens | Estimate · Fast relative class | ollama run gemma4:e4b-it-q8_0lms get "Gemma 4 E4B"@q8_0 |
| Gemma 4 E4B | Q4_K_M | 7.7 GB | tight | +0.3 GB | 16,016 tokens | Estimate · Fast relative class | ollama run gemma4:e4b-it-q4_K_Mlms get "Gemma 4 E4B"@q4_k_m |
| Gemma 4 E4B | Q4_0 | 7.9 GB | tight | +0.1 GB | 10,677 tokens | Estimate · Fast relative class | ollama run gemma4:e4b-it-qatlms get "Gemma 4 E4B"@q4_0 |
LM Studio: run lms get with the displayed quantization suffix, confirm the downloaded key with lms ls, then run lms load <model-key> --gpu max --context-length 8192.
If your model does not fit
Change the model, the tier, or the context before changing everything.
The cheapest path is usually a lower-bit file or shorter context. If the minimum requirement still exceeds 8 GB, compare the 12 GB guide and a model that currently clears this tier's comfort threshold.
Low-cost setup upgrades
Useful accessories for a 8 GB system
Storage, cooling, and power accessories can improve the setup, but none increases VRAM or makes an oversized model fit.
External NVMe SSD for model storage
Keeps large GGUF and MLX files off the boot drive. Fast storage reduces load time, but it does not increase the memory available for inference.
Search current listings on AmazonDesktop RAM upgrade kit
Useful for MoE CPU offload and keeping the operating system out of VRAM. Match the motherboard's memory type, slot layout, capacity limit, and validated part numbers.
Search current listings on AmazonGPU support bracket
A lower-cost mechanical upgrade for heavy three-slot cards. Check clearance, mounting height, and whether the bracket blocks fans or case panels.
Search current listings on AmazonAs an Amazon Associate I earn from qualifying purchases. Links are sponsored search links and do not change the price you pay. Affiliate disclosure.
FAQ
Questions about 8 GB VRAM.
Which local AI models fit 8 GB of VRAM comfortably?
At an 8,192 token context, the comfortable list includes Gemma 3 4B, Phi-4 Mini, Llama 3.2 3B. Each entry uses the highest-ranked published quantization that still clears the 20% comfort threshold.
Why are some models only a tight fit at 8 GB?
A tight result means the calculated requirement is at least the available capacity but below 120% of it. The current tight-only models are Qwen3 8B, DeepSeek R1 8B, Gemma 4 E4B, Mistral 7B. Close other GPU applications, reduce context, or choose a smaller quantization before relying on them.
What context length and memory overhead are used in these tables?
The guide uses 8,192 tokens, adds the model's published weights, a calculated FP16 KV cache, and 1.5 GB of runtime overhead. Longer contexts increase memory use and can move a model from comfortable to tight or too large.
Does a comfortable result guarantee good tokens per second?
No. Memory compatibility is not a speed benchmark. Compute performance, memory bandwidth, quantization kernels, backend support, CPU offload, and concurrent applications can all change generation speed.
What should I do if my chosen model does not fit in 8 GB?
First try the lowest published quantization and reduce context. If the model still misses the minimum requirement, compare the 12 GB guide, choose a smaller model, or use a Mac with more unified memory. Moving up one tier is only useful when the exact model and context also fit there.