Best local AI models for 32 GB of VRAM

37 open models load in 32 GB of graphics memory at Q4_K_M with an 8K context; 15 of them have an independent intelligence score. Here they are ranked, with the longest context each can take.

Top picks for 32 GB

#1 · ECI 149.4

Qwen3.8 27B

Memory: 17 GB
Max context: 128K
Speed: ~66 t/s

#2 · ECI 146.5

Qwen3.6 27B

Memory: 16.5 GB
Max context: 128K
Speed: ~69 t/s

#3 · ECI 143.9

Qwen3.6 35B A3B

Memory: 21.2 GB
Max context: 256K
Speed: ~604 t/s

Speeds on RTX 5090, the newest NVIDIA card with this much memory. Max context is the longest that still loads without spilling into system RAM.

Every scored model that fits 32 GB

#	Model	ECI	Memory	Max context	Speed
1	Qwen3.8 27B	149.4	17 GB	128K	~66 t/s
2	Qwen3.6 27B	146.5	16.5 GB	128K	~69 t/s
3	Qwen3.6 35B A3B	143.9	21.2 GB	256K	~604 t/s
4	Gemma 4 31B	142.7	18.8 GB	128K	~60 t/s
5	Qwen3.5 35B A3B	142.5	21 GB	256K	~611 t/s
6	Gemma 4 26B A4B	141.9	16.5 GB	256K	~419 t/s
7	Qwen3 30B A3B Thinking 2507	139.6	18.3 GB	128K	~487 t/s
8	Qwen3.5 9B	139.5	5.8 GB	256K	~202 t/s
9	gpt-oss 20b	137.8	11.3 GB	128K	~558 t/s
10	Qwen3 30B A3B 2507	137.4	18.3 GB	128K	~487 t/s
11	Magistral Small 2506	133.2	14.9 GB	32K	~78 t/s
12	Mistral Small 3.2 24B	131.7	14.9 GB	64K	~78 t/s
13	Magistral Small 2509	131.4	14.9 GB	64K	~78 t/s
14	Phi-4 14B	130.4	10.1 GB	16K	~121 t/s
15	Llama 3.1 8B	116.6	5.9 GB	128K	~215 t/s