All devices
24 GB VRAM · 768 GB/s · CUDA
24 GB VRAM · 768 GB/s · CUDAAI models for RTX A5000
Here are the best open models for each job on the RTX A5000, and every model's grade below.
Your hardware
…
- Memory
- GB
- Bandwidth
- GB/s
- Context
Most intelligent on your …
Intelligence (ECI) of the most intelligent models that load · colour shows how well they run
Expected speed
Tokens per second on your … for the most intelligent models that run well
Models available with more memory
Open models that run well (grade B or better) at each memory size
Scores from Oct 8, 2026 · 0 new models in the last two months
Every model on the RTX A5000
Q4_K_M, 8K context, 24 GB of memory. Change your setup above for exact numbers.
| Grade | Model | Memory | Speed |
|---|---|---|---|
| SExcellent |  GLM 4.7 Flash 31.2B | 17.8 GB | ~254 tok/s |
| SExcellent |  Qwen3 30B A3B Thinking 2507 30.5B | 18.3 GB | ~209 tok/s |
| SExcellent | Qwen3 30B A3B Instruct 2507 30.5B | 18.3 GB | ~209 tok/s |
| SExcellent |  North Mini Code 1.0 30.5B | 18.2 GB | ~223 tok/s |
| SExcellent |  Gemma 4 26B A4B 25.8B | 16.5 GB | ~180 tok/s |
| SExcellent |  gpt-oss 20b 20.9B | 11.3 GB | ~239 tok/s |
| SExcellent |  Phi-4 14B 14.7B | 10.1 GB | ~52 tok/s |
| SExcellent |  Ministral 3 14B 14B | 9.2 GB | ~56 tok/s |
| SExcellent | Gemma 4 12B 12B | 7.4 GB | ~67 tok/s |
| SExcellent | Qwen3.5 9B 9.7B | 5.8 GB | ~87 tok/s |
| SExcellent | Ministral 3 8B 8.9B | 6.2 GB | ~87 tok/s |
| SExcellent |  Granite 4.2 8B 8.8B | 6.7 GB | ~81 tok/s |
| SExcellent |  Llama 3.1 8B Instruct 8B | 5.9 GB | ~92 tok/s |
| SExcellent | Gemma 4 E4B 8B | 5.1 GB | ~99 tok/s |
| SExcellent |  Olmo 3 7B Instruct 7.3B | 7 GB | ~76 tok/s |
| SExcellent | Gemma 4 E2B 5.1B | 3.3 GB | ~160 tok/s |
| SExcellent | Qwen3.5 4B 4.7B | 3.1 GB | ~175 tok/s |
| SExcellent | Ministral 3 3B 3.9B | 3.1 GB | ~195 tok/s |
| SExcellent | Granite 4.2 3B 3.7B | 3.1 GB | ~190 tok/s |
| SExcellent | Llama 3.2 3B Instruct 3.2B | 3.1 GB | ~202 tok/s |
| SExcellent |  SmolLM3 3B 3.1B | 2.6 GB | ~227 tok/s |
| SExcellent |  LFM2.5 2.6B 2.7B | 2 GB | ~287 tok/s |
| SExcellent | Qwen3.5 2B 2.3B | 1.6 GB | ~378 tok/s |
| SExcellent | Qwen3.5 0.8B 0.9B | 0.9 GB | ~863 tok/s |
| AGood | Qwen3.8 27B 27.8B | 17 GB | ~28 tok/s |
| AGood | Qwen3.6 27B 27.8B | 16.5 GB | ~29 tok/s |
| AGood | Devstral Small 2 24B 24B | 14.9 GB | ~34 tok/s |
| AGood | Magistral Small 2509 24B | 14.9 GB | ~34 tok/s |
| AGood | Mistral Small 3.2 24B 24B | 14.9 GB | ~34 tok/s |
| AGood | Magistral Small 2506 23.6B | 14.9 GB | ~34 tok/s |
| BUsable | Qwen3.6 35B A3B 36B | 21.2 GB | ~259 tok/s |
| BUsable | Qwen3.5 35B A3B 36B | 21 GB | ~262 tok/s |
| BUsable | Olmo 3.1 32B Think 32.2B | 19.7 GB | ~24 tok/s |
| BUsable | Gemma 4 31B 31.3B | 18.8 GB | ~26 tok/s |
| BUsable | Granite 4.2 30B 29.3B | 19.1 GB | ~26 tok/s |
| CSlow |  Kimi Linear 48B A3B 49.1B | 28.4 GB | ~100 tok/s |
| CSlow |  Nemotron 3.5 Lightning 30B A3B 31.6B | 24.1 GB | ~147 tok/s |
| DBarely runs | Llama 3.3 70B Instruct 70.6B | 42.4 GB | ~2 tok/s |
36 models do not fit on the RTX A5000
| Grade | Model | Memory | Speed |
|---|---|---|---|
| FDoesn't fit | Kimi K3 2.8T | 1,584.5 GB | — |
| FDoesn't fit |  DeepSeek V4 Pro 0813 1.7T | 886.8 GB | — |
| FDoesn't fit | DeepSeek V4 Pro 1.6T | 886.8 GB | — |
| FDoesn't fit | Kimi K2.6 1T | 578.8 GB | — |
| FDoesn't fit | Kimi K2.7 Code 1T | 578.8 GB | — |
| FDoesn't fit | Kimi K2.5 1T | 579.4 GB | — |
| FDoesn't fit | Kimi K2 Thinking 1T | 579.4 GB | — |
| FDoesn't fit | Kimi K2 Instruct 1T | 579 GB | — |
| FDoesn't fit | tInkling 952.4B | 534 GB | — |
| FDoesn't fit | DeepSeek V4.1 Flash 763.2B | 435.8 GB | — |
| FDoesn't fit | GLM 5.1 753.9B | 429.2 GB | — |
| FDoesn't fit | GLM 5 753.9B | 425.7 GB | — |
| FDoesn't fit | GLM 5.2 753.3B | 424.5 GB | — |
| FDoesn't fit | DeepSeek V3.2 Exp 685.4B | 391.5 GB | — |
| FDoesn't fit | DeepSeek V3.2 685.4B | 378.5 GB | — |
| FDoesn't fit | DeepSeek V3.1 684.5B | 378.4 GB | — |
| FDoesn't fit | Nemotron 3 Ultra 550B A55B 560.5B | 321.8 GB | — |
| FDoesn't fit |  MiniMax M3 427B | 244.6 GB | — |
| FDoesn't fit | Qwen3.5 397B A17B 403.4B | 227.9 GB | — |
| FDoesn't fit | GLM 4.7 358.3B | 204.8 GB | — |
| FDoesn't fit | GLM 5.3 Flash 321.3B | 176.1 GB | — |
| FDoesn't fit | DeepSeek V4 Flash 0731 304.2B | 161 GB | — |
| FDoesn't fit | DeepSeek V4 Flash 290.9B | 166.7 GB | — |
| FDoesn't fit | tInkling Small 266B | 149 GB | — |
| FDoesn't fit | Qwen3 235B A22B Thinking 2507 235.1B | 134.2 GB | — |
| FDoesn't fit | Qwen3 235B A22B Instruct 2507 235.1B | 134.2 GB | — |
| FDoesn't fit | MiniMax M2.5 228.7B | 131.1 GB | — |
| FDoesn't fit | MiniMax M2.7 228.7B | 131.5 GB | — |
| FDoesn't fit | Command A+ 218.8B | 126.7 GB | — |
| FDoesn't fit | Qwen3.8 Flash Next 180B | 111.9 GB | — |
| FDoesn't fit | Mistral Medium 3.5 128B 127.7B | 72.8 GB | — |
| FDoesn't fit | Qwen3.5 122B A10B 125.1B | 71.8 GB | — |
| FDoesn't fit | Mistral Small 4 119B 119.4B | 68.1 GB | — |
| FDoesn't fit | gpt-oss 120b 116.8B | 59 GB | — |
| FDoesn't fit | Llama 4 Scout 17B 16E 108.6B | 62.7 GB | — |
| FDoesn't fit | Qwen3 Coder Next 79.7B | 45.7 GB | — |