All devices
40 GB VRAM · 1555 GB/s · CUDA
40 GB VRAM · 1555 GB/s · CUDAAI models for A100 40GB (PCIe)
Here are the best open models for each job on the A100 40GB (PCIe), and every model's grade below.
Your hardware
…
- Memory
- GB
- Bandwidth
- GB/s
- Context
Most intelligent on your …
Intelligence (ECI) of the most intelligent models that load · colour shows how well they run
Expected speed
Tokens per second on your … for the most intelligent models that run well
Models available with more memory
Open models that run well (grade B or better) at each memory size
Scores from Oct 8, 2026 · 0 new models in the last two months
Every model on the A100 40GB (PCIe)
Q4_K_M, 8K context, 40 GB of memory. Change your setup above for exact numbers.
| Grade | Model | Memory | Speed |
|---|---|---|---|
| SExcellent |  Kimi Linear 48B A3B 49.1B | 28.4 GB | ~545 tok/s |
| SExcellent |  Qwen3.6 35B A3B 36B | 21.2 GB | ~524 tok/s |
| SExcellent | Qwen3.5 35B A3B 36B | 21 GB | ~530 tok/s |
| SExcellent |  Olmo 3.1 32B Think 32.2B | 19.7 GB | ~50 tok/s |
| SExcellent |  Nemotron 3.5 Lightning 30B A3B 31.6B | 24.1 GB | ~417 tok/s |
| SExcellent |  Gemma 4 31B 31.3B | 18.8 GB | ~52 tok/s |
| SExcellent |  GLM 4.7 Flash 31.2B | 17.8 GB | ~514 tok/s |
| SExcellent | Qwen3 30B A3B Thinking 2507 30.5B | 18.3 GB | ~423 tok/s |
| SExcellent | Qwen3 30B A3B Instruct 2507 30.5B | 18.3 GB | ~423 tok/s |
| SExcellent |  North Mini Code 1.0 30.5B | 18.2 GB | ~452 tok/s |
| SExcellent |  Granite 4.2 30B 29.3B | 19.1 GB | ~53 tok/s |
| SExcellent | Qwen3.8 27B 27.8B | 17 GB | ~58 tok/s |
| SExcellent | Qwen3.6 27B 27.8B | 16.5 GB | ~60 tok/s |
| SExcellent | Gemma 4 26B A4B 25.8B | 16.5 GB | ~364 tok/s |
| SExcellent |  Devstral Small 2 24B 24B | 14.9 GB | ~68 tok/s |
| SExcellent | Magistral Small 2509 24B | 14.9 GB | ~68 tok/s |
| SExcellent | Mistral Small 3.2 24B 24B | 14.9 GB | ~68 tok/s |
| SExcellent | Magistral Small 2506 23.6B | 14.9 GB | ~68 tok/s |
| SExcellent |  gpt-oss 20b 20.9B | 11.3 GB | ~484 tok/s |
| SExcellent |  Phi-4 14B 14.7B | 10.1 GB | ~105 tok/s |
| SExcellent | Ministral 3 14B 14B | 9.2 GB | ~114 tok/s |
| SExcellent | Gemma 4 12B 12B | 7.4 GB | ~135 tok/s |
| SExcellent | Qwen3.5 9B 9.7B | 5.8 GB | ~175 tok/s |
| SExcellent | Ministral 3 8B 8.9B | 6.2 GB | ~177 tok/s |
| SExcellent | Granite 4.2 8B 8.8B | 6.7 GB | ~164 tok/s |
| SExcellent |  Llama 3.1 8B Instruct 8B | 5.9 GB | ~187 tok/s |
| SExcellent | Gemma 4 E4B 8B | 5.1 GB | ~201 tok/s |
| SExcellent | Olmo 3 7B Instruct 7.3B | 7 GB | ~154 tok/s |
| SExcellent | Gemma 4 E2B 5.1B | 3.3 GB | ~323 tok/s |
| SExcellent | Qwen3.5 4B 4.7B | 3.1 GB | ~354 tok/s |
| SExcellent | Ministral 3 3B 3.9B | 3.1 GB | ~394 tok/s |
| SExcellent | Granite 4.2 3B 3.7B | 3.1 GB | ~384 tok/s |
| SExcellent | Llama 3.2 3B Instruct 3.2B | 3.1 GB | ~409 tok/s |
| SExcellent |  SmolLM3 3B 3.1B | 2.6 GB | ~459 tok/s |
| SExcellent |  LFM2.5 2.6B 2.7B | 2 GB | ~582 tok/s |
| SExcellent | Qwen3.5 2B 2.3B | 1.6 GB | ~765 tok/s |
| SExcellent | Qwen3.5 0.8B 0.9B | 0.9 GB | ~1,748 tok/s |
| CSlow | gpt-oss 120b 116.8B | 59 GB | ~47 tok/s |
| CSlow | Qwen3 Coder Next 79.7B | 45.7 GB | ~138 tok/s |
| CSlow | Llama 3.3 70B Instruct 70.6B | 42.4 GB | ~9 tok/s |
34 models do not fit on the A100 40GB (PCIe)
| Grade | Model | Memory | Speed |
|---|---|---|---|
| FDoesn't fit | Kimi K3 2.8T | 1,584.5 GB | — |
| FDoesn't fit |  DeepSeek V4 Pro 0813 1.7T | 886.8 GB | — |
| FDoesn't fit | DeepSeek V4 Pro 1.6T | 886.8 GB | — |
| FDoesn't fit | Kimi K2.6 1T | 578.8 GB | — |
| FDoesn't fit | Kimi K2.7 Code 1T | 578.8 GB | — |
| FDoesn't fit | Kimi K2.5 1T | 579.4 GB | — |
| FDoesn't fit | Kimi K2 Thinking 1T | 579.4 GB | — |
| FDoesn't fit | Kimi K2 Instruct 1T | 579 GB | — |
| FDoesn't fit | tInkling 952.4B | 534 GB | — |
| FDoesn't fit | DeepSeek V4.1 Flash 763.2B | 435.8 GB | — |
| FDoesn't fit | GLM 5.1 753.9B | 429.2 GB | — |
| FDoesn't fit | GLM 5 753.9B | 425.7 GB | — |
| FDoesn't fit | GLM 5.2 753.3B | 424.5 GB | — |
| FDoesn't fit | DeepSeek V3.2 Exp 685.4B | 391.5 GB | — |
| FDoesn't fit | DeepSeek V3.2 685.4B | 378.5 GB | — |
| FDoesn't fit | DeepSeek V3.1 684.5B | 378.4 GB | — |
| FDoesn't fit | Nemotron 3 Ultra 550B A55B 560.5B | 321.8 GB | — |
| FDoesn't fit |  MiniMax M3 427B | 244.6 GB | — |
| FDoesn't fit | Qwen3.5 397B A17B 403.4B | 227.9 GB | — |
| FDoesn't fit | GLM 4.7 358.3B | 204.8 GB | — |
| FDoesn't fit | GLM 5.3 Flash 321.3B | 176.1 GB | — |
| FDoesn't fit | DeepSeek V4 Flash 0731 304.2B | 161 GB | — |
| FDoesn't fit | DeepSeek V4 Flash 290.9B | 166.7 GB | — |
| FDoesn't fit | tInkling Small 266B | 149 GB | — |
| FDoesn't fit | Qwen3 235B A22B Thinking 2507 235.1B | 134.2 GB | — |
| FDoesn't fit | Qwen3 235B A22B Instruct 2507 235.1B | 134.2 GB | — |
| FDoesn't fit | MiniMax M2.5 228.7B | 131.1 GB | — |
| FDoesn't fit | MiniMax M2.7 228.7B | 131.5 GB | — |
| FDoesn't fit | Command A+ 218.8B | 126.7 GB | — |
| FDoesn't fit | Qwen3.8 Flash Next 180B | 111.9 GB | — |
| FDoesn't fit | Mistral Medium 3.5 128B 127.7B | 72.8 GB | — |
| FDoesn't fit | Qwen3.5 122B A10B 125.1B | 71.8 GB | — |
| FDoesn't fit | Mistral Small 4 119B 119.4B | 68.1 GB | — |
| FDoesn't fit | Llama 4 Scout 17B 16E 108.6B | 62.7 GB | — |