Dedicated GPU memory guides
These Windows/NVIDIA guide pages use dedicated GPU memory as the primary fit budget and keep partial offload separate from a full GPU fit.
No conservative full-fit guide tier is available yet.
Moonshot AI · Kimi
Review available GGUF variants, quantization choices, and conservative memory-planning levels for running Kimi K2 Thinkinglocally.
| Variant | Quantization | Estimated file | Minimum load | Planning memory |
|---|---|---|---|---|
| Kimi K2 Thinking · Q4_K_M | Q4_K_M | 621 GB | 714.2 GB | 838.4 GB |
These Windows/NVIDIA guide pages use dedicated GPU memory as the primary fit budget and keep partial offload separate from a full GPU fit.
Apple Silicon pages use conservative unified-memory budgets and Metal-capable local inference terminology instead of treating the Mac as a CUDA workstation.
These are deterministic memory-planning checks, not benchmark claims. Context length, runtime settings, offload behavior, and concurrent apps can change practical fit.