Skip to content

Local coding LLM hardware fit

Best Local LLMs for Coding by Hardware

These recommendations start with models already tagged for coding in the AIPCFit catalog, then use practical memory fit, GGUF quantization, VRAM tiers, and hardware compatibility signals. The best local LLM for coding here means the best practical local fit for programming work on your hardware, not a coding benchmark leaderboard.

Catalog starting points

A coding tag is a discovery filter, not proof of superior coding performance. Shortlist reasons link to publisher model cards; memory estimates come from AIPCFit's fit rules.

Fit before preference

VRAM, quantization, model size, load memory, and runtime overhead determine whether a local LLM for programming is a practical fit.

Speed is separate

A model can fit in memory without being the right speed or quality match for every codebase, runtime, or coding task.

Coding models

Coding model recommendations

Models are grouped by the catalog size class, not by claimed code intelligence. Use this section to compare local coding models by provider, parameter count, quantization options, and available VRAM fit links.

Lightweight coding models

Smaller coding-tagged models that can be practical on lower-memory local systems.

Mid-size coding models

Coding-tagged models in the catalog's medium size class.

Larger coding models

Higher-parameter coding-tagged models where fit depends more heavily on VRAM, quantization, and offload behavior.

Larger

gpt-oss 20B

Provider
OpenAI
Parameters
21B
Quantizations
MXFP4
Use case
Coding
No highlighted VRAM tier yet

Very Large

Kimi K2 Instruct 0905

Provider
Moonshot AI
Parameters
1000B
Quantizations
Q4_K_M
Use case
Coding
No highlighted VRAM tier yet

Very Large

Kimi K2 Thinking

Provider
Moonshot AI
Parameters
1000B
Quantizations
Q4_K_M
Use case
Coding
No highlighted VRAM tier yet

Coding by VRAM

Best coding LLMs by VRAM tier

Each tier highlights distinct configurations for memory headroom, weight precision, or model architecture, then checks them with the existing fit rules. These use a synthetic NVIDIA GPU on Windows 11 with at least 32GB system RAM (or twice the VRAM tier, if larger). Planning memory includes generic headroom, not a KV-cache calculation at a guaranteed context length. The order is editorial, not a quality ranking.

Hardware tier

8GB VRAM

Open VRAM guide

Start with compact weights and short prompts. Compare a small instruction model, an 8B general model, and a reasoning distill while leaving room for the KV cache.

  • Phi-4 Mini · Q4_K_M · Good fit · 4.2 GB planning memory

    A compact starting point for a constrained GPU. Microsoft's card describes a model intended for limited-memory environments; test it on your own code tasks.

    Publisher model card
  • Qwen3 8B · Q4_K_M · Good fit · 7 GB planning memory

    An 8B alternative with documented thinking and non-thinking modes. The Q4 configuration leaves more memory than its Q5 variant on this tier.

    Publisher model card
  • DeepSeek-R1 Distill Qwen 7B · Q4_K_M · Good fit · 6.1 GB planning memory

    A reasoning-distilled alternative for comparing explanations and debugging prompts. Its reasoning output also consumes the available token budget.

    Publisher model card

Hardware tier

12GB VRAM

Open VRAM guide

Compare an 8B model at Q5 with a tighter 14B Q4 configuration. Keep a compact alternative when context and other GPU tasks matter more than parameter count.

  • Qwen3 8B · Q5_K_M · Good fit · 7.8 GB planning memory

    An 8B starting point at higher weight precision than Q4. It leaves more planning headroom than the 14B option; this is not evidence of better code output.

    Publisher model card
  • Qwen3 14B · Q4_K_M · Fits · 12.2 GB planning memory

    A larger same-family comparison with documented thinking modes. Planning headroom is tight at 12GB, so keep context short and verify whether any layers offload.

    Publisher model card

Hardware tier

16GB VRAM

Open VRAM guide

Use the extra capacity to compare 14B models or retain more context with an 8B model. A larger parameter count still needs a task-specific quality check.

  • Qwen3 14B · Q5_K_M · Good fit · 13.6 GB planning memory

    A higher-precision alternative to the 12GB tier's 14B Q4 candidate, with less headroom than a smaller model.

    Publisher model card
  • DeepSeek-R1 Distill Qwen 14B · Q4_K_M · Good fit · 10.9 GB planning memory

    A 14B reasoning-distilled candidate for comparing step-by-step debugging with Qwen3 on your own tests.

    Publisher model card
  • Qwen3 8B · Q5_K_M · Good fit · 7.8 GB planning memory

    Keep the smaller configuration when additional prompt context matters more than loading a 14B model.

    Publisher model card

Hardware tier

24GB VRAM

Open VRAM guide

This tier opens larger dense and mixture-of-experts configurations. Active expert counts do not replace total weight storage when estimating memory.

  • Qwen3 30B-A3B · Q4_K_M · Fits · 25.7 GB planning memory

    A mixture-of-experts alternative. Its model card distinguishes total and activated parameters; all resident weights still need storage.

    Publisher model card
  • Mistral Small 3.2 24B · Q4_K_M · Good fit · 20.3 GB planning memory

    A dense instruction-model alternative to the MoE configuration. Compare code edits with your tests; multimodal support is not evaluated by the memory rules.

    Publisher model card
  • Qwen3 14B · Q5_K_M · Good fit · 13.6 GB planning memory

    A smaller choice when you want to reserve more of this tier for context or other applications.

    Publisher model card

Hardware tier

32GB+ VRAM

Open VRAM guide

Compare a 32B reasoning distill with larger MoE or dense models. More VRAM can also be spent on context instead of on a larger weight file.

  • DeepSeek-R1 Distill Qwen 32B · Q4_K_M · Good fit · 27 GB planning memory

    A larger reasoning-distilled configuration that becomes a practical memory candidate here. Publisher evaluations do not measure this quantized setup on your GPU.

    Publisher model card
  • Qwen3 30B-A3B · Q4_K_M · Good fit · 25.7 GB planning memory

    Retain the MoE candidate with more room beyond its weights than at 24GB; verify actual context allocations in your runtime.

    Publisher model card
  • Mistral Small 3.2 24B · Q8_0 · Fits · 35.1 GB planning memory

    A higher-precision dense alternative. This is a tight planning fit, so compare it with Q4 before committing memory to longer prompts.

    Publisher model card

Choosing a local coding LLM for 12GB VRAM

Start by comparing Qwen3 8B Q4_K_M and Q5_K_M on the same coding task. Q4 leaves more room for context; Q5 stores weights at higher precision. Qwen3 14B Q4_K_M offers a larger-model comparison, but exceeds the catalog's recommended headroom on this tier. Phi-4 Mini Q8_0 is a compact alternative. None of these memory comparisons establishes which model writes better code.

12GB dedicated VRAM, Windows 11 NVIDIA profile, 32GB system RAM. All memory figures below are planning estimates except explicitly recorded files.
Model / weightsFile sizePlanning memoryFit
Qwen3 8BQ4_K_M5.0 GBCatalog-recorded7 GBGood fit
Qwen3 8BQ5_K_M5.8 GBEstimated7.8 GBGood fit
Qwen3 14BQ4_K_M9 GBCatalog-recorded12.2 GBFits

The catalog adds the larger of 2 GB or 35% of the weight size for recommended planning memory. Its minimum-load estimate uses the larger of 1 GB or 15%. These generic estimates do not reserve an exact KV cache. Begin testing at 4,096 context tokens with one request at a time, then increase context while watching actual memory. That setting is a suggested trial, not a measured fit guarantee. Long files, retrieved snippets, conversation history, and generated reasoning all use context.

A tight result may need shorter context or CPU offload. GPU-only operation and CPU/GPU offload can behave very differently. Keep your input, output budget, runtime version, and quantization fixed when comparing models. Try a small bug fix, a test-writing task, and a code explanation from your own repository, then run the generated tests and review the diff. Publisher evaluations of original models do not validate these local GGUF variants.

Selection guide

How to choose a local coding LLM

Start with model size and quantization. Smaller or lower-bit GGUF variants usually have lower memory requirements, while larger models need more VRAM, system RAM, or offloading headroom.

Check context length and runtime behavior for your coding workflow. Long files, multi-file prompts, retrieval workflows, and IDE integrations can change memory use beyond the model file size.

Fit and speed are separate questions. The fit engine can show whether a local llm for programming is practical for a memory tier, but coding quality and latency still vary by task, prompt, runtime, drivers, and software settings.

Setup paths

Coding setup links

AIPCFit has GPU-specific runtime pages rather than a single universal runtime score. Start with your GPU, then compare local model fit for Ollama, LM Studio, and compatibility pages.

Methodology

What this page does not claim

These are hardware-fit recommendations for local coding LLMs. They are not a coding benchmark leaderboard, model-quality ranking, or universal speed ranking.

Coding quality varies by programming language, repository size, prompt, runtime, context settings, and tool integration. A practical memory fit is the starting point, not the final answer for every developer workflow.