Small
Phi-3.5 Mini
- Provider
- Microsoft
- Parameters
- 3.8B
- Quantizations
- Q4_K_M
- Use case
- Coding
Local coding LLM hardware fit
These recommendations start with models already tagged for coding in the AIPCFit catalog, then use practical memory fit, GGUF quantization, VRAM tiers, and hardware compatibility signals. The best local LLM for coding here means the best practical local fit for programming work on your hardware, not a coding benchmark leaderboard.
Catalog starting points
A coding tag is a discovery filter, not proof of superior coding performance. Shortlist reasons link to publisher model cards; memory estimates come from AIPCFit's fit rules.
Fit before preference
VRAM, quantization, model size, load memory, and runtime overhead determine whether a local LLM for programming is a practical fit.
Speed is separate
A model can fit in memory without being the right speed or quality match for every codebase, runtime, or coding task.
Coding models
Models are grouped by the catalog size class, not by claimed code intelligence. Use this section to compare local coding models by provider, parameter count, quantization options, and available VRAM fit links.
Smaller coding-tagged models that can be practical on lower-memory local systems.
Small
Small
Small
Small
Small
Small
Small
Small
Coding-tagged models in the catalog's medium size class.
Medium
Medium
Medium
Medium
Medium
Higher-parameter coding-tagged models where fit depends more heavily on VRAM, quantization, and offload behavior.
Larger
Larger
Larger
Larger
Larger
Very Large
Very Large
Very Large
Coding by VRAM
Each tier highlights distinct configurations for memory headroom, weight precision, or model architecture, then checks them with the existing fit rules. These use a synthetic NVIDIA GPU on Windows 11 with at least 32GB system RAM (or twice the VRAM tier, if larger). Planning memory includes generic headroom, not a KV-cache calculation at a guaranteed context length. The order is editorial, not a quality ranking.
Hardware tier
Start with compact weights and short prompts. Compare a small instruction model, an 8B general model, and a reasoning distill while leaving room for the KV cache.
A compact starting point for a constrained GPU. Microsoft's card describes a model intended for limited-memory environments; test it on your own code tasks.
Publisher model cardAn 8B alternative with documented thinking and non-thinking modes. The Q4 configuration leaves more memory than its Q5 variant on this tier.
Publisher model cardA reasoning-distilled alternative for comparing explanations and debugging prompts. Its reasoning output also consumes the available token budget.
Publisher model cardHardware tier
Compare an 8B model at Q5 with a tighter 14B Q4 configuration. Keep a compact alternative when context and other GPU tasks matter more than parameter count.
An 8B starting point at higher weight precision than Q4. It leaves more planning headroom than the 14B option; this is not evidence of better code output.
Publisher model cardA larger same-family comparison with documented thinking modes. Planning headroom is tight at 12GB, so keep context short and verify whether any layers offload.
Publisher model cardHardware tier
Use the extra capacity to compare 14B models or retain more context with an 8B model. A larger parameter count still needs a task-specific quality check.
A higher-precision alternative to the 12GB tier's 14B Q4 candidate, with less headroom than a smaller model.
Publisher model cardA 14B reasoning-distilled candidate for comparing step-by-step debugging with Qwen3 on your own tests.
Publisher model cardKeep the smaller configuration when additional prompt context matters more than loading a 14B model.
Publisher model cardHardware tier
This tier opens larger dense and mixture-of-experts configurations. Active expert counts do not replace total weight storage when estimating memory.
A mixture-of-experts alternative. Its model card distinguishes total and activated parameters; all resident weights still need storage.
Publisher model cardA dense instruction-model alternative to the MoE configuration. Compare code edits with your tests; multimodal support is not evaluated by the memory rules.
Publisher model cardA smaller choice when you want to reserve more of this tier for context or other applications.
Publisher model cardHardware tier
Compare a 32B reasoning distill with larger MoE or dense models. More VRAM can also be spent on context instead of on a larger weight file.
A larger reasoning-distilled configuration that becomes a practical memory candidate here. Publisher evaluations do not measure this quantized setup on your GPU.
Publisher model cardRetain the MoE candidate with more room beyond its weights than at 24GB; verify actual context allocations in your runtime.
Publisher model cardA higher-precision dense alternative. This is a tight planning fit, so compare it with Q4 before committing memory to longer prompts.
Publisher model cardStart by comparing Qwen3 8B Q4_K_M and Q5_K_M on the same coding task. Q4 leaves more room for context; Q5 stores weights at higher precision. Qwen3 14B Q4_K_M offers a larger-model comparison, but exceeds the catalog's recommended headroom on this tier. Phi-4 Mini Q8_0 is a compact alternative. None of these memory comparisons establishes which model writes better code.
| Model / weights | File size | Planning memory | Fit |
|---|---|---|---|
| Qwen3 8BQ4_K_M | 5.0 GBCatalog-recorded | 7 GB | Good fit |
| Qwen3 8BQ5_K_M | 5.8 GBEstimated | 7.8 GB | Good fit |
| Qwen3 14BQ4_K_M | 9 GBCatalog-recorded | 12.2 GB | Fits |
The catalog adds the larger of 2 GB or 35% of the weight size for recommended planning memory. Its minimum-load estimate uses the larger of 1 GB or 15%. These generic estimates do not reserve an exact KV cache. Begin testing at 4,096 context tokens with one request at a time, then increase context while watching actual memory. That setting is a suggested trial, not a measured fit guarantee. Long files, retrieved snippets, conversation history, and generated reasoning all use context.
A tight result may need shorter context or CPU offload. GPU-only operation and CPU/GPU offload can behave very differently. Keep your input, output budget, runtime version, and quantization fixed when comparing models. Try a small bug fix, a test-writing task, and a code explanation from your own repository, then run the generated tests and review the diff. Publisher evaluations of original models do not validate these local GGUF variants.
Selection guide
Start with model size and quantization. Smaller or lower-bit GGUF variants usually have lower memory requirements, while larger models need more VRAM, system RAM, or offloading headroom.
Check context length and runtime behavior for your coding workflow. Long files, multi-file prompts, retrieval workflows, and IDE integrations can change memory use beyond the model file size.
Fit and speed are separate questions. The fit engine can show whether a local llm for programming is practical for a memory tier, but coding quality and latency still vary by task, prompt, runtime, drivers, and software settings.
Setup paths
AIPCFit has GPU-specific runtime pages rather than a single universal runtime score. Start with your GPU, then compare local model fit for Ollama, LM Studio, and compatibility pages.
Methodology
These are hardware-fit recommendations for local coding LLMs. They are not a coding benchmark leaderboard, model-quality ranking, or universal speed ranking.
Coding quality varies by programming language, repository size, prompt, runtime, context settings, and tool integration. A practical memory fit is the starting point, not the final answer for every developer workflow.