Quick answer
What is Ollama context length?
Context length is the token budget a local model can attend to while generating. A bigger window can help with longer documents, chats, and coding tasks, but it can require more memory. Set the window, load a local model, and run ollama ps to inspect its CONTEXT allocation and PROCESSOR placement.
The model's advertised maximum context, its active runtime allocation, and the number of tokens in one actual prompt are not the same number. Defaults also vary by Ollama version and hardware: verify the active setting instead of assuming everyone starts at 4K.
Step 1 · Inspect
Check your active context and GPU offload
Run a local model and send a prompt first. Open a second terminal while it is loaded. ollama ps displays loaded models; in supported versions, its CONTEXT column reports the allocated window. PROCESSOR shows where the model is loaded, not its tokens-per-second speed.
ollama run llama3.2:3b
# Send a short prompt, then in another terminal:
ollama ps100% GPU
The model is loaded on the GPU. This is a placement signal, not a speed benchmark.
100% CPU
The model is loaded in system memory. Check whether that is expected for your device.
CPU/GPU split
The model is partially offloaded. Memory pressure may be involved, but inspect the full setup.
If no model appears, issue another prompt before it unloads. If your Ollama version does not show CONTEXT, check the installed release and official documentation rather than inferring the value from a file name.
Step 2 · Configure
How to change Ollama context length
Ollama app or interactive CLI
In the Ollama app, use its context-length setting if available. In an interactive ollama run session, Ollama documents the parameter command below. Recheck the loaded model afterward.
/set parameter num_ctx 8192Server-wide default
When starting and managing your own Ollama server, set its context budget with an environment variable. Quit an already-running Ollama app or service first; launching a second server on the same port can fail. Service-managed Windows, Linux, and macOS setups need platform-specific configuration.
OLLAMA_CONTEXT_LENGTH=8192 ollama servePer-model Modelfile
A Modelfile lets you create a named configuration from a local base model. Make sure the base model is installed, then save and use a file named Modelfile.
FROM llama3.2:3b
PARAMETER num_ctx 8192ollama create llama3.2-context8k -f Modelfile
ollama run llama3.2-context8kPer-request REST API
For a local API request, use options.num_ctx. This sends a setting for the request, not a permanent change to every model on the machine. The example assumes Ollama is already serving locally and the model is installed.
curl -s http://localhost:11434/api/generate -d '{
"model": "llama3.2:3b",
"prompt": "Summarize the benefits of local AI.",
"stream": false,
"options": { "num_ctx": 8192 }
}'Important: a CLI option, a Modelfile default, a server environment setting, and a request-specific override have different scopes. Reload the model and verify the running configuration before comparing results. See the official FAQ for service-specific environment-variable instructions.
Step 3 · Compare
Test 4K, 8K, and 16K on the same GPU
These are test settings, not measured results. Run one model with a fixed quantization and prompt. Record the actual CONTEXT and PROCESSOR fields plus available GPU memory for each run. AIPCFit does not claim that any particular GPU will cross an offload threshold at one of these settings.
| Requested context | Value | What to check |
|---|---|---|
| 4K | 4096 | Start with a short conversation or prompt; record the baseline allocation. |
| 8K | 8192 | Keep the same model and prompt, then compare the loaded allocation. |
| 16K | 16384 | Check whether extra context pushes the same model toward CPU offload. |
- Write down your OS, GPU, total and free VRAM, RAM, Ollama version, exact model tag/digest, and quantization.
- Keep model, prompt, output limit, cache configuration, and other GPU workloads as similar as possible.
- Start with
num_ctx 4096. After a prompt, recordollama psand, for NVIDIA, GPU memory usage vianvidia-smi. - Unload the model with
ollama stop llama3.2:3bif that is the model you used. Repeat at 8192 and 16384, reloading between runs. - Record the active CONTEXT, PROCESSOR split, baseline GPU memory, and any load error. A loaded model and a peak during generation are separate observations.
ollama --version
ollama ps
# NVIDIA only (Windows/Linux with supported drivers):
nvidia-smi
# Unload between measurements:
ollama stop llama3.2:3bDo not infer benchmark results from this worksheet. The numbers 4096, 8192, and 16384 are configuration values only. The actual active context and memory pressure depend on the hardware, model, backend, and current software version. NVIDIA memory readings are device-level snapshots, not an isolated accounting of Ollama alone.
Memory explanation
Why context length changes the memory budget
The model's weights are only part of the runtime footprint. A longer context can require more KV-cache storage, alongside other runtime buffers and concurrent work. The actual allocation depends on the model architecture and implementation.
Model weights
Same model and quantization across the test.
KV cache
Can increase as the allocated context budget grows.
Runtime buffers
Depend on backend, batch size, and other settings.
Illustrative explanatory diagram only. No numerical memory measurement is shown.
A recorded GGUF file size is not a required VRAM figure. See our Q4_K_M quantization guide for the weight-format distinction. Apple silicon uses unified memory; do not treat the entire unified-memory total as equivalent to discrete NVIDIA VRAM.
Step 4 · Interpret
What if a longer context moves part of the model to CPU?
Keep the model, reduce context
Return to the last comfortable window, reload the model, and compare PROCESSOR again. Use the smallest context that still meets your task.
Keep the context, choose a smaller model
Try a smaller model or supported quantization. Compare the new artifact with the same workload rather than assuming a certain speed or memory saving.
CPU-only even at baseline?
Investigate GPU detection, driver support, and server logs instead of treating this as only a context problem.
Follow the Ollama not using GPU troubleshooting guide when GPU discovery is the problem, or check CPU-only Ollama guidance if running without a dedicated accelerator is intentional.
Next step
Choose a model and context that suit your PC
The checker and calculator give model-fit planning guidance, not a measured result for arbitrary context settings. Use them to narrow the model shortlist, then verify the loaded allocation in Ollama.
Common questions
Does increasing Ollama context length use more VRAM?
It can. A larger context budget can increase KV-cache and runtime memory needs. The amount varies by model architecture, cache configuration, number of parallel requests, backend, and available memory. Check the loaded model rather than assuming a fixed GB-per-token rule.
Is num_ctx the same as a model's maximum supported context?
No. num_ctx sets a requested runtime window. The model's published maximum, runtime allocation, and actual tokens processed are different things. A larger requested value does not make a model reliably understand a longer prompt beyond its supported limit.
Why does ollama ps show CPU/GPU after I raise context?
A larger runtime allocation may exceed available GPU memory, causing partial offload. The exact cause needs checking: compare available VRAM, model size, cache settings, other loaded models, and Ollama logs. CPU/GPU split alone does not prove the cause.
What if ollama ps does not show a CONTEXT column?
Check your installed Ollama version and current documentation. CLI output can change across versions. Restart or reload the model and confirm that the setting took effect before interpreting a comparison.
Official technical sources
- Ollama: context length and verification
- Ollama: local FAQ and num_ctx examples
- Ollama: Modelfile reference
- Ollama: generate API
- Ollama: hardware support
Research and documentation review: November 17, 2026. This guide does not present hands-on benchmark data, manufacturer-guaranteed minimums, or universal VRAM thresholds.
