Google · Gemma 4 · Local inference planning
Gemma 4 12B VRAM requirements and Ollama setup
For Gemma 4 12B Q4_K_M, plan above an 8 GB GPU: the download is about 8 GB, while our estimated load and planning levels are 9.2 GB and 10.8 GB. A 12 GB available VRAM budget meets that planning estimate; Q8_0 reaches it at the 24 GB tier in this guide.
These are estimates from AIPCFit's existing memory formula, not Google minimum requirements, measured VRAM, or certified GPU compatibility. Actual fit depends on free memory, context, and the runtime.
Sources reviewed . No device benchmarks or tokens-per-second results are claimed.
Verified model specifications
- Publisher / checkpoint
- Google / instruction-tuned (it)
- Total parameters
- 11.95B (Google model card)
- Architecture
- Dense, encoder-free Unified decoder
- Model name
- 12B Unified; 12B is the size label
- Checkpoint context limit
- 262,144 tokens
- Checkpoint input / output
- Text, image, audio input / text output; video as frames
- Verified local artifacts
- GGUF Q4_K_M and Q8_0 through Ollama
- Measured hardware minimum
- Unknown; planning estimates below
Sources: Google model card, Google's Hugging Face checkpoint, and checkpoint configuration.
Google's 12B Unified model has 11.95B parameters. Its image and audio inputs use lightweight projections into the decoder, rather than separate encoders. Ollama still packages those projection tensors as a projector file. Encoder-free architecture does not mean the downloadable package contains only the main weights.
Checkpoint capabilities and runtime support are different claims. The exact Ollama pages document text/image input. Audio through these packages remains unverified here, even where projector metadata includes audio tensors. Neither this guide nor the catalog certifies LM Studio or standalone llama.cpp support for these exact packages.
Q4_K_M and Q8_0: exact artifacts and download sizes
| Exact tag / source | Download (rounded) | Estimated load minimum | Estimated planning memory |
|---|---|---|---|
| gemma4:12b-it-q4_K_M | 8 GB | 9.2 GB | 10.8 GB |
| gemma4:12b-it-q8_0 | 13 GB | 15 GB | 17.6 GB |
Ollama package 6114515d63c1: rounded 8.0 GB download includes a 7.4 GB Q4_K_M GGUF model, 175 MB BF16 projector, and 465 MB Q8_0 draft model (423M parameters). Q4_K_M describes the main model; this is not a measured VRAM requirement.
Ollama package 41c402fdddc2: rounded 13 GB download including the Q8_0 GGUF model and BF16 projector; this is not a measured VRAM requirement.
Ollama registry requirements for these reviewed artifacts: Q4_K_M requires Ollama 0.30.9 or later; Q8_0 requires Ollama 0.30.5 or later. These version requirements do not guarantee hardware compatibility.
The Q4_K_M and Q8_0 labels describe the main model's quantization. The projector uses BF16; a draft model's Q8_0 label does not turn the main Q4_K_M model into Q8_0. Component sizes are rounded independently; do not expect their displayed sum to be byte-exact. We have not measured a speed or quality advantage for either package or its draft model.
These explicit tags avoid the generic or latest alias. Tags can still change: the recorded package identifiers and source links describe the reviewed artifacts, not an immutable promise about future downloads.
Download size is not required runtime memory
A download occupies storage. Running inference also requires allocations for the loaded weights, KV cache, work buffers, and any active projector or draft model. Which allocations occupy dedicated VRAM depends on the runtime and placement. An 8 GB package does not establish a full fit in 8 GB VRAM.
For the reviewed download size S, AIPCFit's existing heuristic estimates minimum load as S + max(1 GB, 15% of S), and planning memory as S + max(2 GB, 35% of S), rounded to one decimal. These are project estimates; neither formula measures peak memory or calculates the cache for a particular context.
We use the complete recorded package size as the starting signal, including additional components. Do not add those component download sizes to the table again. Rounded listing GB and a nominal GPU capacity are only coarse planning inputs, not a byte-accurate allocation model. See Ollama's context guidance and its memory and cache FAQ.
Gemma 4 12B on 4, 6, 8, 12, 16, and 24 GB VRAM
Every cell below is an estimate derived from the same planning engine as the VRAM calculator. Compare against available dedicated GPU memory after other GPU use. Installed capacity alone does not establish that budget. The screen assumes a compatible runtime/backend, considers each variant separately, and makes no measured full-GPU-fit claim.
| Available VRAM | Q4_K_M estimate | Q8_0 estimate |
|---|---|---|
| 4 GB | Below package-size signal | Below package-size signal |
| 6 GB | Below package-size signal | Below package-size signal |
| 8 GB | Below estimated load minimum | Below package-size signal |
| 12 GB | Planning estimate met; verify runtime | Below package-size signal |
| 16 GB | Planning estimate met; verify runtime | Load estimate met; limited headroom |
| 24 GB | Planning estimate met; verify runtime | Planning estimate met; verify runtime |
4 and 6 GB are below both package-size signals. At 8 GB, even Q4 is below the estimated load minimum. At 12 GB, Q4 meets the planning estimate but Q8 is below its package-size signal. At 16 GB, Q8 meets its load estimate but not its planning level. Both meet the planning estimates at 24 GB.
Meeting a planning estimate means capacity is a candidate for testing. It does not guarantee maximum context, concurrent sessions, multimodal workloads, or a particular GPU model. Check Ollama's GPU and driver support separately, and compare recorded capacities in the GPU catalog.
Gemma 4 12B on Apple Silicon: shared RAM, not dedicated VRAM
Apple describes unified memory as a pool shared by CPU and GPU. Ollama supports Apple GPU acceleration through Metal. Leave room for macOS, apps, and inference allocations rather than treating total Mac memory as an exclusive model budget.
The following figures use AIPCFit's existing reserve policy. They are estimates of a usable AI budget, not Apple or Google requirements, guaranteed Metal allocations, or tested Mac configurations.
| Total unified memory | Estimated AI budget | Q4_K_M estimate | Q8_0 estimate |
|---|---|---|---|
| 8 GB | 5 GB | Below package-size signal | Below package-size signal |
| 16 GB | 11 GB | Planning estimate met; verify runtime | Below package-size signal |
| 24 GB | 18 GB | Planning estimate met; verify runtime | Planning estimate met; verify runtime |
| 32 GB | 24 GB | Planning estimate met; verify runtime | Planning estimate met; verify runtime |
A 16 GB Mac has an estimated 11 GB budget under this policy. That clears Q4's planning estimate for Gemma 4 12B, but not Q8's. Monitor memory pressure with your workload; swap use is not evidence of a comfortable fit. These Apple Silicon estimates do not apply unchanged to Intel Macs.
Context window limits and KV-cache memory
The checkpoint configuration specifies 262,144 positions. The Google architecture description combines sliding-window and global attention; it is not a license to assume a conventional all-global-attention cache size.
The KV cache retains attention state for tokens. Increasing context can increase memory even when the weight download stays unchanged. Its footprint depends on attention layout, cache precision, and runtime behavior. Ollama documents cache precision and parallel requests; weight quantization such as Q4_K_M is separate from KV-cache precision.
These tables do not model a context-specific cache or multimodal token budget. They cannot certify full-context inference. Set an explicit 4096-token context as a testing starting point, not a measured fit condition; inspect memory and increase gradually. Verify the allocated context because runtime defaults can vary. See Ollama context settings.
Ollama setup: exact Gemma 4 12B run commands
Install or update Ollama for your operating system, ensure its service is running, and choose a release that can load these artifacts. A minimum compatible version has not been verified here. The commands below match the linked artifact pages.
Q4_K_M
ollama pull gemma4:12b-it-q4_K_M
ollama run gemma4:12b-it-q4_K_MQ8_0
ollama pull gemma4:12b-it-q8_0
ollama run gemma4:12b-it-q8_0Inside the interactive session, set context before sending your test prompt:
/set parameter num_ctx 4096In another terminal, inspect the loaded model:
ollama psThe Ollama FAQ explains the CPU/GPU placement column. Check it alongside allocated context and actual memory use, then test your intended input. An empty listing means there is no currently loaded model to inspect. Consult the CLI reference if commands differ in your installed release.
CPU and system-memory offload: what it does and does not prove
The Ollama placement FAQ documents both CPU execution and mixed CPU/GPU placement. If VRAM is below the estimates, sufficient system RAM may allow offloading in a compatible runtime. The current fit engine reports partial offload separately; it does not count it as a full GPU fit or add RAM to VRAM as one interchangeable pool.
We have not measured this model's offload split, peak system RAM, or response speed. CPU computation and transfers can change latency; offloading may be unsuitable for your workload even if it loads. Check actual placement and responsiveness before relying on it. A below-signal VRAM result alone does not prove CPU execution impossible.
Gemma 4 12B hardware and setup FAQs
Can Gemma 4 12B run on an 8 GB GPU?
The rounded 8 GB Q4_K_M download is not an 8 GB VRAM requirement. Our estimated load threshold is 9.2 GB and planning level is 10.8 GB. An 8 GB GPU falls below both. CPU offload may be possible with enough system RAM; it is not a full GPU fit.
Should I choose Gemma 4 12B Q4_K_M or Q8_0?
The recorded downloads are 8 GB for Q4_K_M and 13 GB for Q8_0. Our planning estimates are 10.8 GB and 17.6 GB. Q4 is the smaller storage choice. We have no measured quality or speed comparison for these packages.
Why does the Gemma 4 12B Q4 download include a draft model?
The verified Q4 package lists a 423M-parameter Q8_0 draft model, a BF16 projector, and the main Q4_K_M model. Its draft_num_predict setting is 3. The rounded 8.0 GB package size includes these components; it is not just the main weights. We claim no measured speed benefit.
Can Gemma 4 12B use its full context within these memory estimates?
That is unverified. The checkpoint limit is 262,144 tokens. The estimates do not calculate a context-specific KV cache. Start with an explicit 4096-token context, then measure memory before increasing it; 4096 is a test starting point, not a calibrated guarantee.
Can I run Gemma 4 12B on a 16 GB Apple Silicon Mac?
AIPCFit's reserve policy leaves an estimated 11 GB AI budget from 16 GB total unified memory. That meets the 10.8 GB Q4 planning estimate but not the 17.6 GB Q8 planning estimate. This is a capacity screen, not a tested Mac requirement; macOS, other apps, and inference share memory.
Can Gemma 4 12B run without enough GPU VRAM?
Ollama can place inference in system memory or split it between CPU and GPU. AIPCFit does not measure this model's offload allocation or throughput. Enough system RAM and runtime support are still needed. Do not add RAM and VRAM together as if they were one dedicated GPU pool.
Does Gemma 4 12B support audio in Ollama?
Google documents audio input for the checkpoint, alongside text and image input with text output. The verified Ollama artifact pages describe text/image input. Audio support through these exact packages and an end-user application remains unverified here; encoder metadata alone does not establish working audio support.
Check your hardware and compare Gemma 4 models
- VRAM calculator — compare model and quantization planning estimates.
- GPU catalog — inspect recorded dedicated memory.
- Model catalog — browse the available local models.
- Gemma 4 E4B VRAM requirements and Ollama setup — review the other model's artifacts and limits.
Official sources and verification limits
Reviewed 2026-11-04. Package sizes and specifications are sourced below. Numerical memory thresholds and capacity screens are AIPCFit estimates. Actual GPU/Mac fit, peak runtime memory, speed, and Ollama audio support remain unverified.
- Google Gemma 4 model card
- Google checkpoint on Hugging Face
- Google checkpoint configuration
- Ollama gemma4:12b-it-q4_K_M
- Ollama gemma4:12b-it-q8_0
- Q4_K_M main model metadata
- Q4 package projector metadata
- Q4 package draft model
- Ollama GPU and Metal support
- Ollama context length and memory
- Ollama context, placement, and cache FAQ
- Ollama CLI reference
- Apple: unified memory on Apple silicon
