Paths are relative to src/frontends/gguf/. Read only the route the task needs. First identify the
input path: a native .gguf file or a supplied GgufDecoder (llama.cpp cgraph). They share
converters; builders and validation differ. Registration or conversion alone never establishes accuracy.
| Task | Read |
|---|---|
| Is a model supported? | supported_models.md or mmproj.md; quote the checkpoint result and limitations, not just the catalog entry |
| Load a model, make it stateful/GenAI/PA-ready, tokenizer, IR export | runtime.md |
| Add or port an architecture | architectures.md; extensions.md when shipping it as a plugin |
| Add or replace an mmproj branch | extensions.md, then the graph contracts in mmproj.md |
Missing ggml operation or op_case |
how_to_add_op.md |
| Wrong numbers | debugging_accuracy.md |
| GenAI/mmproj token, media or PA mismatch | multimodal integration bisection |
| Unsupported weight type | quantization.md; the fix belongs in src/quant/, not a builder |
| Build, run tests, regenerate fixtures | testing.md |
| Run or resume GGUF validation across OpenVINO, GenAI and llama.cpp | validation.md; use existing GHA workflows and focused local tests, preserving revisions, coverage and failures |
Easy to miss
- Which layer is missing. An unclaimed
general.architectureneeds a catalog row orArchitectureExtension; a new mmproj type needs aProjectorExtension; a structurally different use of an existing op needs anop_case. Only a genuinely new GGML op needs a translator. - Registration timing. Architecture/projector extensions before
load(); converters and passes beforeconvert(), on the same frontend instance.GGUFMakeStatefulandAdaptToGenAIare C++ only. - Test build. New translator or builder sources go into the explicit
FRONTEND_SRCSlist intests/CMakeLists.txt, or the test binary fails to link. Runov_gguf_frontend_testsunfiltered before finishing: the op-coverage gate runs in teardown, so check the exit status.GGUFArchConversionskips without generated headers unlessOV_GGUF_REQUIRE_ARCH_FIXTURES=1requires them; generate them with gen_arch_fixtures.py--fetch. Check actual selected counts and skips; registration, conversion, decoder numerics and GenAI media/API integration establish different levels of support. - References. Layout-sensitive expectations come from real ggml CPU via a committed oracle under
tests/, with several heads/tokens and unequal Q/KV dimensions; NumPy or single-head tests can share the bug. Do not tightenexpect_neartolerances from an x86-only run. - Comparisons. Use full logits or features on identical inputs and token histories, with
INFERENCE_PRECISION_HINT=f32,DYNAMIC_QUANTIZATION_GROUP_SIZE=0and the reference's KV precision. SetOV_GGUF_Q4_K_ZP_F16=1before the process starts. Report quantized-CPU and represented-F32 comparisons separately and keep failed results.GGML_OPENVINO_*variables configure only the llama.cpp backend. Inspect tensor-type inventory rather than checkpoint names. Identical quantized weights do not rule out CPU arithmetic differences; exact expansions isolate them but do not waive the original parity gate. Shared exported-model layout does not imply identical media geometry. - Shared changes. Translator, VIEW/
op_caseor shared block changes rerunGGUFArchitectureAccuracy,GGUFArchConversionand the mmproj suites, not just the target model. - Long runs. Redirect output to a file, keep the launched PID and stop only that process. Large Q2 MoE checkpoints expand at CPU compilation; check memory before starting one.
- Ineffective rebuilds. Verify graph attributes, plugin/executor parameters and the linked,
loaded CPU variants before repeating a sweep. Use build and runtime identity
for overlays or multiple worktrees; version strings and
lddalone do not prove execution identity. - Validation batches. Extend shared-path regression checks to tokenizer, mask and PA changes with representative SDPA/PA/media/chat cases. The validation guide separates numerical parity, model quality and Optimum compatibility; a known arithmetic difference remains a failed parity check unless the user changes the acceptance contract.