Catalog / llama.cpp

0.4.1

todayaddedfixedchangedOriginal notes

Overview

llama.cpp 0.4.1 adds Maple 20B-A1B, Tencent Hy 4, and Spark2.5 support, improves JSON schema handling, chat parsing, logging, and server child-process management, and updates ggml to v0.24.0.

API changes

  • Changed llama_sampler_chain_n() to return int32_t instead of int (#28631).
  • Added server_subproc and waiter for router child-process monitoring in server-common.h (#28555).

New models

  • Added Maple 20B-A1B ternary MoE architecture (CPU) (#27000).
  • Added Tencent Hy 4 (hy_v4) preview architecture support (#28127).
  • Added Spark2.5 support (#27868).

Core changes

  • Added Kimi-K3 recurrent-state rollback support (#28466).
  • Fixed MTP context KV cache allocation for DeepSeek2, GLM-MoE, and related models (#28630).
  • Fixed GDN normalization from max to rsqrt for affected Qwen/Kimi/GLM models (#28068).
  • Fixed Granite parameter counts and Granite3 MoE unknown parameter counts (#28643, #28632).
  • Fixed MIMO2 SWA pattern loading and incorrect get_key_or_arr uses (#28865, #28868).
  • Added --fuse-qkv conversion flag to fuse Q/K/V tensors during HF-to-GGUF conversion (#22780).
  • Wrote explicit recurrent_layers metadata for Qwen3-Next/Qwen3.5 conversion (#28208).
  • Refactored JSON schema handling with a common_schema internal representation (#28736).
  • Split specialized chat parsers into common/parsers (#27764).
  • Added structured JSONL logging via --log-jsonl and LOG_JSON (#28437, #28586).
  • Removed deprecated --mmap/--mlock/--direct-io args in favor of --load-mode (#28334).
  • Made mmproj and draft devices default to the global --device selection (#28390).
  • Disabled lazy tensor loading by default on iGPUs (#28326).
  • Fixed speculative decoding after multimodal input and DFlash mtmd chunk decode (#28715, #28587).
  • Fixed Jinja dot-property integer literals and null in membership (#28817, #28620).
  • Improved grammar insert/move handling and max repetition threshold (#26885, #28469).
  • Improved qwen3-coder complex-type parsing and typed content detection (#28742, #28511).
  • Added PCH and unity build support to improve build times (#28091).
  • Updated cpp-httplib to 0.56.0 (#28787).

Multi-modality changes

  • Propagated video hash to bitmap IDs ({hash}+{frame}) for cache identification (#28601).
  • Fixed DFlash speculative decoding failures on mtmd chunks (#28587).

Server changes

  • Refactored server child-process handling into a single monitor thread (#28555).
  • Fixed LRU hang on multiple requests for the same model (#28539).
  • Allow model downloads when --models-max is already reached (#28530).
  • Fixed context checkpoint eviction on prompts shorter than checkpoint_min_step (#28302).
  • Fixed speculative decoding after image input (#28715).
  • Frame router child state commands as whole lines (#28747).
  • Enabled --reasoning-preserve by default (#28437).

UI changes

  • Improved chat message rendering performance with lazy mounting and compositor-friendly animations (#28460).
  • Added UI asset cache for faster builds (#28802).
  • Embedded UI assets directly with CMake, removing the build-time helper (#28445).
  • Fixed MCP image attachments not displayed in tool blocks (#28089).
  • Export conversations from the database instead of the cached store (#27432).

ggml changes

  • Bumped ggml from v0.23.0 to v0.24.0 (release).

Version 0.24.0 focuses on expanding backend coverage and robustness, with a new precision-control API, major Vulkan/SYCL/Hexagon/OpenCL work, and numerous correctness and performance fixes across CPU, CUDA, Metal, and other backends.

Assets

Nightly build: b10964

More info

Changelog since v0.4.0

b29c606e2 llama.cpp : bump version to 0.4.1 (#28900) d9e03f107 sync : ggml eeea73161 ggml : bump version to 0.24.0 (ggml/1627) bbdd9f246 tests : add fusion baseline README and broaden fusion CI triggers (#28893) 97e4ca735 models : fix incorrect uses of get_key_or_arr (#28868) 1aca1f9fc models : fix mimo2 swa pattern load (#28865) be2c6d7d1 tests(s390x): add non-vxe build to tests (#28776) 3d10bcd19 llama: add Maple 20B-A1B ternary MoE architecture (CPU) (#27000) 21f6b0d22 sycl: rfc: Use radix select for top_k (#28670) 2f539596c ggml-cpu : disable PCH and fix CACHE_LINE_SIZE ambiguity to fix heap corruption (#28882) 89fe24240 ci : trigger self-hosted CI on changes to ci/run.sh (#28859) 15d8f2d59 ci : remove gg_sum summary logic (#28857) 661643e43 sycl : fix oneDNN scratchpad breaking the pool free order (#28704) 093a2f86c common : move llama_n_rs_seq to before llama_decode (#28749) ad6c66839 ggml-cuda: fallback to F32 on device without BF16 hardware acceleration (#28846) 7a16a6ce3 grammar : coalesce find + insert into a single insert and adjust move/copy mechanics (#26885) 5f436dddb tests : exclude HY_V4 from WebGPU test-llama-archs tests (#28855) e49d2c276 models : guard the expert FFN size fallback in nemotron-h against a zero divisor (#28779) 697805298 ggml-cpu(s390x): guard VXE-only repack helpers (#28775) 243a3082d tests : fix typo in test-quant-type-selection for nemotron 3 nano (#28835) b6b003d2c sycl : Fix get mem error (#28227) c95f8e47b ci : run editorconfig and code-style checks on ubuntu-slim (#28854) bc52a12b3 pi : prefer PI_MODEL_NAME env var for model disclosure (#28853) 4a8993735 tests : reduce FA test sizes (#28842) 37b3a9e0c ci : remove leftover command (#28839) 002a12ad2 ci : cap test-backend-ops parallel jobs at 2 and add a 3600s timeout (#28833) f1e44dcc1 vulkan: workaround NV queuesubmit driver bug (#28830) 56b9eb280 opencl: apply the noshuffle row-alignment rule to q4_K, q5_K and q8_0, not just q6_K (#28575) 790cf51aa chat : improve parsing of complex types in qwen3-coder (#28742) 8e330954a common: add LOG_JSON macro to log structured data (#28586) acecd5603 common : implement common_schema internal representation for JSON schemas (#28736) ae9afff8d jinja : support dot property integer literals (#28817) 737e0980f cmake: leave the timestamp out of precompiled headers on clang (#28816) 3057bb66c ui : add cache (#28802) 56381e407 server : allow model downloads at model limit fix issue #26809 (#28530) c8edceb06 ggml-cuda: hip add specific config table for AMD GCN (#27841) e192abb40 server : add missing headers (#28795) 718f7b417 vendor : update cpp-httplib to 0.56.0 (#28787) 2a3005c23 syscl : Handle (fail gracefully) unsupported tq1_0 quants (#28681) f3a33dff2 rpc : fix linking when compiling with BUILD_SHARED_LIBS=OFF (#28492) c069aa7f5 server: frame the router child state command as a whole line (#28747) 8a56aedd6 opencl: fix several bugs where the backend aborts (#27630) 07fc97716 opencl: add bin kernel kernel_gemm_noshuffle_q4_k_f32_32b_trans_ila_a8_bin (#28677) 3f5e94d7c webgpu: align tensor bindings to the type block size (#28382) eafe15a5e hexagon: support for multi-device model split (aka row-split) (#28589) d3146f2b5 ggml-webgpu: Update to a recent version of Dawn (#28683) 82d6bb284 server: refactor subproc handling (#28555) 8ea290247 cmake : skip PCH for llama-server PCH when using MSVC (#28763) b78a39a2f ci : run test-backend-ops as a dedicated ci/run.sh test (#28740) 982937a33 tests: extend test-quantize-fns to test nrc=2 (i8mm) kernels (#16234) 8172e6577 tests: tolerate a shared pool abort in test_completion_unified (#28759) 43f3dda62 ggml: skip 0-sized ids tensor when offloading selected experts (#28739) 5bda51bfb metal : skip the empty half of the mul_mm_id token tile (#28301) 3bcfeb700 cmake : add PCH and unity build to improve build times (#28091) 1dfe94e04 common : fix typo in speculative.cpp comment [no ci] (#28750) a2878d30d metal : single-source fusion table + fusion debug rework (#28164) aac810230 metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) 5cdd3d1da model : fix MTP context kv cache allocation for deepseek2, glm4moe, … (#28630) b0dcb8192 server: fix speculation after an image (#28715) 16378d93f CUDA/HIP: Flash Attention tuning (gfx1201) (#28102) 451b89bae ci : key cache to sanitizer matrix (#28708) 481c65f09 vulkan: fix data race and OOB access in argsort(large) (#28705) df03399b8 opencl: add A8 Q4_0 mm binary kernel support (#28268) 28ff09582 vulkan: use CPU writes in ggml_backend_vk_cpy_tensor_async if the context is idle (#28618) 50182a53f vulkan: use add_alloc_dep to enable topk_moe fusion for prefill (#28422) 6788edb4f vulkan: small M matrix optimizations for qwen (#28457) 52d426865 ci : add self-hosted-gpu-cuda and server-sanitize to hf-jobs (#28693) 18c17b4d6 ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (#28687) fa6769818 spec: fix failed to decode mtmd chunk with DFlash (#28587) 41fc7584f scripts : use sed instead of grep for version parsing [no ci] (#28700) d344123fe models: clean up some dead switch branches in old models (#28669) c32d1dabe tests : increase tolerance for Add fusion tests (#28691) e5a8d439c tests : drop SYCL special-casing in test-backend-ops.cpp (#28688) 3ff67eb43 vulkan: fall back to shared-memory reduction for dmmv on PowerVR (#28341) 8c322d5bc convert : expand Nemotron H conversion fix (#28689) 311d4211b memory : avoid allocating V cache for indexer (it's not used) (#28330) 72797e891 vulkan : add command-buffer debug labels for GPU profilers (#28101) 4ea6d1bb6 ggml-cpu(s390x): add repack support for q4_0 (#28667) f1b6fbf35 ggml-cpu(s390x): add Q1_0 vector intrinsic support (#28606) d7e86430a model: fix all granite family parameter counts (#28643) 434ddbbc0 ci: fix sanitizer tests (#28583) 91f6a6cf3 vulkan: use spec constant for matrix matrix multiplication A-type (#25773) 6d9c82ea2 hexagon: rope updates (#28628) 22397c31a vulkan: Convert FILL to distribute workgroups in 2D to avoid exceeding maxComputeWorkGroupCount (#28592) 4b98ab805 py : lower numpy to 2.2.6 (#28654) 9cf3bf256 py : bump numpy to 2.4.6 (#28649) 4850c7727 llama : use int32_t for llama_sampler_chain_n return type (#28631) d4abd573f CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#28552) 5a4d0feca CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (#28079) 14a9d09f7 args: officially deprecate --mmap|mlock|dio (#28334) e2d2c0d6a model: fix granite3 moe unknown parameter count (#28632) 6de9cdb26 mtmd: propagate video ID to bitmap (#28601) b31b71f3a jinja: treat a null left operand of in as a plain lookup (#28620) df750f76b vulkan: add dedicated iq4_xs mat-vec shader (#28426) 1945e0920 vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 (#27471) 30b6a755e tests : use less threads for data initialization (#28325) 304665fe7 Add IQ type handling for MoE (#28476) f3f1a8f27 llama: disable lazy tensor loading by default on iGPUs (#28326) 9113cc188 ggml : fix msvc+clang ggml_vld1q_u32 (#28284) d4389a4dd Revert "ggml-cuda : restore prop.integrated on HIP builds (#24233)" (#28604) 5d806aa25 server : apply checkpoint min-step eviction only when the checkpoint list is full (#28302) 88ada91c1 metal : fix idle threads in mul_mv_iq3_xxs for ne00 < 1024 (#28086) 415e909d8 spec: single device drafter should create meta backend wrapper (#28390) 03fa73cb2 ci : disable npm gha cache (#28600) 1744c6bde ci : add PYTEST_WORKERS=1 to fix server-self-hosted job (#28603) ca86fb222 llama : add missing headers (#28566) 64e9bceb2 vulkan : fuse UNARY(GELU|SIGMOID|SILU|SOFTPLUS) + MUL (#27220) f014bfef8 Fix Vulkan-Hpp handle usage on 32-bit targets. (#22892) 895c045fd chat : split specialized parsers into common/parsers (#27764) 7d701b592 opencl: properly handle non-contiguous inputs to conv2d (#28503) 5a6caa05f ggml : update ggml_prec specification (#26675) 9dcf84e5a model : support Kimi-K3 recurrent-state rollback (#28466) 050dde50c hexagon: add RELU and LEAKY_RELU ops (#28585) 67672dc5b ci : bump ty to 0.0.78 (#28548) f114f91f9 tests : initialize the L2_NORM batch array (#28553) e71b80510 Revert "CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#24546)" (#28551) ccc3646c6 nix : update deprecated expressions (#28145) c0b1871bc webgpu: format the GET_ROWS case block (#28542) 160bd031b server: fix LRU hang on multiple requests same model (#28539) dbeb37548 sycl: add a batched L2_NORM kernel (#28222) 7a333e724 vulkan: add DeepSeek-V4 hyper-connection fused ops (DSV4_HC_COMB/PRE/POST) (#26578) 0c963452e CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#24546) 473599738 ggml: add gfx90c HIP support (#26454) d23c47f2a convert : refactor Hy4-preview conversion - move HC tensor mapping to the global map (#28451) 73ab7599b CUDA: branchless Q4_K/Q5_K unpack to speed up mmvq, L2 prefetch on DGX Spark (#26705) 0cae43063 vulkan: support type-aligned GET_ROWS (#28253) 1173700b9 examples : print ggml_version and ggml_commit in test-cmake [no ci] (#28538) 5202104b5 caps : recheck typed content if template checks for string (#28511) 9a7570587 convert : write explicit recurrent_layers for Qwen3-Next / Qwen3.5 (#28208) b74f590ea ggml-cuda: fix divergent barrier in f16 flash attention (#27870) 992cb503c ggml: allow backend inputs to not create another split (#28387) 9ac8c408a vulkan: rms_norm fusion opportunities (#28024) 2092353c8 ci : add container image checking and tagging (wip) (#28394) 8fe90e1fb vulkan: add TQ1_0 support (mm, mat-vec, mat-vec-id, dequant, get_rows) (#27765) 465e49b9c convert : add --fuse-qkv flag to fuse Q/K/V into QKV during HF-to-GGUF conversion (#22780) 5fdfa6282 models : fix GDN normalization from max to rsqrt (#28068) 3ad1ba733 [Model] Support for Spark2_5ForCausalLM implementation (#27868) d03efa5d5 opencl: properly choose weights pack for q4_K, q5_K mul_mat (#28402) 73a43d1f6 cuda: fixes races in mmid and mmf (#28475) 9e0e22059 grammar : fix max repetition threshold (#28469) 0afb805b1 ui: Improve Chat Messages rendering performance (#28460) 7620399f5 common: add --log-jsonl (#28437) c457e3bf7 ui : embed assets directly with CMake (#28445) 971595d66 metal : add remaining fa-vec tunings for M2 Max (#28458) 74a7c897f Github: limit blank issues to maintainers (#28435) 6a1a922d2 metal : fix memory leak in early return (#28399) 4d9176092 sycl : fix test-backend-ops CI break && restore Kronecker product FWHT support (#28016) (#28254) cd8cdf397 sycl: attribute device allocations by site (GGML_SYCL_MEMTRACE) (#27631) 427291b5b metal : add remaining fa-vec tunings for M3 (#28396) 85d5703a3 ui : fix MCP image attachments not displayed in tool block (#25789) (#28089) 1548a240e opencl: extend the elementwise and data‐movement op coverage (#27633) 4acf4a4cb opencl: add Adreno xmem SDPA path (#26331) 8b4b3558f ci : move more jobs to ccache-buckets (#28375) 1863ac033 ui: export conversations from database instead of cached store (#27432) 49c0dc82b model : add Tencent Hy 4 (hy_v4) preview architecture support (#28127)