Catalog / llama.cpp

0.8.0

todayaddedfixedchangedOriginal notes

Overview

This release adds the LiquidAI LFM3 family (text, vision, audio and omni) together with a new experimental streaming-audio API in mtmd, and wires realtime audio in and out of the server through the new /v1/realtime and /tts endpoints. A batch of new architectures landed: DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, plus decision/embedding models from LiquidAI, Baidu and NVIDIA. The core got sparse-attention (QSA/DSA) reworked onto a fused masking op, a 50/50 SLRU cache for host-resident MoE experts, and cleaner no_alloc memory handling; the server got router memory-management and tool-calling improvements; the UI can now browse and download models straight from HF Hub, search conversations and compact them. Finally, ggml was bumped to v0.28.0, bringing GGML_OP_TOP_K_MASK, -1 expert-index semantics and large backend performance work (CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, Hexagon HTP rework, KleidiAI SVE kernels).

Highlights

  • LFM3 family: LFM3-1.7B (text) and LFM3-VL-2B (vision), verified against the reference HF implementation llama/31252
  • LFM3-Audio-2B and LFM3-Omni-2B: ASR, TTS and voice chat from a single model, with streaming audio input and output llama/31257

API changes

  • DRY sampler: new LLAMA_DRY_MAX_SEQ_BREAKERS (64) caps distinct sequence breakers, each costing O(n_vocab) at init llama/31035
  • mtmd: new experimental mtmd_audio_stream API turning PCM fed as it arrives into decoder-ready embedding rows llama/30629
  • mtmd: audio generation is now stateful - mtmd_gen_state replaces the raw state_data buffers of the audio gen API llama/30902
  • mtmd: mtmd_caps gained gen_audio; new TTS types SOPRANO and LFM3A; repetition penalty for sampled codes llama/28607 llama/31257
  • mtmd_helper_get_capabilities() to query the capabilities of an mmproj file from helper code llama/31183
  • Server: new /v1/realtime endpoint implementing the OpenAI realtime protocol for conversation and transcription llama/31269 llama/30636
  • Server: new POST /tts endpoint for text-to-speech models llama/26603
  • Server: reasoning effort levels of the chat template are exposed and selectable via the API llama/31271
  • Chat API: support for deferred tools and tool_reference content parts llama/31256
  • CLI: --model or -hf is now required llama/31242

New models

Core changes

Multi-modality changes

  • Streaming audio input via mtmd_audio_stream, used to support voxtral-realtime llama/30629
  • Soprano TTS support and MiniCPM-OCR 1.0 support llama/28607 llama/30979
  • Image preprocessing aligned with transformers/torchvision: LFM2 resize, tiles and layout, Qwen-VL resize llama/30943 llama/30261 llama/30736
  • LFM2 tiling parameters are read from the GGUF metadata llama/30840
  • lightonocr defaults to native resolution instead of forcing a 1540 longest edge llama/30600
  • build_inp_temporal_merge moved to the mtmd base class; fix Gemma-4 FFN activation llama/31065 llama/30882

Server changes

  • Realtime conversation and transcription on /v1/realtime, with response.done sent after the committed audio drains llama/31269 llama/30636 llama/30994
  • POST /tts endpoint for TTS models llama/26603
  • Expose the reasoning effort levels of the chat template llama/31271
  • Router: configurable device memory margin for dynamic unloading of models llama/21231
  • Router: propagate client disconnects to the child task, stop re-adding the mmproj margin on sleep/resume llama/29707 llama/28233
  • Rerank endpoint accepts typed content input and substitutes the placeholder from the original template llama/29625 llama/30864
  • Anthropic /v1/messages: forward llama.cpp sampling params and pass id_slot through the conversion llama/30856 llama/28554
  • Support logprobs together with speculative decoding llama/27196
  • Fix usage.completion_tokens undercounting with n > 1 and missing output_index in Responses API SSE events llama/29486 llama/30863
  • Fix SWA checkpoint invalidation logic llama/30666
  • New --connect flag to allow p2p access to the server outside the LAN llama/28277

UI changes

  • Discover and download models directly from HF Hub, with org-less models listed first llama/29584 llama/31215
  • New search dialog for conversations and messages llama/31173
  • Conversation compaction to keep long chats within the context llama/27401
  • Models manager moved to its own route, model details open in a drawer without leaving the page llama/30988 llama/31176
  • Reasoning menu built from the chat template's effort levels, with a clearer highlighted toggle llama/31273 llama/31212
  • Support OGG audio attachments by converting them to WAV llama/31237
  • New "Show tool call in progress" setting llama/30681
  • Save and reset settings buttons shown only when there are unsaved changes llama/31248
  • Sidebar resizing and general layout/UX improvements llama/31119 llama/31030
  • Fixes: draft lost after "Add System Message", system row stretching in WebKit llama/31214 llama/31246
  • Fixes: clipped attachment text preview, markdown tables and lists cut off on phones llama/30806 llama/30687

ggml changes

  • Bumped from v0.27.0 to ggml v0.28.0
  • New GGML_OP_TOP_K_MASK fused sparse-attention mask, plus accessors for the precision configured on a compute node
  • -1 is now a valid expert index in MUL_MAT_ID/ADD_ID: the slot is skipped and the output row zeroed
  • SCALE accepts non-contiguous row views; SUM accepts non-contiguous inputs on CPU, CUDA, Vulkan and WebGPU
  • Backends: fused MoE top-k and gate/up+GLU with chunked graph launches on CUDA, graph record/replay on SYCL
  • Backends: Q2_K/Q3_K weights and a lightning indexer on OpenCL, subgroup reductions and src1 precision on Vulkan
  • More: restructured Hexagon HTP backend, flash attention on ZDNN, Q2_0 on WebGPU, KleidiAI SVE GEMV/GEMM kernels

Assets

Nightly build: b11960

More info

Changelog since v0.7.0

8642541b2 llama.cpp : bump version to 0.8.0 (#31309) d4f1b18b3 mtmd: enable video input support for LFM3 model family (#31283) d1eb0deba server : support tool calls in /v1/realtime (#31287) 5d7793d6f sync : ggml ec941ee85 ggml : bump version to v.28.0 (ggml/1676) e486a4b55 ui: build the reasoning menu from the template effort levels (#31273) f1a7dea99 tests : adjust constants to reduce irrelevant warnings (#31254) a7af149a2 cuda : add top-k MoE fusion for 384 experts (#31288) 2864937a7 vulkan: clamp A prefetch to end_k in coopmat1 int8 matmul (#30283) a71c00f0b chat : stop qwen3-coder reasoning and content before template separators (#30557) f4d8af608 ui : show save and reset settings buttons only when relevant (#31248) 02d502f31 ui: Support OGG audio attachments by converting them to WAV (#31237) b1756b23f ui: Fix draft for new message disappearing after using "Add System Message" action (#31214) 6886628cb ui : Search dialog (#31173) 8ca4060b7 server : add realtime conversation and transcription on /v1/realtime (#31269) d05706b4f model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275) 1b4554eff server: expose the reasoning effort levels of the chat template (#31271) a91917994 tests: fix CI riscv timeout in test-llama-context (#31265) 338fce651 HIP/CUDA: remove pragma unroll from loops in conv2d where itteration count cant be infered at compile time (#31255) 19ff817fb model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257) eb7818c0b common : support deferred tools and tool_reference content parts (#31256) 6c4459104 snapdragon: switch QDC artifact upload and log download to v2 API (#31240) 19cce6919 model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252) e86f5fd1e cmake: WASI doesn't support spawning (#30751) 27b7a8a00 llama : build the k-pool sparse selection independently of the kq_mask shape (#31243) 7ecb57f7a rpc : catch std::length_error in recv_msg (#31204) 8542e9435 tests : speed up tensor data generation and model saving (#31145) 922f53ed3 vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MUL_MAT (#31239) 0c8c59ef3 mtmd-cli: Reduced scope of batch pointer (#31155) (#31165) 3ab37aedf ui: fix system message row stretching to full prompt height in WebKit (#31246) 85cc432e2 server: add router device memory margin parameter for dynamic unloading (#21231) 74a1fd3d0 common : --model or -hf is required (#31242) 7eed6b768 model: Add MiniCPM-OCR-Jev (#30980) 276378515 metal : add GGML_OP_TOP_K_MASK (#31230) d78f40d9d glm5-next : QSA without kq_mask (+ DSA) (#31009) fae9ac561 ci : use ccache depend mode for ROCm release builds (#31231) a8276b70c ci : force python 3.12 on ubuntu 26.04 (#31222) de5122d87 ggml : relax scale to non-contiguous views (#31146) 6666a7330 tests : fix multi-seq split replay and partial-state cases for trained models (#31216) 7a2e5aba9 llama : remove obsolete n_pad from memory instances (#31200) 0d5ca6009 ui: list org-less models first in the model selector (#31215) d930fe3c2 opencl: refine mul mat id -1 index (#31170) a2b3e121c opencl: initial support for q3_k (#31172) 823df7d0f ui: Highlight reasoning toggle on hover and align it with model selector (#31212) 1a97e387d ui: add conversation compaction (#27401) bc262f33c ui: Open model details in a drawer without leaving the page (#31176) f5892ca5a ui: use shared confirmation dialog for conversation deletion (#31187) 7aca0d2ea hexagon: add GGML_HEXAGON_VTCM_SIZE (#31221) 961c376a2 llama : enable moe amax rescaling for laguna (#31147) 4244ad7ac ci : add windows behavior check (#31191) 63b4b303b opencl: write KV-cache views in set_tensor so a state restore is not dropped (#30471) caa8191df hexagon: MUL_MAT_ID row skipping optimizations and improvements (#31169) a4699400e llama : move no_alloc out of llama_hparams (#31141) 4b851fbee models : replace ggml_tensor** out-params with output structs (#31142) 4d71b42fd ggml-cpu : rewrite concat as one row kernel for all types (#31179) 70c5f2632 vulkan: rms norm optimization using subgroup reductions (#29882) 5cd62ffee sycl: template the plain MMVQ launch triples (#31013) 38645c2ad sycl: template the per-quant concat impls (#31008) 02269b4c0 sycl: add set_tensor_2d for faster tensor-split loading (#30358) 5320c87ff sycl : bound reorder temp memory to 32 MiB (#31062) 74b104c9b common: pin model resolution on real HF repo layouts (#30548) c6fb3b5be mtmd: add mtmd_helper_get_capabilities (#31183) 8b28c2394 ggml-cpu : fix unused variables in arm arch (#31180) fd8f8451a tests : refactor test-llama-archs to not link internal libllama symbols (#30880) 9125a2ebe ci : fix race in webgpu memset (#31162) 9b8446b91 metal : stop overflow from happening in large matrix output offsets (#30072) dc9e3bed4 ui: UX improvements (#31030) 69b3cc406 ui: Sidebar resizing improvements (#31119) 224d94c02 ci : bump server timeout during startup (#31174) 48c36110a ci : drop the macos-latest-x64 job (#31175) a07421535 deepseekv4.1: follow-up review fixes (#31137) d854d98ba cuda : use std::string instead of VLA for archName (#30463) 48c9f08b7 common, grammar: add recursive depth limit, make it a requirement (#31153) 6050b467c tests : adjust default threads for test-llama-context (#31157) 2280f0fd1 qwen: merge tensors on convert (#31028) 1a0b56cec ci : disable test-llama-context with fa off in the webgpu (#30977) 63412d873 llama: fix exhausting CPU and MEM caused by dry sequence breaker (#31035) 5c965fd8c llama-quant : fix 4D tensor quantization using ggml_nrows() and name-based expert detection (#30820) f1168ac41 grammar: limit recursion depth to prevent stack overflow (#29495) (#29755) fa3645feb llama : respect no_alloc in all memory caches (#31087) 7790a9231 tools : reject truncated GGUF tensor data in merge (#31044) c49edf096 CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows (#29948) a55c16e4e ggml-cuda: fuse qwen MRoPE + set-rows (#31025) e321bdad7 vulkan: Add a larger tile size for conv3d (coopmat2 only) (#29920) 459a1ba54 vulkan : one warp tile per subgroup in the small matmul tiles (#30106) f0fb5984b hexagon: generic dims collapse/reshape for unary, binary and matmul ops (#31071) 9a4580b61 opencl: add topk and lightning indexer (#31026) b083c384f opencl: add initial q2_k support (#30999) 344be0a60 ci : update CPU job assignments (#31136) b3fa69955 chat : reject JSON nested more than 128 levels before walking it (#30242) 74bca3950 model: DSV4.1 flash (#31086) d327bf1f0 jinja : fix select and selectattr argument handling (#31059) 78a8ae14b hexagon: KV-head team split for multicore HMX flash attention (#31055) f3269ef96 server: refactor server entrypoint (#31113) 3f03b1772 metal : remove mul_mm_id amax rescaling (#30924) 9a0fde996 rpc : add bounds check in rdma_recv (#30399) e168e041e ui: Models Manager as Route + UI improvements (#30988) ea1d7d66f fix(ui): align hover card and context popup styling with dropdown menu (#30978) 71255fefc ci : pass per-runner results dir to self-hosted jobs on T4 runners (#31100) 73adf89e3 minimax-m3 : forward quant scales to the graph (fixes #31063) (#31097) df9beac71 tests : fix test-llama-context for large n_batch (#31083) 4d32b29aa ci : exclude test-speculative-rejection-sampling from macos-latest-arm64 (#31088) 303220975 llama : deny -sm tensor for arches with moe amax rescaling (#31081) 701fd25b3 ggml : allow MUL_MAT_ID -1 index to skip the computation (#26631) 3642238cc tests : add speculative rejection draft sampling test (#30213) 6b9ba4f2e ci : fix data race between runners sharing the model cache (#31046) 8e5a58df8 vulkan: ggml src1 precision support (#30656) 1ddffe9f7 HIP: add byte permute fallback when __builtin_amdgcn_perm is unavailable (#30915) 7894e8b92 mtmd: move build_inp_temporal_merge to base class (#31065) edc353842 mtmd : lfm2 resize matches torchvision (#30943) 9ae73e68f mtmd : add Soprano TTS support (#28607) 0ae0e1976 hexagon: add SGN op (#30346) 21ec30714 common : allow concurrent downloads across processes (#29312) 9145ddb95 hip: fix bf16 WMMA operand vector type for gfx11/gfx12 (#30885) 1a24df163 ggm-cuda: Enforce DP4A MMQ tile height multiple of warp size (#30837) 9ff0d7e90 CUDA: fix build after chunked CUDA graphs (#30804) (#31053) 06230cb87 allows the usage of 4-10 gpus for stepfun and laguna in tensor parallel (#24554) 5c3114268 qwen4exp : build the QSA selection without the kq_mask (#30965) da2ee1e7a vendor : update LibreSSL to 4.4.0 [no ci] (#31049) 280141e6d CUDA: chunk graphs to amortize graph launch (#30804) 36e36e63a hexagon: add CEIL op (#31006) e1e3b2c40 ui : bump packages to clear npm audit (#31038) 4231b23dc ggml : fix time overflow in ggml_time_us (#30969) f00c9083b ci : add behavior check workflow (#31036) f5027fb21 CANN: missing const qualifiers added for consistency (#30933) f64a49819 sycl : pass device into mul_mat_uses_onednn (#31014) 1ab09d632 Extend probabilistic sampling for DFlash2 drafts (#30663) f9599d0cc vocab : fix Nemotron-3.5-Embed tokenization (#30974) 643b0cdac hexagon: Op Fusion refactoring, optimizations and new fusions like BINARY_NX (back-to-back ADDs, etc) (#30843) 0a9ef0067 sycl: Replace host waits in fattn-mkl with explicit dependencies (#30684) 7377bada8 sycl: Add graph record and replay (#28725) 184f65360 server: Substitute rerank placeholder from the original template only. (#30864) d814cbb88 server: realtime API: send complete after committed audio drains (#30994) 970b51ce7 opencl: add depthwise conv2d (#30900) a08ab75ae mtmd : add MiniCPM-OCR 1.0 support (#30979) 19ab5a5e6 clef : include for std::find and std::copy (#30981) 36cbf415f Fix !30793 5c19a5a75 glm5 : optimize inputs (#30932) d9ee007a0 model : fix qwen4exp layer inputs for DFlash drafters (#30927) d81d0d8fb Multi-column on Intel XMX matrix engines refactored (#29864) d77256999 vulkan: replace std::aligned_alloc with posix_memalign for termux compatibility (#30828) 0693dbfb9 nemotron : support non-standard GQA, BPE tokenizer and domyn-small pre-type (#23343) a5d22276b server : support typed content input for RERANK endpoint (#29625) 5f5acfe25 ci : disable slow WebGPU tests (#30963) 1f4c2eeaa ci : be able to reconfigure HTTP port for server pytest workers (#30972) 7170cad4d sycl: fix CONV_2D wrong results for non-contiguous input (#30363) d88040c70 vocab : add clean_spaces GGUF key (#30940) e197cef45 vocab : use Tekken pre-tokenizer for pixtral rather than the Llama 3 one (#30960) bf039c849 models : Record layer inputs for all granite models (#30957) dfeea5200 kleidiai : add Q8_0 SVE dotprod GEMV and SVE i8mm GEMM kernels (#30808) e3797b184 sycl: merge the fp16 and fp32 dequant switches (#30740) 0b1281aee webgpu : add Q2_0 support (#30914) 4912ac24f dflash : load optional FC input scale (#30945) 84746c5c9 ui: Add "Show tool call in progress" setting (#30681) 0682a576a lora : support prism.hadamard folded weights (#30910) a6b83d426 ui : key draft sidecar badges by model instead of repo (#30919) c7fc50941 opencl: add F32-to-I32 ARGMAX (#30897) a37cc4610 models : add Flint model (FlintForCausalLM) (#30928) 99fb99718 jinja : fix namespace attributes shadowed by dict methods (#30941) 7775ffb7e model: Add support for SenseNova U1 model (#28919) d454a204e llama: create new fused op for QSA mask (#30640) 0756e83fc sycl: q8 fast dequant (#30775) 9febe63d5 jinja: support hexadecimal string escapes (#30929) 9b56ccf00 ggml-opencl : add IM2COL_3D support (#30895) 0a1ef309c mtmd: add state object for mtmd_gen (#30902) 261700b89 server : forward llama.cpp sampling params on /v1/messages (#30856) f84909536 vendor: update BoringSSL to 0.20261020.0 (#30889) 824eaecf4 llama : optimize vocabulary loading (#30679) eb182d85c llama : place d2t by its set_rows use so it can stay on the GPU (#30912) ea9e3446b metal : fix memory leaks and cleanup logic in GPU capture path (#30890) 0efe8e4c6 hexagon: neon-vectorized and threaded repack (tiling) (#30901) b531e4c74 opencl: add dp4a GEMM variants to vector load activation (#30753) 74b115ca6 sycl : fix fa crash for Query head dimension mismatch (neq0 != HSK / D=2048/4096) (#30286) 96caa8005 sycl: collapse duplicate need_check launch bodies in mmq (#30739) 479a52779 ggml-cpu : compute f32 gelu_quick and geglu_quick without the fp16 table (#30737) 8baabd53b llama : scale moe down projection per expert token (#30861) 493253fe5 Sycl iq3 crash fix (#30604) 339f3242c opencl: fix rope program build error with newer Clang (#30893) 9a3b91c68 server : fix usage.completion_tokens undercount with n > 1 (#29486) 5c0b49a23 server : add POST /tts endpoint for TTS models (#26603) 1242be66f server: add realtime transcription API (#30636) 84e66d032 refactor : move LFM2 shortconv and FFN blocks into a shared LFM base (#30874) 52ffa0274 model : support Nemotron-3(.5)-Embed (#30877) 839a572a4 qwen: reshape output to 2d instead of 3d (#30823) ead9f3205 server: Support logprobs with speculative decoding (#27196) b684b498b hexagon: fix VTCM size query on images that need layout out-params (#30655) 4a45de4f2 mtmd : lfm2 tiles and layout match transformers (#30261) 7e3eeee04 model : add H2O-Lightning-4B decision model (#30361) 5ba3b9fd3 mtmd: fix gemma 4 ffn act (#30882) 5a9a34d88 server : avoid repeated mmproj memory margin addition on sleep/resume (#28233) 39da130fc ggml-cuda : support ssm_conv kernel size 7 (#30873) 4e201388c server: Router wasn't propagating the client disconnects fully to the child, the response wait could cause a delayed cancellation (#29707) 1ba568c5a server: fix missing output_index in Responses API SSE events for compatibility (#30863) 36140dbee mtmd : read LFM2 tiling params from GGUF metadata (#30840) df1170213 metal : optimize argmax and pool_1d (#30862) 9df0b73c7 model : drop the redundant precision overrides in the Kolibri-1 router (#30865) b38960f52 ci : remove superfluous webgpu jobs (#30868) a01ce9ffc opencl: add several gemm variants for small batches and some fixes (#30850) 8115b4c31 sycl : replace memset due to not always ordered (#30812) 119f68d7d ggml-cpu : select the NEON fp32->fp16 conversion (#30505) b68449857 HIP/CUDA: Fix reduction in mmf getting tile widths not devisiable by 64 (#30836) 522f2b6ee ggml-zdnn: introduce flash attention op (#30512) 83c2d012a spec : save the dynamic ngram cache file (#22055) 13786fbb3 CUDA: Clear CUDA graphs after legacy memory pool flush (#26771) d80619397 server : pass id_slot through the Anthropic /v1/messages conversion (#28554) 704366d76 metal : fix missing MoE kernel variant for 9 experts per token (#30610) (#30699) 707616feb ci: Fix --strict mode for API check (#30623) d8d92973c vocab : add \p{M} to lookaheads in Tekken and GPT-4o pre-tokenizer regexes (#30845) 7879491cf tests : make MoE router selection independent of the backend noise (#30818) 846639535 hexagon: update to snapdragon-toolchain v0.8 and enable hexagon v85 (#30854) dd002b91f sycl: fix compiler warnings - missing prototypes, dead code, extra semicolons (#30271) 1d88cb05f ui: hide the native clear button in SearchInput (#30719) 3bb63cbcb sycl : fix get_rows on reordered quantized weights (#30430) f711c8f30 mtmd : match the transformers Qwen-VL resize (#30736) 921257f50 ci, docs: add WASI CI and docs (#30754) 3ffb62935 ci : re-enable type/lint-check for skills (#30713) 7adf160c4 server: add --connect, allow p2p access to server outside LAN (#28277) ab7ed0d4b ui : fix attachment text preview clipped when a line is longer than the container (#30806) 4c3c5810f hybrid dna tokenizer fixes (#30825) 1b29621ba opencl: add cooperative-k dp4a gemm for q4_0, q4_k and q6_k (#30741) 6fa4f6cd3 opencl: add get_rows for i32, cpy for i32 and f32 (#30756) 0ce8cd114 jinja : implement unique filter (#30819) f174553e1 server : fix swa checkpoint invalidation logic (#30666) 7951f601e vendor : update cpp-httplib to 0.62.0 (#30785) 33d6135cb moe-cache : replace MoE LRU cache with a 50/50 SLRU cache (#30637) 098ce526b metal : fix moe_reduce fusion race when dst aliases its inputs (#30795) f3fb2fcdb cann : refactor #ifdef blocks to avoid unreachable code after return (#25273) fda00f18c tests: add big-endian conversion tests (#30705) 4866db350 CUDA: fix warning for nonexistent extra semicolon (#30774) 42b70032b ggml-cpu: vectorize ggml_v_silu and ggml_v_expf (#30791) 4c1004c49 cuda: cap MMQ Stream-K blocks at available work (#30553) 412a363c6 qwen3 : bidirectional attention and dense_2 output projection (#30607) 2b57148b0 ui: Fix markdown tables and lists being cut off on phones (#30687) 8f97d82da tools : reject truncated GGUF tensor data in split and export tools (#30497) 27af58168 tests : update Metal fusion baseline for beam (#30764) 7db0d3d78 pi : turn .pi/gg into a system prompt addendum (#30762) 99db719bd CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec (#30573) 31288c9c6 hexagon: FLASH_ATTN_EXT improvements for longer context and SWA models (#30731) fe8841a78 hexagon: allow multistream FA (#30218) ff5a23376 cuda : fuse qwen4exp's SCALE -> SIGMOID -> SCALE -> hc_post chain (#30602) 0878f7c30 ggml: support non-contiguous SUM (CPU, CUDA, Vulkan, WebGPU) (#28851) 5aeac37a3 ggml-cuda: fattn do not check oob if k_VKQ_max = (k_VKQ_max - nbatch_fa) (#30572) 547208dca Restore ESIMD kernels on FFN path (#30239) 848bbda90 hexagon: add support for hadamard MUL_MAT and GET_ROW for flat and tiled Q1_0 and Q2_0 (#30627) 797de63b2 hexagon: add support for Q1_0 and Q2_0 (#30122) 9d1878a43 ggml-cuda: HIP: use __builtin_amdgcn_perm for q1_0 unpacking (#29927) c0fd78626 ui : Discover & Download Models from HF Hub (#29584) 2744e48d1 CUDA: faster MMVQ for low-bit types (#30517) 10b8b7553 tests: give the random model generator a per-tensor filler table (#30674) 9d1e03616 cuda : fix -Wtautological-compare in fattn vec dispatch (#30519) f2f667b02 cuda : add grouped experts top-k fusion (#29181) 62318cd57 model : add Kolibri-1 (Aleph-Alpha) support (#30673) 83f938941 ci : added ABI reports as github artifact (#30619) 1268edac5 model : Granite switch decision 256 options (#30693) f74e2fa6a sycl: fix prompt-processing regression after Q4_0/Q8_0 weight reorder (#30374) ea0c4c2c8 skills: add slice-model (#30677) 6420cf5dd ci: update device matrix for Snapdragon jobs by removing SM8750 (#30620) 27c7a769d hexagon: vectorize softplus with HVX (#30435) 2f2763db6 opencl: Add MxFP4 MoE bin kernel for Adreno X1 devices (#30691) 01fc3aae8 model: add Beam (Reflection) (#30670) 23f7aabc1 model : Granite Switch Decision (#30676) 7182ef71e common: honor HTTP_PROXY/HTTPS_PROXY env vars in http client (#21752) 2a667032d convert : respect Mistral format for DeepseekV2Model (#30669) f3363593f Add more benchmarks to llama-eval (#26174) ceefd16da hexagon: correct HMX FA sinks (#30581) 9a16f1dbe model : MTP support for Mellum2.1 (#30612) b84ea5cf5 convert : speed up model endian conversion and add more datatype support (#30647) a30fb4924 [SYCL]: fix memory errors in mul_mat, split buffer, host pool (#29889) 495c5805c gguf-py : add numpy Q1_0 and Q2_0 de/quantization support (#30630) 623c16393 CUDA: HIP: rms_norm: add variants that load float4 at a time and store x in registers (#29720) 13b804892 common : read ngram cache parts by reference instead of copying (#30061) 0f4376268 convert : support text-only checkpoints in pplx-decider (#30575) 3dbcb7fca cpu : reduce AVX2 tiled accumulators together (#30296) c80475452 ggml-cpu: refactor s309x repack helper functions (#30588) c0083e305 sycl : reduce the build threads to half (#30236) 13de11e78 sycl: wide stores for the q4_K/q5_K weight dequant (#29696) 88aaf325a tests : catch vanished rows in the fusion baseline (#30618) a8d9a7e1e vulkan: fix gpu-visible host memory issues (#30468) f12aaa344 Fix failing RocM tests (#30611) 5e03a1ff4 graph : add precision handling for MELLUM architecture (#30584) b797b9ca8 hexagon: tensor repacking rework and support for tiled and flat quantized and fp weights (#30635) 6afa9a313 llama : share the k-pool input views between layers (#30613) b6334e37e Limit max CUDA pinned host buffer size to 4 GB (#30449) e707790c7 sycl: support reordered Q8_0 MoE weights (#29337) 43d57fb8c [SYCL] Enable ssl in build script and CI (#30456) 2851b3bcb mtmd: add mtmd_audio_stream + support voxtral-realtime (#30629) 59d5fad7c convert : fix Granite Switch block_count for MultiSwitch checkpoints (#30621) e287bd872 convert : fix regression in mapping of Granite speech tensors (#30628) 2f729cfa4 mtmd: lightonocr does not require longest edge to always be 1540, native resolution is the default (#30600) ba3c05f71 model: add Muse Spark Open (text, vision and audio) (#30616) b76b13379 pi : expand AI disclosure with hardware and stats (#30615)