liminfei-amd [Wed, 24 Jun 2026 09:42:03 +0000 (17:42 +0800)]
vulkan: fail the build when a shader fails to compile (#24450)
* vulkan-shaders-gen: fail the build when a shader fails to compile
vulkan-shaders-gen did not detect shader-compile subprocess failures, so a
broken libggml-vulkan could be produced while the build reported success and
the breakage only surfaced at run time. execute_command() discarded the child
exit code (POSIX waitpid passed nullptr for status; the Windows branch never
called GetExitCodeProcess) and string_to_spv decided success only from whether
stderr was empty, so a non-zero exit with empty stderr, or a subprocess that
failed to launch, was treated as success.
Return the child exit code from execute_command() (WEXITSTATUS on POSIX,
GetExitCodeProcess on Windows), treat a non-zero exit or non-empty stderr or a
launch exception as a failure, and record it in an atomic flag. main() checks
the flag after process_shaders() and returns EXIT_FAILURE before writing the
output files, so the build stops instead of emitting a broken backend.
Fixes #24393
Signed-off-by: liminfei-amd <redacted>
* vulkan-shaders-gen: simplify compile_failed access and drop unreachable return
Address review feedback on #24450:
- Access the std::atomic<bool> compile_failed directly (= / implicit bool)
instead of .store()/.load(); the flag stays atomic because the worker
threads in process_shaders() set it concurrently.
- Remove the unreachable trailing return -1 in execute_command(): on POSIX the
child _exit()s after execvp and the parent returns (fork()<0 throws); on
Windows the block returns the exit code.
Pascal [Wed, 24 Jun 2026 08:50:44 +0000 (10:50 +0200)]
ui: loading bar below the model picker (#24931)
* ui: show model load progress on the selector trigger
Mirror the in-dropdown stage progress as a thin bar on the selector
trigger, so the active model's load percent stays visible when the menu
is closed. Same status gating and composite fraction as the dropdown
row, so both bars track the selected model in sync.
Suggested-by: Julien Chaumond <@julien-c>
* ui: show model load progress bar on the in-conversation model selector
* ui: tune model load indicator to a pulsing highlight (suggested by @ngxson)
Also wire the indicator onto the mobile sheet trigger, which was missing
it since mobile uses the sheet instead of the dropdown.
* ui: thin (@allozaur) pulsating (@ngxson) model load bar
Wyatt Caldwell [Tue, 23 Jun 2026 10:55:46 +0000 (03:55 -0700)]
vulkan: link ggml-cpu when GGML_VULKAN_CHECK_RESULTS / RUN_TESTS are enabled (#24444)
The result-checking and test debug paths in ggml-vulkan.cpp call ggml_graph_compute_with_ctx() to compute a CPU reference graph, but that symbol is defined in ggml-cpu, which ggml-vulkan does not link. Enabling -DGGML_VULKAN_CHECK_RESULTS=ON (or -DGGML_VULKAN_RUN_TESTS=ON) therefore fails to link with an unresolved external (e.g. LNK2019 on MSVC, undefined reference on GCC/Clang). This regressed after ggml-cpu was split into its own library. Link ggml-cpu under those two options so the debug builds link again.
Gabe Goodhart [Tue, 23 Jun 2026 10:03:31 +0000 (04:03 -0600)]
model: Granite Speech Plus (#24818)
* feat: Add conversion support for Granite Speech Plus
Branch: GraniteSpeechPlus
AI-usage: full (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <redacted>
* feat: Extend granite_speech to support plus multi-layer concatenation
Branch: GraniteSpeechPlus
AI-usage: draft (Bob, OpenCode + Qwen3.6-35b) Signed-off-by: Gabe Goodhart <redacted>
* fix(conversion): Fix plural naming for feature_layers for audio
Branch: GraniteSpeechPlus
AI-usage: none Signed-off-by: Gabe Goodhart <redacted>
* fix(mtmd): Align feature_layer usage and naming everywhere
Branch: GraniteSpeechPlus
AI-usage: none Signed-off-by: Gabe Goodhart <redacted>
* style: Use fstring for log
Signed-off-by: Gabe Goodhart <redacted> Co-authored-by: Xuan-Son Nguyen <redacted>
---------
Pascal [Mon, 22 Jun 2026 08:55:28 +0000 (10:55 +0200)]
server: fix edit_file crash on append at end of file (line_start -1) (#24893)
line_start -1 normalized to n+1, so append inserted at lines.begin() + n + 1,
one past end() -> heap-buffer-overflow in vector::_M_range_insert.
Normalize -1 to n (insert at end()), restrict -1 to append mode and reject it
for replace/delete instead of silently clobbering the last line. Parenthesize
the insert offset so empty-file append computes the position as int first,
avoiding a transient begin() - 1 on a null vector data pointer.
make_utf8_argv rebuilds argv from GetCommandLineW to fix utf8 handling of
non ascii arguments on windows. the override runs unconditionally inside
common_params_parse, so it also clobbers a programmatic argv passed by a
caller. test-arg-parser builds a synthetic argv but then sees the real
process command line instead, the model argument is never parsed, and the
assert that expects success aborts via fastfail (0xC0000409). this shows up
as a random failure in the openvino windows workflow.
only override argv when its length matches the caller argc, so the utf8
repair still applies to real binaries while a programmatic argv stays intact.
davidrhodus [Sat, 20 Jun 2026 10:48:24 +0000 (03:48 -0700)]
model : glm-dsa load DSA indexer tensors as optional (#24770)
GLM-5.2 ships the DSA "lightning indexer" on only a subset of layers (the
"full" layers; others omit it), but the GLM_DSA loader created the five
indexer tensors on every layer as required, so loading any GLM-5.2 GGUF
failed with e.g. `missing tensor 'blk.3.indexer.k_norm.weight'`.
GLM_DSA's graph is llama_model_deepseek2::graph (plain MLA) and does not use
the indexer tensors (indexer runtime not yet implemented), so they are
loaded-but-unused. Marking them TENSOR_NOT_REQUIRED lets layers without an
indexer load as nullptr and the model runs as full MLA attention.
DeepSeek-V3.2 (uniform indexer on all layers) is unaffected.
Georgi Gerganov [Fri, 19 Jun 2026 06:22:34 +0000 (09:22 +0300)]
server : consolidate slot selection into get_available_slot (#24755)
Absorb get_slot_by_id logic into get_available_slot so slot selection
is handled by a single function call. When a specific slot id is
requested, the LCP similarity check still runs to enable proper
prompt cache updates.
shalinib-ibm [Fri, 19 Jun 2026 05:55:38 +0000 (11:25 +0530)]
ggml-cpu: support K tails in power10 Q8/Q4 MMA matmul (#24753)
* ggml-cpu: support K tails in Power10 MMA Q8/Q4 matmul
This patch removes the requirement that K be divisible by kc in the tinyBlas_Q0_PPC tiled matmul path. Process the final K panel using its actual depth and pass the reduced panel size through packing and kernel execution. This allows more workloads to use the MMA kernel and reduces fallback to mnpack.
Reguna [Thu, 18 Jun 2026 20:01:24 +0000 (04:01 +0800)]
server: add "X-Accel-Buffering": "no" header to streaming endpoints (#24774)
* server: add "X-Accel-Buffering": "no" header to streaming endpoints
This header tells Nginx (as a reverse proxy) to NOT buffer responses. (only affects streaming endpoints)
Without it, Nginx will break streaming with certain applications (notably the Pi coding harness).
Amos Wong [Thu, 18 Jun 2026 11:14:20 +0000 (19:14 +0800)]
ui: provide touch accessible model selection UI (#24604)
* ui : add model selector storybook stories
Covers list, favorites, single-model, all status states
(loading/loaded/sleeping/failed/idle), and selection states.
* ui : improve model selector mobile UX with hover media queries
Use @media (hover:none) to show action buttons directly on touch
devices and color-code them by model status (amber=sleeping,
green=loaded, muted=idle). Status dots hidden on touch. Desktop
hover behavior unchanged.
Anuj Attri [Thu, 18 Jun 2026 10:49:14 +0000 (06:49 -0400)]
server : return HTTP 400 on invalid grammar (#24144) (#24154)
Throw on grammar parse failure so the server returns HTTP 400
instead of silently dropping the constraint.
Add a regression test for the invalid-grammar response.
shalinib-ibm [Wed, 17 Jun 2026 18:45:19 +0000 (00:15 +0530)]
ggml-cpu: Conditionally enable power11 backend based on compiler support (#24687)
* ggml: Conditionally enable power11 backend based on compiler support
Guard POWER11 backend creation behind a compiler flag check for -mcpu=power11. This avoids build failures on current GCC/Clang toolchains while preserving forward compatibility once POWER11 support becomes available.
Georgi Gerganov [Wed, 17 Jun 2026 16:38:55 +0000 (19:38 +0300)]
metal : add f16 and bf16 support for concat operator (#24724)
* metal : add f16 and bf16 support for concat operator
Extend the Metal backend concat operator to support f16 and bf16 tensor
types in addition to the existing f32 and i32 support.
- Template kernel_concat on type T with specializations for float, half,
bfloat, and int
- Add type-specific pipeline getter ggml_metal_library_get_pipeline_concat()
- Update device support check to allow f16 unconditionally and bf16 when
device supports bfloat16
- Update dispatch to select the correct kernel specialization by type
Assisted-by: pi:llama.cpp/Qwen3.6-27B
* metal : extend concat operator to support f16, bf16, i8, i16 and i64
overview / pkg / ggml / sources / llama.cpp / log