]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
vulkan: Implement set_tensor_async and the event interfaces (llama/18047)
authorJeff Bolz <redacted>
Sun, 21 Dec 2025 20:52:09 +0000 (14:52 -0600)
committerGeorgi Gerganov <redacted>
Wed, 31 Dec 2025 15:52:09 +0000 (17:52 +0200)
commit98e59a43d13592d9487ae5cc66991f8c4f28c22a
tree6bcc13412b76555fd31211b4ac1bda6d3b7d7053
parentb68b12f2d595add6e283e40697612b12e1657a30
vulkan: Implement set_tensor_async and the event interfaces (llama/18047)

The goal is to enable the async loading code paths in
llama_model_loader::load_all_data, originally from #7896. This works and the
loads themselves are faster, but with host visible vidmem I think the cost of
allocating/mapping vidmem moves and becomes more expensive, and I don't see a
benefit by default. But with GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 I do see a
significant improvement in model loading time.
ggml/src/ggml-vulkan/ggml-vulkan.cpp