]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit
CUDA: mul_mat_v support for batch sizes > 1 (llama/14262)
authorJohannes Gäßler <redacted>
Mon, 23 Jun 2025 11:11:31 +0000 (13:11 +0200)
committerGeorgi Gerganov <redacted>
Tue, 1 Jul 2025 14:54:53 +0000 (17:54 +0300)
commit06b01ba87b4198f0520fdae9b7dff00824ec6df1
treed9b405a6d4e91e70e4f34d6b7dfafcbb3df8099c
parent791201a974a8ced41afdb5728c175114aa29eaad
CUDA: mul_mat_v support for batch sizes > 1 (llama/14262)

* CUDA: mul_mat_v support for batch sizes > 1

* use 64 bit math for initial offset calculation
ggml/src/ggml-cuda/common.cuh
ggml/src/ggml-cuda/ggml-cuda.cu
ggml/src/ggml-cuda/mmv.cu
ggml/src/ggml-cuda/mmv.cuh