git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit

]> git.djapps.eu Git - pkg/ggml/sources/whisper.cpp/commit

overview / pkg / ggml / sources / whisper.cpp / commit

author	Aman Gupta <redacted>
	Sat, 27 Sep 2025 16:49:32 +0000 (00:49 +0800)
committer	Georgi Gerganov <redacted>
	Mon, 29 Sep 2025 12:18:11 +0000 (15:18 +0300)
commit	85e4455cd3c15f97fc64a542548a2649d626213b
tree	9ff965536b8d15a8a9bc7e4a9293caabb0a5f9e1	tree
parent	e856483cd6adb0014ee6c7e7ebdb6dc6c841944b	commit \| diff

CUDA: mul_mat_id for mmf for bs <= 64 for f16 and bs <= 32 for f32 (llama/16277)

* CUDA: mul_mat_id for mmf for bs <= 64 for f16 and bs <= 32 for f32

This commit adds mul_mat_id support for ncols_dst >= 16. It does this by
packing ncols_dst tiles into the blockDim.y.

My tests on a RTX 3090 show that this is faster than the cuBLAS fallback
for f16 till bs=64, and for f32 till bs=32

* Review: refactor if statement

ggml/src/ggml-cuda/ggml-cuda.cu		diff \| blob \| history
ggml/src/ggml-cuda/mmf.cu		diff \| blob \| history
ggml/src/ggml-cuda/mmf.cuh		diff \| blob \| history

Packaging of ggerganov/whisper.cpp

RSS Atom