]> git.djapps.eu Git - pkg/ggml/sources/ggml/commit
llama: Add support for RWKV v7 architecture (llama/12412)
authorMolly Sophia <redacted>
Mon, 17 Mar 2025 23:27:50 +0000 (07:27 +0800)
committerGeorgi Gerganov <redacted>
Thu, 27 Mar 2025 07:35:24 +0000 (09:35 +0200)
commiteaa28e09c25613ba6073f9fb0807aa9103cf8f77
tree050c39e773a29fcee28a7dc963fd8e302417e883
parent75ddfe4fa97ce6cbc627c3040467e6d63db86fd7
llama: Add support for RWKV v7 architecture (llama/12412)

* ggml: Add op l2_norm

Signed-off-by: Molly Sophia <redacted>
* ggml: Add op rwkv_wkv7

Signed-off-by: Molly Sophia <redacted>
* llama: Add support for RWKV7 and ARWKV7 models

Signed-off-by: Molly Sophia <redacted>
* llama: fix inference with RWKV6Qwen2

Signed-off-by: Molly Sophia <redacted>
* llama: add more (a)rwkv7 variants in size

Signed-off-by: Molly Sophia <redacted>
* Apply code-format changes

Signed-off-by: Molly Sophia <redacted>
* fix MUSA build

Signed-off-by: Molly Sophia <redacted>
* llama: fix shape error with rwkv using llama-parallel

Signed-off-by: Molly Sophia <redacted>
---------

Signed-off-by: Molly Sophia <redacted>
22 files changed:
include/ggml.h
src/ggml-cpu/ggml-cpu.c
src/ggml-cuda/ggml-cuda.cu
src/ggml-cuda/norm.cu
src/ggml-cuda/norm.cuh
src/ggml-cuda/wkv.cu [new file with mode: 0644]
src/ggml-cuda/wkv.cuh [new file with mode: 0644]
src/ggml-metal/ggml-metal-impl.h
src/ggml-metal/ggml-metal.m
src/ggml-metal/ggml-metal.metal
src/ggml-sycl/backend.hpp
src/ggml-sycl/ggml-sycl.cpp
src/ggml-sycl/norm.cpp
src/ggml-sycl/norm.hpp
src/ggml-sycl/wkv.cpp [new file with mode: 0644]
src/ggml-sycl/wkv.hpp [new file with mode: 0644]
src/ggml-vulkan/ggml-vulkan.cpp
src/ggml-vulkan/vulkan-shaders/l2_norm.comp [new file with mode: 0644]
src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
src/ggml-vulkan/vulkan-shaders/wkv7.comp [new file with mode: 0644]
src/ggml.c
tests/test-backend-ops.cpp