Skip to content

Navigation Menu

Appearance settings

View all features
- BY COMPANY SIZE
  Enterprises
  Small and medium teams
  Startups
  Nonprofits
- BY USE CASE
  App Modernization
  DevSecOps
  DevOps
  CI/CD
  View all use cases
- BY INDUSTRY
  Healthcare
  Financial services
  Manufacturing
  Government
  View all industries
View all solutions
- EXPLORE BY TOPIC
  AI
  Software Development
  DevOps
  Security
  View all topics
- EXPLORE BY TYPE
  Customer stories
  Events & webinars
  Ebooks & reports
  Business insights
  GitHub Skills
- SUPPORT & SERVICES
  Documentation
  Customer support
  Community forum
  Trust center
  Partners
View all resources
- COMMUNITY
  GitHub SponsorsFund open source developers
- PROGRAMS
  Security Lab
  Maintainer Community
  Accelerator
  GitHub Stars
  Archive Program
- REPOSITORIES
  Topics
  Trending
  Collections
- ENTERPRISE SOLUTIONS
  Enterprise platformAI-powered developer platform
- AVAILABLE ADD-ONS
  GitHub Advanced SecurityEnterprise-grade security features
  Copilot for BusinessEnterprise-grade AI features
  Premium SupportEnterprise-grade 24/7 support
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

ggml-org / llama.cpp Public

Notifications You must be signed in to change notification settings
Fork 18.2k
Star 110k

Code
Issues 645
Pull requests 983
Discussions
Actions
Projects
Wiki
Security and quality 13
Insights

Additional navigation options

Code
Issues
Pull requests
Discussions
Actions
Projects
Wiki
Security and quality
Insights

llama : support RWKV v6 models#8980

Merged

ggerganov merged 53 commits into

ggml-org:masterggml-org/llama.cpp:masterfrom

MollySophia:for-upstreamMollySophia/llama.cpp:for-upstreamCopy head branch name to clipboard

Sep 1, 2024

Conversation Commits53 (53)Checks Files changed

Merged

llama : support RWKV v6 models#8980
ggerganov merged 53 commits into
ggml-org:masterfrom
MollySophia:for-upstream

Commits

Commits on Aug 28, 2024

convert_hf_to_gguf: Add support for RWKV v6
MollySophia
committed
Add RWKV tokenization

LaylBongers
authored and
MollySophia
committed
Fix build
MollySophia
committed
Do not use special tokens when matching in RWKV tokenizer

LaylBongers
authored and
MollySophia
committed
Fix model loading

LaylBongers
authored and
MollySophia
committed
Add (broken) placeholder graph builder for RWKV

LaylBongers
authored and
MollySophia
committed
Add workaround for kv cache

LaylBongers
authored and
MollySophia
committed
Add logits conversion to rwkv5

LaylBongers
authored and
MollySophia
committed
Add rwkv5 layer norms

LaylBongers
authored and
MollySophia
committed
Add time mix KVRG & correct merge mistake

LaylBongers
authored and
MollySophia
committed
Add remaining time mix parameters

LaylBongers
authored and
MollySophia
committed
Add time mix output loading

LaylBongers
authored and
MollySophia
committed
Add placeholder llm_build_time_mix

LaylBongers
authored and
MollySophia
committed
Fix build
MollySophia
committed
Load more tensors for rwkv v6
MollySophia
committed
Fix rwkv tokenizer
MollySophia
committed
ggml: Add unary operator Exp
MollySophia
committed
RWKV v6 graph building
MollySophia
committed
Add ``rescale_every_n_layers`` parameter
MollySophia
committed
Add ``wkv.head_size`` key for RWKV
MollySophia
committed
Fix offloading layers to CUDA
MollySophia
committed
Fix parallel inferencing for RWKV
MollySophia
committed
Remove trailing whitespaces
MollySophia
committed
build_rwkv: Avoid using inplace operations
MollySophia
committed
convert_hf_to_gguf: rwkv: Avoid using ``eval``
MollySophia
committed
convert_hf_to_gguf: rwkv tokenizer: Don't escape sequences manually
MollySophia
committed
Update convert_hf_to_gguf.py

MollySophia
and
compilade
committed
ggml: Add backward computation for unary op ``exp``
MollySophia
committed
Update convert_hf_to_gguf.py

MollySophia
and
compilade
committed
Update convert_hf_to_gguf.py

MollySophia
and
compilade
committed
Use MODEL_ARCH.RWKV6 instead of MODEL_ARCH.RWKV
MollySophia
committed
build_rwkv6: Simplify graph
MollySophia
committed
llama: rwkv6: Detect model.type
MollySophia
committed
llama: rwkv6: Fix tensor loading for 7B/14B models
MollySophia
committed
llama: rwkv6: Fix group_norm assertion failure with Metal
MollySophia
committed
llama: rwkv6: Clean up
MollySophia
committed
llama: rwkv6: Add quantization tensor exclusion
MollySophia
committed
llama: rwkv6: Use the new advanced batch splits
MollySophia
committed
Update src/llama.cpp

MollySophia
and
compilade
committed
llama: rwkv6: Use ``ggml_norm`` instead of ``ggml_group_norm``

MollySophia
and
compilade
committed
llama: rwkv6: Apply code style and misc changes
MollySophia
committed
converter: Use class name ``Rwkv6Model``
MollySophia
committed
llama: rwkv6: Make use of key ``feed_forward_length``
MollySophia
committed
llama: rwkv6: Add kv ``time_mix_extra_dim`` and ``time_decay_extra_dim``
MollySophia
committed
converter: Match ``new_name`` instead of ``name`` for float32 explicit tensors
MollySophia
committed
llama: rwkv6: Keep ``time_mix_w1/w2`` as F32
MollySophia
committed
llama: rwkv6: Remove unused nodes
MollySophia
committed
llama: rwkv6: Apply code format changes
MollySophia
committed

Commits on Aug 30, 2024

llama: rwkv6: Add lora for some supported tensors
MollySophia
committed
rwkv : speed-up tokenization using trie
ggerganov
committed
minor : style + indentation
ggerganov
committed

Commits on Aug 31, 2024

llama: rwkv6: Avoid division by zero

MollySophia
and
compilade
authored
ggml: rwkv_wkv: Avoid copying the state
MollySophia
committed

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Community
Docs
Contact

You can’t perform that action at this time.