A conversion of Qwen2.5 72B into the RKWV architecture, which is done by deleting the attention layers and replacing them with RKWV layers, followed by distillation of the logits from the original, transformer-based model. The implication of this process is that the knowledge is in the FeedForward layers, not the attention layers.
Specs
Params72B
LicenseTongyi Qianwen
Tags
Similarity · VAIL
VAIL Fingerprint
0085:00b2:00e0:0137:018d:0205:04a3:4f06
Explore other models with behavioral similarity to QRWKV-72B.
Adoption · Hugging Face
RAM score
—
Hugging Face Downloads
133
last 30d
18.5K
all time
HF Likes
69
Relative Adoption Metric contextualizes downloads against the model's size bucket.
Related Models
