A capable MoE with a lot of architectural tricks, including its own attention variant. It was trained entirely on AMD GPUs. An 8B variant has also been released and comes with a tech report.
Specs
Params74B
LicenseApache-2.0
Tags
Resources
Adoption · Hugging Face
RAM @ 30d
Relative Adoption Metric: 0.01×. Below benchmark. Measured at 30 days.
Hugging Face Downloads
6.8K
last 30d
8.6K
all time
HF Likes
49
Relative Adoption Metric contextualizes downloads against the model's size bucket.
Related Models
