A capable MoE with a lot of architectural tricks, including its own attention variant. It was trained entirely on AMD GPUs. An 8B variant has also been released and comes with a tech report.
Specs
Params8B
LicenseApache-2.0
Tags
Resources
Adoption · Hugging Face
RAM score
Relative Adoption Metric not scored because required parameter, download, or API metadata is not cataloged. This is not a zero score.
Hugging Face Downloads
15.4K
last 30d
331.1K
all time
HF Likes
587
Relative Adoption Metric not scored because required parameter, download, or API metadata is not cataloged. This is not a zero score.
Related Models
