A capable MoE with a lot of architectural tricks, including its own attention variant. It was trained entirely on AMD GPUs. An 8B variant has also been released and comes with a tech report.
Specs
Params8B
LicenseApache-2.0
Tags
Resources
Adoption · Hugging Face
RAM score
Relative Adoption Metric not scored because required parameter, download, or API metadata is not cataloged. This is not a zero score.
Hugging Face Downloads
5.4K
last 30d
350.7K
all time
HF Likes
598
Relative Adoption Metric not scored because required parameter, download, or API metadata is not cataloged. This is not a zero score.
Related Models
