A capable MoE with a lot of architectural tricks, including its own attention variant. It was trained entirely on AMD GPUs. An 8B variant has also been released and comes with a tech report.
Specs
Params74B
LicenseApache-2.0
Tags
modelsmodels/llmsmodels/llms/reasoningartifacts/21
Related Models
