ZAYA1-8B

A capable MoE with a lot of architectural tricks, including its own attention variant. It was trained entirely on AMD GPUs. An 8B variant has also been released and comes with a tech report.