Arcee AI
The reasoning version of Trinity, one of the best Western open models. It has topped the OpenRouter charts for a while and can power agentic applications such as OpenClaw.
An ultra-sparse MoE with 400B total and 13B active parameters, trained by an American company. They also released a tech report and two base models, one "true" base model pre-annealing and the base model after the pre-training phase. Many more insights, including technical details and their motivation, can be found in our interview with the founders and pre-training lead:
As we've said in our 2026 predictions, attention variants and alternatives will be one of the main themes in terms of architecture decisions for the coming year. Arcee is experimenting with KDA, which is Kimi's variant of Gated Deltanet.
Arcee is not an unknown entity to the avid Artifacts reader. Now they are coming with a series of models: Nano, a 6B-A1B MoE and Mini, a 26B-A3B MoE are available today and trained on 10T tokens. They also plan to release Large, a 420B-A13B MoE trained on 20T tokens, in the coming weeks. We played with the Mini model and were impressed by its capabilities! As readers know, we're also very happy to highlight new and rapidly improving open model builders in the U.S. using permissive licenses.
A competent small model by Arcee, which we featured extensively in a previous episode.
Aside from all the other Arcee models this episode, they also worked at context length extension for their pre-trained models. To do this, they used the capable GLM-4 for experiments and released it. The tech report is well written and goes into detail.
Arcee AI, which we've featured in previous episodes with some of their fine-tunes, have started pre-training their own models, starting with a 4.5B parameter model (blog). This has led to them opening their previously closed models to the public, released under permissive licenses. This model is their flagship model, a 72B fine-tune of Qwen2.5. The move to open older models is really commendable, we hope others will follow suit!
Another model by Arcee: One of the first Qwen3-based fine-tunes, whose data is used to fine-tune Mistral-Nemo, a 12B model.
A GRPO fine-tuned model on top of R1-Distill-Qwen-7B.
A "true", logit-based distillation from DeepSeek V3 logits onto Mistral Small 24B, similar to Virtuoso Medium, which we've covered in the previous episode.
A "true" distillation of DeepSeek V3 onto Qwen2.5-32B. Most models use supervised fine-tuning on outputs generated by the teacher, but this model is trained directly on V3's logits. For this to work, the model had to be re-trained on the V3 tokenizer, then distilled to finally use the original tokenizer and post-train the model.
A fine-tune of Qwen 2.5-14B-Instruct that is distilled from a mix of Llama 3.1-405B and Qwen 2.5-72B-Instruct with a "cross-architecture distillation pipeline." I wish it was clearer what that means.
At the same time, there's not a good academic and evaluation ecosystem for these types of models. We've seen other strong open models in the past that are fine-tuned multiple times. Nexusflow released Athene 70b just before Llama 3.1 and it was very strong, Acree AI has a strong model, the top models on the RewardBench evaluation benchmark are this, a 9B model fine-tuned from Gemma 9B Instruct matches Google's Gemma 27B fine-tune in ChatBotArena, and more examples I missed. Reflection is one of these. Llama-3.1-SuperNova-Lite by arcee-ai: A fine-tune trained with the distillation loss from Llama 405B. I've been waiting for this to happen! More on this soon.