Prime Intellect

A post-trained version of GLM-4.5 Air with INTELLECT-3's performance coming close or even surpassing the bigger GLM-4.5. The paper goes into the challenges, especially in terms of infrastructure, of RL training models of this size.

After INTELLECT-1, the globally pre-trained 10B model on 1T tokens, the PI team is tackling globally decentralized post-training. This model, based on QwQ 32B, is an impressive technical achievement. Their technical report goes into more detail.

A 10B model based on the Llama architecture. The model was trained and distributed on a global scale, spanning multiple countries and even continents. Distributed training could have substantial policy implications. For now, we know it works okay. Many policy changes could make international players invest way more in these techniques. PrimeIntellect is smart about it - they train on smaller blocks of GPUs not reserved by customers. Might as well figure out how to put those extra GPUs to use?