dphn

An uncensored version of Trinity Nano. The blog goes into great detail about the reward design to achieve this.

While the model is meant as a research artifact, the accompanying blog post gives insights into the quirkinesses of (RL) training and is an interesting read.