OREAL-7B

A bit of a different take on the current rage of reasoning models. Quoting the model card: "Our method leverages best-of-N (BoN) sampling for behavior cloning and reshapes negative sample rewards to ensure gradient consistency. Also, to address the challenge of sparse rewards in long chain-of-thought reasoning, we incorporate an on-policy token-level reward model that identifies key tokens in reasoning trajectories for importance sampling." There are more details in the paper, but mostly this goes to show that reward models (specifically, outcome reward models) are still important to reasoning and it isn't that "explicit verification is all you need."