NovaSky-AI

An improved version of Sky-T1 with shorter responses for simpler questions. This is achieved by picking shorter, but still, correct responses, rewriting the responses, and by using a length-normalized version of DPO.

A model fine-tuned from Qwen2.5 32B by training it on outputs of QwQ-32B and rejecting invalid outputs by checking them against the ground truth (for math problems) or running the unit tests (for coding problems). We will see more of these models in the future, similar to the emergence of Alpaca and related models in 2023.