granite-3.2-8b-instruct-preview

IBM is constantly churning out new (mostly small-ish) models, but is still a relatively unknown player among the model makers. This model is their first shot at a reasoning model and builds upon Granite 3.1 8B, which we covered in the previous episode. While a lot of reasoning models released recently rely on distillation from R1, IBM uses "own reinforcement learning-based techniques for triggering chain-of-thought reasoning across any domain", which does not need a teacher model as outlined in their blog. Furthermore, the reasoning mode of the model can be toggled on or off by setting a specific system message. It is obvious that the distinction between "normal" LLMs and "reasoning" LLMs will become non-existent in the future as RL becomes an increasingly big part of the training. Future models will learn when and how much inference should be spent before the final answer.