Qwen

An update to the Qwen 3.5 series targeting one of the most widely used sizes.

The long-awaited update to Qwen is finally here. It comes in various sizes from 0.8B to 27B (dense) and 35B-A3B to 397B-A17B (MoE), some of them even with base models. All of them are multi-modal, use reasoning by default and are based on the Qwen-Next architecture with GDN layers. We tested these models over the last few days, and they are a clear upgrade over the previous version: There are a lot of substantial improvements across the board, making them perfect workhorses for a wide range of tasks. Their style and instruction-following have improved, and the models are even better at multilingual tasks, covering more languages. However, at least the small models (still) tend to overthink. You can turn off reasoning by disabling it in the chat template.

The last model of the Qwen-Next series, whose architecture is used for Qwen3.5. This is a capable (and fast) coding model with just 3B active parameters.

An ASR model for 52 languages and dialects by the Qwen team.

A speech generation model covering 10 major languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian).

The Qwen embedding series goes multimodal, too.

An update to Qwen-Image.

A new image generation model by Qwen which is able to decompose images into layers, similar to Photoshop. This makes image editing more accessible to everyone.

The 2B and 32B version of Qwen3 also get an update. Similar to the 8B version covered in last month's artifact, the 32B vision model has better text benchmark scores than the initial release of the 32B text-only model.

The Qwen VL series finally gets its long-awaited and anticipated update with small (4B, 8B) dense and larger (30B-A3B, 235B-A22B) MoEs in both instruct and reasoning versions. We want to shine a special spotlight on the 8B variants: Their text benchmarks have also improved across the board compared to the initial 8B release - reinforcing our point on the challenge of hybrid reasoning. As the 8B versions did not get a 2507 refresh, these versions should be a no-brainer update and drop-in replacement if you were using Qwen3 8B (or are still using Llama3.1 8B).

An editing variant of Qwen-Image.

In case you don't have enough Qwens in your tool-box: Of course the Omni model also gets an update to the current version.

Of course, Qwen is also exploring different architectures, releasing a LLM with hybrid attention, consisting of Gated DeltaNet and Gated Attention. This model is trained on over 15T tokens and could be the groundwork for the next generation of Qwen models. Junyang Lin writes in a tweet about this series:

The image editing version of Qwen-Image (generation) from last episode.

In the last episode, we highlighted the new instruct version of their large MoEs, as Qwen moves away from hybrid reasoning models. This cycle we want to highlight the new 4B versions (Instruct-2507, Thinking-2507). While everyone keeps training and releasing MoE models for very good reasons, dense models are the backbone for academic research and the local community. Qwen3 4B, especially in the instruct-only version, is a great model and I (Florian) have started using it locally for simple tasks (like translation), as the model is really capable and the overall latency is faster than sending requests to the cloud.

Yes, we know! Yet another Qwen model featured prominently in this series. But they are impossible to ignore, release model after model with (OSI-approved) licenses, and the models are good! For image generation, FLUX.1 dev racks up millions of downloads despite its non-commercial license. Qwen-Image uses Apache 2.0 and the outputs are really good. However, it is quite a chonky model, clocking in at 20B params (Flux.1 dev uses 13B params), which might hinder local adoption. However, there are first attempts at distillation to run it quicker.

The small MoE model also gets the same treatment as the large model: A split into a reasoning and non-reasoning version, 1M context and a performance boost across the board.

The small MoE model also gets the same treatment as the large model: A split into a reasoning and non-reasoning version, 1M context and a performance boost across the board.

Qwen also released the reasoning-only version of the large MoE. With a simple config change, these models support up to 1M context.

While the dual-thinking mode wasn't introduced by Qwen, they helped popularize it with a simple mode switch by including either /no_think or /think in the prompt. To the surprise of some, Qwen is abandoning the concept after talking to the community and released an update to the big (now non-reasoning) MoE model. The scores are impressive (beating the Kimi K2 model we recently hyped as a major release), including 41.8 on ARC-AGI. It turns out training a hybrid reasoning model is more challenging technically than it is worth relative to the upside of downstream serving (where training two separate models, one thinker one not, is much easier). The best part of this release is that it has come with multiple reports of strong vibe tests. Historically, Qwen has been known to be among the benchmark-maximizing labs - there are a few papers that have come out recently highlighting signs of data contamination in Qwen base models - but the Qwen models are improving in the robustness of normal testing. We've written multiple times on Interconnects about how labs will first shoot for strong benchmarks to get on the map, and then move to models that are more precisely those that people want to use. Quoting from our Qwen 3 post: "We'll start to see if Qwen has taste/vibes. They have the benchmarks complete, and now we'll see how they compare to the likes of R1, o3, and Gemini 2.5 Pro for staying power at the frontier." Qwen team members mentioned a new flagship thinking model is on the way and joked about coding models coming soon. The evaluation scores are below relative to other models without a <think> section. Again, as we mentioned in our Kimi K2 post, these models are trained extensively with reinforcement learning still, but the goals of the model are more constrained. These instruct, non-thinking, models are best for when the user wants a fast time-to-first token or other automation tasks. The evaluation summary is here:

Besides all the Qwen models from the last episode, Qwen just can't stop releasing new models. Qwen3 Coder comes in two different sizes - the larger of which is actually competitive with Sonnet in various tests. The model is so popular that inference providers offer subscriptions to use it akin to Claude Max to gain market share. As we mentioned last time, Qwen is becoming increasingly professionalized and playing the Google playbook, offering 1,000-2,000 free requests per day (depending on location) for their Gemini CLI fork. Qwen thus takes coding very seriously and is certainly a credible alternative to all those closed-source models.

Qwen has entered the retrieval scene as well. While the community already fine-tuned Qwen2 and Qwen2.5 for various embedding tasks, the Qwen team has now started to release models on their own. As usual for the team, the models are solid in the related benchmarks and likely perform well in downstream tasks.

To complete the RAG pipeline, Qwen also provides a reranker alongside the embedding model.

Qwen released a large preference model, based on Qwen2.5 72B, trained on 15M preference pairs from StackExchange, Reddit and Quora. They demonstrate that the model follows similar scaling laws as language models and that the preference data generalizes across domains.

A small model from the Qwen2.5-Omni series.

A multimodal model from the Qwen team that can handle text, images, audio, and video inputs while generating text or audio outputs. The model features a Time-aligned Multimodal RoPE implementation for effective positional encoding of video content with synchronized audio.

While Qwen2.5 VL is not new, the 32B version was missing. The outputs from this model differ from the rest of the series, being more focused on reasoning. This brings the performance close to the 72B version.

The final version of QwQ, the reasoning model by Alibaba's Qwen, which was previously released as a preview version. It is also released under Apache 2.0. It is a strong model, especially considering its size.

Qwen updated their vision models. Like the previous generation, they are among the strongest options out there and can rival their closed competitors. Besides the usual capabilities (like OCR, bounding boxes, and video understanding), this model iteration is capable of parsing documents into HTML. However, the licensing is all over the place: The 3B version uses the non-commercial Qwen research license, while the 7B model has the Apache 2.0 license; the 72B version has the Qwen license.

Minimax is not the only model breaching a context size of a million tokens. They use Dual Chunk Attention with eight phases of long-context training. The accompanying blog (and technical report) goes into more detail. They also contributed to vLLM to make the models easily runnable for others, which will help more long-context tooling be built around open LMs in the future! The long-context recipe is below.

A version of QwQ which is vision-capable.

The Qwen models, which are known for their performance, also gained a vision update by adding a vision encoder to the Qwen 2 model series. It is capable of handling arbitrary image sizes and videos.

A preview of Qwen's reasoning model, which was released before R1, so researchers have been using this as a model for their experiments over R1 and its distilled versions.

Qwen2.5 is far from an insider tip anymore, the Coder models are a continuous pre-training on top of Qwen2.5, breaching the 20T threshold in the open model space for the first time.

The two models of interest are the Qwen2.5-72B-Instruct and Qwen2.5-Math-RM-72B by Qwen. Good reward models are far and few between these days, especially for Math. The Instruct model has been scoring extremely high on evals and some vibe checks.

The two models of interest are the Qwen2.5-72B-Instruct and Qwen2.5-Math-RM-72B by Qwen. Good reward models are far and few between these days, especially for Math. The Instruct model has been scoring extremely high on evals and some vibe checks.

This, and Qwen 2.5 math and the DeepSeek Math models, are really pushing the MATH scores of open models. The DeepSeek paper is great, and supposedly more Qwen reports are coming soon.

Another very strong and recent open model. The instruct version came in around the same level of Command R Plus, but is the top open-weight Chinese model on LMSYS. Two API models, Yi-Large and GLM-4-0520 are still ahead of it (but we don't know what they are).