On August 26, 2026, Alibaba's Qwen team released the weights of the Qwen3.8-Flash-Next model — a multimodal MoE architecture with 125 billion parameters in the main model and an additional 51 billion in the N-gram embedding layer, with only 6 billion parameters activated per token. This is an early preview of the Qwen4 architecture, intended to allow the community to adapt infrastructure and tools before the full release of the Qwen4 family.
The model architecture introduces four key innovations: hybrid attention combining Gated DeltaNet (GDN) with Qwen Sparse Attention (QSA), gated residual connections, N-gram embedding, and the Muon optimizer. Compared to Qwen3.7-Plus (397 billion parameters, 17 billion active), training cost about nine times less, while performance in coding and office scenarios noticeably improved. This resulted not just from incremental scaling, but from a radical reworking of activation mechanisms and optimization.
Official benchmarks show superiority over previous Qwen versions and competitive results: 62,5 on SWE-bench Pro (compared to Claude Opus 4.6 data), 73,9 on CoWorkBench, 58,7 on DeepSWE 1.1, and 84,5 on AndroidWorld. The native context of 262 thousand tokens extends to 1 million via YaRN. With 90% prefill cache hit rate, Qwen3.8-Flash-Next achieves a 8,6-fold increase in prefill throughput compared to Qwen3.7-Plus, and at 1 million tokens, QSA provides 7,6x attention acceleration on prefill and 4,9x on decode.
It is important to note that the evaluation methodology relies primarily on Qwen's internal tests, including agentic coding and office benchmarks. At the time of release, there are no independent third-party verifications, and the published comparisons are selective — mainly on tasks where sparse activation wins. This is typical for Asian labs, where full training and data details remain closed, but Alibaba chose to open the weights for early community feedback.
In the MoE model landscape, Qwen3.8-Flash-Next differs from DeepSeek-V4-Flash (284 billion / 13 billion active) by having fewer active parameters while achieving comparable or better performance on agentic tasks. This difference in architectural choices reflects a broader shift in approaches: while American labs continue to invest in scaling dense models, Chinese teams increasingly bet on sparsity, hybrid attention, and specialized embedding layers (like N-gram) to reduce training and inference costs.
The release of open weights for Qwen3.8-Flash-Next allows developers to test the architecture before full Qwen4. vLLM and SGLang already provide day-0 support on NVIDIA and AMD GPUs, which accelerates practical integration: engineers will be able to verify how well QSA hybrid attention and the Muon optimizer scale on real workloads and in which scenarios sparse activation yields the greatest gains.
It remains unclear how robust the claimed advantages are under independent reproduction and on a broader range of tasks, including complex multimodal scenarios beyond coding. Subsequent research will likely focus on ablations of individual architectural components and comparisons of QSA with other hybrid and sparse approaches.
The main takeaway from the release is a demonstration that at the AI frontier, further progress in efficiency is achieved not only by increasing parameters, but also through radical reworking of activation mechanisms, hybrid attention, and optimization. In the open-weights ecosystem, this creates real competition for closed frontier models on the basis of cost and engineering efficiency.
