Qwen3.8-Flash and Qwen4 architecture preview
- Alibaba's Qwen team released Qwen3.8-Flash and Qwen3.8-Flash-Next on August 26 as open-weight previews of the Qwen4 architecture, with an FP8 variant also open-sourced at launch.
- The 125B-parameter models activate only 6B per token, add Qwen Sparse Attention, gated residual connections, and a 51B N-gram embedding table that offloads to host RAM; Qwen reports training cost one-ninth that of Qwen3.7-Plus.
- Same-day support arrived from SGLang, vLLM, UnslothAI, and TokenSpeed; the QwenCloud API starts at $0.16 per million input tokens, and the release topped Hacker News with 272 points.
The release combines a novel architecture preview of Qwen4 with competitive cost efficiency claims and broad same-day inference framework support, giving developers immediate access to weights and multiple deployment paths. The N-gram embedding design, if it delivers on the capacity-without-compute claim, represents a departure from standard MoE scaling approaches.