DeepSeek releases DeepInfer as open source under Apache 2.0; framework reaches 6,300+ GitHub stars within eight hours
Following·New·2 sources
DeepSeek Open-Sources DeepInfer Inference Framework Under Apache 2.0
The gist
Releasing a production inference stack gives developers direct access to the techniques DeepSeek uses to serve a large-scale model with high throughput and low latency. The scope is limited, however: only one model is supported and several production-serving components are absent from the release.
The full picture
DeepSeek released DeepInfer, its in-house inference framework, as open source under the Apache 2.0 license on October 20, 2026. The framework powers DeepSeek-V4.1-Flash in production. DeepSeek said it is sharing the production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. The release drew over 6,300 GitHub stars within eight hours. DeepInfer currently supports only DeepSeek-V4.1-Flash and runs on NVIDIA SM100 GPUs or Huawei Ascend 950 chips. The release covers offline inference only and omits production-serving components including KV cache management, prefill/decode disaggregation, and load balancing. Notable technical components include DSpark, a speculative decoding module that drafts up to 5 tokens ahead and uses confidence estimates to schedule verification, and a bounded replay technique that reduces prefill cost by recomputing only a 140-token suffix aligned to the sliding attention window. A companion library called deepseek-recipe converts Chat Completions, Responses, and Messages API requests into DeepSeek V4/V4.1 prompts matching official API behavior.
How it developed
Sources
Related
The daily email
Want this in your inbox?
I send one email each morning with the stories that moved. If you would rather just read here, that works too.
Subscribe free