Orthrus-Qwen3 open source: 4-7.8 times accelerated AI reasoning framework, lossless generation Qwen3 compatible

Core conclusion

On May 16, 2026, the open source community welcomed a major project - Orthrus. This is a new dual-architecture inference framework that unifies the precise generation quality of autoregressive LLMs with the efficient parallel generation capabilities of diffusion models. Based on the Qwen3 backbone model, Orthrus achieved 4.25× to 7.8× inference acceleration while maintaining a strictly consistent output distribution.

For those who use AI tools for content automation, this means: Same model capabilities, faster inference speed, and lower token latency. Whether running batch content generation, real-time conversational agents, or local deployment of LLM, this project is worthy of attention.

Key Points

• Release time: 2026-05-13 (Created on GitHub), 2026-05-16 on HN homepage
• Core innovation: Dual-View Diffusion Decoding
• Basic model: all based on Qwen3 backbone (the latest open source series of Alibaba Tongyi Qianwen)
• Acceleration range: 4.25× for 1.7B model → up to 7.8× for 8B model
• Output quality: strictly lossless, completely consistent with the native Qwen3 output distribution
• License: MIT open source, free for commercial use
• Developer: chiến Nguyễn (chiennv2000)

Background

Traditional large language model inference relies on autoregressive decoding - one token is generated each time, and the current token depends on all previous tokens. Although this method produces high quality, its speed is limited by sequential execution and cannot take advantage of the parallel capabilities of the GPU.

Although the diffusion model can be generated in parallel, the generation quality is often not as good as the autoregressive model.

The core idea of ​​​​Orthrus: Maintain two "perspective lines" at the same time - an autoregressive path to ensure generation accuracy, and a diffusion path to achieve parallel acceleration. The two paths are calibrated through the "intra-model consensus mechanism" to ensure that the final output is consistent with the native model.

Model specifications

Orthrus currently offers models in three sizes, all available for download on Hugging Face:

In actual measurements, Orthrus-Qwen3-8B can achieve up to 7.8× acceleration in generation tasks, and the API response speed is close to real-time.

Installation and use

The installation of Orthrus is very simple. It is recommended to use the uv package manager:

#
uv pip install -e .
uv pip install ninja packaging
uv pip install flash-attn --no-build-isolation

Hugging Face Transformers ， generate() use_diffusion_mode=True 。

vLLM SGLang，。

AI

waytoclawearn ，Orthrus

**1. **1.7B 4× ， AI 。 LM Studio ，M4 Mac 。

**2. API **（ DeepSeek V4、Qwen3）， Orthrus 4-7 ， token 。

**3. **（），，。

Orthrus Qwen3、Hugging Face、DeepSeek、LM Studio、vLLM、GitHub.

More information

• GitHub ：chiennv2000/orthrus
• Hugging Face ：chiennv/Orthrus-Qwen3
• HN

Internal link guidance

• Want to run local models on actual devices? Watch the tutorial: How to run local AI models on M4 Mac with LM Studio: A complete 30-minute tutorial
• Want to know more extreme inference optimization solutions? Watch the tutorial: Claude Code + DeepSeek V4 Building Tutorial: API Fees Dropped by 90% (15 Minutes)
• Real case: A real case of a data analyst using Claude Code + n8n to build an automated report SaaS with a monthly income of $3,800