DeepSeek releases "Thinking with Visual Primitives", a new multi-modal paradigm: embedding coordinates into thinking chains, 7056 times compression to solve the AI ​​visual reasoning reference gap

Core conclusion

On April 30, 2026, DeepSeek, together with Peking University and Tsinghua University, officially released a new multi-modal model technology paradigm "Thinking with Visual Primitives" on GitHub.

This work goes straight to the fundamental weakness of all current large multimodal models: models can "see" but not necessarily "think clearly." When faced with scenarios such as counting heads in dense crowds or comparing complex circuit diagrams, models such as GPT-5.4 and Claude Sonnet 4.6 will be inaccurate due to the fuzziness of natural language descriptions during the reasoning process.

DeepSeek proposed a disruptive solution - use bounding box coordinates and point coordinates as the basic "thinking units" for reasoning, and embed them in Chain of Thinking (CoT) like text. At the same time, an astonishing 7056 times visual compression is achieved, allowing the model to complete accurate reasoning with very few visual tokens.

Key Points

• Event time: 2026-04-30 21:12 CST (GitHub release + technical report public)
• Participating institutions: DeepSeek, Peking University, Tsinghua University
• Language Backbone: DeepSeek V4-Flash (284B total parameters / 13B MoE model activated during inference)
• Core Innovation: Coordinate embedded thinking chain (Reference Gap solution) + 7056 times visual compression
• Available Status: Online on DeepSeek web and App, open source on GitHub

Background: The “reference gap” of AI visual reasoning

There is a fundamental problem with current large multimodal models: they use natural language to build chains of thought (CoT), but natural language is inherently vague. Descriptions such as "the big one on the left" and "the red object near the center" cannot be accurately positioned in dense scenes. The model's attention gradually "drifts" during the reasoning process, and the more it talks, the more confusing it becomes, and eventually it reaches the wrong conclusion.

DeepSeek named this problem "Reference Gap".

The previous response plan of the academic community was mainly to make the model "see more clearly": high-resolution cutting and dynamic segmentation of the image. But this solves the "perception gap" (Perception Gap), not the problem of reference. DeepSeek's paper clearly states: No matter how strong the perceptual ability is, it cannot replace the accurate "referring ability".

Key innovation: coordinates become "thinking units

Innovation 1: Coordinates are thinking units

During the reasoning process of the model, whenever a visual object is mentioned, its coordinates are output synchronously. For example:

"Scan the picture for bears and find a <|ref|>bear<|/ref|><|box|>[[452,23,804,411]]<|/box|>. It is climbing a tree and is not on the ground. Eliminate it. Look to the lower left and find another one. <|ref|>Bear<|/ref|><|box|>[[50,447,647,771]]<|/box|>, standing on the edge of the rock, meets the conditions."

It's like when humans count things, they use their fingers to point by point. Coordinates are no longer answers but anchors to disambiguate the reasoning process.

Innovation 2: 7056 times visual compression

The model only requires a very small amount of visual tokens to complete reasoning, achieving ultra-large-scale compression and significantly reducing computing costs compared to traditional methods.

Architecture details

• Language Backbone: DeepSeek V4-Flash (284B total parameters / 13B activation)
• Visual Coding: Self-developed ViT (Visual Transformer), supports input at any resolution
• Training philosophy: Use very few visual tokens to teach the model to accurately refer to visual objects during reasoning

Impact on AI Automation

Adaptation suggestions

1. Update visual reasoning pipeline: If the existing automated process relies on visual understanding (such as automatic chart interpretation, screenshot analysis), you can evaluate integrating DeepSeek visual primitives into the pipeline
2. Reduced Visual Inference Cost: Thanks to 7056x compression, more visual verification steps can be introduced within the inference budget
3. Follow open source progress: The project has been open sourced on GitHub (github.com/deepseek-ai/Thinking-with-Visual-Primitives), and the technical report can be freely downloaded

References

• GitHub project: Thinking with Visual Primitives
• Technical Report: PDF
• Machine Heart Report: 36
• DeepSeek V4-Flash: 284B parameter MoE, 13B activated during inference

Tool entry

This report involves the following tools/platforms. The platform side will automatically match the maintained tools library: DeepSeek, Claude, ChatGPT, GPT-5, Claude Code

Internal link guidance

• Want to know how DeepSeek V4 reduces API costs by 97%? Watch: DeepSeek V4 vs Claude Code: 90% Cheaper, Same Quality
• Someone used Claude Code to achieve monthly income $9,000: Real case: Claude Code 48 hours to start a business: one person + US$29 monthly fee, monthly income in 3 months $9,000
• AI Agent tool practical tutorial: AI Agent Tools 2026 Complete Tutorial: 5 Tools to Build an Automated Pipeline in 30 Minutes