Needle releases 26M parameter tool call model: using Gemini distillation technology to let AI Agent run on mobile phones

Core conclusion

On May 13, 2026, the Cactus team open sourced Needle on Hacker News - a special model for function calling (Tool Calling) with only 26M parameters. Its core breakthrough is to prove that tool calls are essentially retrieval assembly tasks rather than inference tasks, and do not require large models with tens of billions of parameters.

Needle reaches a prefill speed of 6000 tok/s and a decode speed of 1200 tok/s on consumer-grade devices. Training only takes 27 hours (16 TPU v6e) plus 45 minutes of post-training.

Key Points

• Published: 2026-05-12 (HN hot post, 216 points)
• Model size: only 26M parameters, 14MB weight file
• Training cost: 200B token pre-training + 2B token tool calling data post-training
• Core Innovation: Simple Attention Networks (pure attention + gating, no MLP layer)
• Applicable scenarios: Local AI Agent on mobile phones, watches, smart glasses, and IoT devices

Background and trigger events

AI Agent is becoming a consensus direction in the industry, but there is always a bottleneck in deploying large-scale models on consumer devices - a 7B parameter model requires about 4GB of memory, while a 26M parameter only requires 14MB. The Cactus team found that the core of the Agent experience is tool calling, which is essentially a retrieval assembly process: matching query → extracting parameters → outputting JSON.

This does not require the reasoning capabilities of a large model, a small model with cross-attention is enough.

The Cactus team said on HN: "We were deeply dissatisfied with how little effort was put into building an Agent model that could run on cheap phones, so we took matters into our own hands."

Key Impact (by Dimension)

Adaptation suggestions

For individual developers and small teams running AI content and automation tools:

1. Local Agent Experiment: Run Needle on M4 Mac or Linux, and use n8n or OpenClaw to conduct a local Agent experiment without calling the cloud API.
2. Reduce API costs: Use Needle as a pre-filtering layer for tool calls. Only complex inferences are routed to large models. It is expected to reduce API costs by 70%.
3. Deployment: Explore integrating Needle in Home Assistant, Android applications or CLI tools to implement local AI functions

Task List (Example)

• Download Needle model weights (14MB, GitHub release)
• Run the playground locally to test the tool calling ability
• Evaluate whether it can be used as a pre-step in the AI Agent workflow

Example: Running Needle locally

# Needle
git clone https://github.com/cactus-compute/needle.git
cd needle

# （Python 3.10+）
pip install -r requirements.txt

# （15MB）
wget https://huggingface.co/Cactus-Compute/needle/resolve/main/needle.pt

# —
python run.py --model needle.pt --prompt " 10 "

HN

• Simon Willison playground ，， VPS
• Needle ** Siri**
• HN Home Assistant
• Needle CLI ——

• GitHub: cactus-compute/needle
• Hacker News
• Simple Attention Networks

Gemini、OpenAI、Claude、n8n、Hermes Agent. The above are all common names in the AI ​​tool ecosystem. The platform side will automatically match and generate tool_mentions floating cards based on the maintained tools library.

Internal link guidance

• Want to learn to build an AI Agent workflow? See: AI Agent Tools 2026 Complete Tutorial: 5 Tools to Build an Automated Pipeline in 30 Minutes
• Someone used AI Agent to create a SaaS with a monthly income of $5,000: 18-Year-Old Built a $5,000/mo SaaS With AI Agents — Zero Hand-Written Code
• Recommended reading: How to use n8n + ChatGPT to build an AI content automated distribution system: How to build an AI content automated distribution system with n8n + ChatGPT: a complete 30-minute tutorial