Microsoft open source VibeVoice: Whisper-level speech recognition + built-in speaker separation, free for commercial use under MIT license

Core conclusion

VibeVoice, released by Microsoft in January 2026 and once again arousing heated discussions in the community on April 27, is a full-stack voice AI open source project under the MIT protocol. Not only is it a Whisper-level speech-to-text (STT) model, it also has a built-in Speaker Diarization function that can directly identify "who said what at what time" - this is a new option worthy of attention for content creators, podcast producers, and automated workflows.

Key Points

• Event time: The model was first released on January 21, 2026, and exploded HN again on April 27 due to Simon Willison’s review -Affected objects: content creators, AI automated pipelines, podcast transcription tool users
• Core change: Microsoft has open sourced the 17.3B parameter speech AI model under the MIT license. Mac users can run it locally with the mlx-audio one-line command.

Background and trigger events

On the evening of April 27, well-known Python/AI blogger Simon Willison published a VibeVoice review, showing how to transcribe his latest podcast episode on MacBook Pro with one line of commands. The evaluation results show that for 99 minutes of podcast audio, the processing time is only 524 seconds (about 8 minutes and 45 seconds), 20,248 tokens are output, and the peak memory is about 30GB.

Measured data: 99 minutes of audio → 8.7 minutes to complete processing, generating approximately 38.6 tokens per second, and outputting a JSON structured transcription containing speaker_id.

The original review article quickly received 35 points and a lot of discussion on Hacker News. Community feedback focused on: although the model has slow inference and high memory usage (30-60GB), the built-in speaker separation function is unique among similar open source models.

Key Impact (by Dimension)

Adaptation suggestions

The emergence of VibeVoice means that there is a new option under the MIT protocol in the field of speech-to-text. For readers who use waytoclawearn’s automated content production line, the following directions are worthy of attention:

1. Local transcription replaces cloud API

For batch transcription of podcast/video audio content, VibeVoice + MLX can be done completely on your local Mac without sending audio data to any third party:

# Mac VibeVoice（ mlx-audio）
uv run --with mlx-audio mlx_audio.stt.generate \
  --model mlx-community/VibeVoice-ASR-4bit \
  --audio interview.mp3 --output-path interview \
  --format json --verbose --max-tokens 32768

JSON ， segment text、start、end、duration speaker_id。

2.

VibeVoice speaker_id 、，。Simon ，。

3.

60 ，。 55 ， 1-2 ，， speaker_id 。

• VibeVoice-ASR-4bit（5.71GB） 17.3GB ，
• OpenAI Whisper API
• speaker_id
• VibeVoice /

JSON

{
  "text": "And an open question for me is how many other knowledge work fields are actually prone to these agent loops?",
  "start": 13.85,
  "end": 19.5,
  "duration": 5.65,
  "speaker_id": 0
},
{
  "text": "Now that we have this power, people almost underestimate what they can do with it.",
  "start": 19.5,
  "end": 22.78,
  "duration": 3.28,
  "speaker_id": 1
}

Datasette Lite ，。

（）

Microsoft、OpenAI（Whisper ）、MLX、llama.cpp (can be run through GGUF format)

Internal link guidance

• Want to use AI to build content automation workflow? Watch: n8n + OpenAI
• A real case of automated monetization of audio content: OpenClaw + Claude Automated Publishing: $1,500–$2,500/mo Case Study