microsoft/VibeVoice
47.3
Weak · 14 November 2026
16.5k
lines of production code
Python
primary language
6
measurements over time
What this system is
VibeVoice is an AI system providing high-performance Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) capabilities with a focus on real-time streaming inference. It supports model customization through LoRA fine-tuning and integrates with vLLM for scalable, high-throughput serving of ASR workloads. The system includes comprehensive tooling for deployment, tokenizer preparation, and various demo interfaces to facilitate testing and usage across different hardware configurations.
Features
Add vLLM plugin for high-performance VibeVoice ASR serving
Introduces a new vLLM plugin that enables the VibeVoice ASR model to be loaded and served via vLLM for high-performance inference. The plugin registers the VibeVoice model architecture, configuration, tokenizer, and processor with their respective registries, allowing seamless integration with vLLM's multimodal pipeline. It includes an audio input mapper that handles various audio formats (file paths, bytes, numpy arrays) and validates audio duration to prevent out-of-memory errors, with a configurable limit via the VIBEVOICE\_MAX\_AUDIO\_DURATION environment variable. The implementation patches vLLM's AudioMediaIO to use FFmpeg for consistent audio decoding across formats.
_vllm\plugin · high confidence
Added tokenizer generation tool for VibeVoice ASR support
A new standalone script, generate\_tokenizer\_files.py, has been added to the vLLM plugin tools to prepare tokenizers for high-performance Automatic Speech Recognition (ASR) serving. This tool downloads the base Qwen2.5 tokenizer and patches it with VibeVoice-specific audio tokens and a modified chat template that correctly handles audio inputs, enabling the plugin to serve streaming ASR workloads.
_vllm\plugin/tools · high confidence
Introduce VibeVoice ASR and streaming capabilities with LoRA fine-tuning support
This update adds a new VibeVoice-ASR model variant for automatic speech recognition, including a dedicated LoRA fine-tuning script (\finetuning-asr/lora\finetune.py\) and configuration files for 1.5B and 7B model sizes. It also introduces a modular streaming architecture (\VibeVoiceStreamingConfig\, \VibeVoiceStreamingModel\) designed for real-time inference, featuring a split language model structure (text encoding vs. TTS generation) and compatibility layers for transformers \>= 4.57. Additionally, the package now exposes streaming components and processors via \vibevoice/\\init\\.py\ and \vibevoice/modular/\\init\\_.py\.
vibevoice · high confidence
Introduce VibeVoice-Realtime streaming TTS demo
A new web-based demo application has been added to the \demo/web\ directory, providing a user interface for the VibeVoice-Realtime text-to-speech model. The backend (\app.py\) implements a streaming TTS service that loads the model and voice presets, supporting device-specific optimizations (CUDA, MPS, CPU) and safe loading of voice preset files. The frontend (\index.html\) offers a modern UI for text input, voice selection, and real-time audio preview, enabling users to interact with the streaming inference capabilities directly in a browser.
demo/web · high confidence
New LoRA fine-tuning and inference scripts for VibeVoice ASR
The \finetuning-asr\ directory now includes \lora\_finetune.py\ and \inference\_lora.py\ scripts, enabling users to fine-tune the VibeVoice ASR model using Low-Rank Adaptation (LoRA) and perform inference with the resulting adapters. The entry point \inference\_lora.py\ loads a base model and LoRA weights via the \peft\ library, supporting features like context injection and configurable generation parameters. A \toy\_dataset\ with synthetic audio and JSON labels is provided for demonstration, alongside a \README\ detailing the data format, training hyperparameters, and usage instructions.
finetuning-asr · high confidence
New VibeVoice demos: streaming ASR, realtime TTS, and Colab notebook
The demo area now includes a suite of new demonstration scripts and a Colab notebook for VibeVoice. A new Gradio-based ASR demo (vibevoice\_asr\_gradio\_demo.py) and a FastAPI streaming ASR demo (vibevoice\_asr\_streaming\_fastapi\_demo.py) enable real-time speech-to-text transcription, with the Gradio demo supporting MPS/Apple Silicon and CPU inference. A batch inference script (vibevoice\_asr\_inference\_from\_file.py) and a streaming file-based inference script (vibevoice\_asr\_streaming\_inference\_from\_file.py) are also added for offline and chunked transcription. For text-to-speech, a realtime demo (vibevoice\_realtime\_demo.py) and its corresponding file-based inference script (realtime\_model\_inference\_from\_file.py) are introduced, supporting MPS/Apple Silicon and CPU devices. Additionally, a Colab notebook (vibevoice\_realtime\_colab.ipynb) provides a quickstart for the realtime TTS model, and a shell script (download\_experimental\_voices.sh) automates the download of experimental voice presets.
demo · high confidence
One-click deployment and streaming ASR demos for vLLM plugin
The vllm\_plugin/scripts area now provides a unified one-click deployment script (start\_server.py) that installs dependencies, downloads models, and launches the vLLM server with optional nginx-based data parallel load balancing for improved throughput. Additionally, new demo scripts (fastapi\_asr\_streaming\_demo\_api.py, gradio\_asr\_demo\_api\_video.py, start\_streaming\_server.py) enable users to easily test streaming ASR capabilities via WebSocket or Gradio interfaces, including video support and cloudflared tunneling for remote access.
_vllm\plugin/scripts · high confidence
Behavioural changes
2 commits (0 fixes) modifying demo/voices
A change to existing behaviour in demo/voices — 2 commits, 25 files.
demo/voices · medium confidence · unverified
Test coverage
Added test suite for vLLM ASR plugin features; Added tests for streaming ASR audio input handling.
Dependencies
Initial project setup with Python 3.10+ and core AI dependencies
The project now includes a pyproject.toml manifest defining the package 'vibevoice' with a minimum Python version of 3.10. It establishes core dependencies for AI inference including torch, transformers (constrained to \>=4.51.3,\<5.0.0), accelerate, diffusers, and vLLM plugin support via an entry point. Optional streaming TTS support is available via the 'streamingtts' extra which pins transformers to 4.51.3. Additional dependencies include llvmlite, numba, librosa, gradio, fastapi, and uvicorn for serving and UI capabilities.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 48 → 47 (-0.4)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 60 → 58 (-1.9)
- Architecture 99 → 98 (-1.4)
- Maturity 50 → 56 (+6.1)
- Readiness 41 → 39 (-2.3)
- Security 41 → 41 (+0.0)
- Performance 100 → 100 (+0.0)
Resolved (5)
- Duplicated block (15 lines × 2) (vibevoice/processor/vibevoice_processor.py)
- Duplicated block (21–22 lines × 2) (vllm_plugin/scripts/start_server.py)
- Duplicated block (43–47 lines × 2) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
- No ADRs found
- Off-boarding risk: anonymized user #1
New (22)
- Edited copies of one member (3 members, 4+ corresponding lines) (vibevoice/modular/modeling_vibevoice.py)
- Edited copies of one member (3 members, 55+ corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
- Edited copy of a member (10 corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
- Edited copy of a member (116 corresponding lines) (vllm_plugin/scripts/start_server.py)
- Edited copy of a member (13 corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
- Edited copy of a member (15 corresponding lines) (demo/vibevoice_asr_gradio_demo.py)
- Edited copy of a member (19 corresponding lines) (vibevoice/modular/modeling_vibevoice.py)
- Edited copy of a member (22 corresponding lines) (vllm_plugin/scripts/fastapi_asr_streaming_demo_api.py)
- Edited copy of a member (24 corresponding lines) (demo/vibevoice_asr_streaming_inference_from_file.py)
- Edited copy of a member (24 corresponding lines) (vibevoice/modular/configuration_vibevoice.py)
- Edited copy of a member (29 corresponding lines) (vllm_plugin/model.py)
- Edited copy of a member (32 corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
- Edited copy of a member (4 corresponding lines) (vibevoice/modular/modular_vibevoice_diffusion_head.py)
- Edited copy of a member (48 corresponding lines) (demo/vibevoice_asr_gradio_demo.py)
- Edited copy of a member (5 corresponding lines) (vibevoice/processor/vibevoice_processor.py)
- Edited copy of a member (71 corresponding lines) (demo/vibevoice_asr_gradio_demo.py)
- Edited copy of a member (8 corresponding lines) (vibevoice/modular/modular_vibevoice_tokenizer.py)
- Edited copy of a member (8 corresponding lines) (vibevoice/modular/streamer.py)
- No ADRs found (pyproject.toml)
- Off-boarding risk: anonymized user #1
- …and 2 more
Architecture
- Unchanged — 0 containers · 1 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
microsoft/VibeVoice was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 14 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 16fb2cb1217c9934a886e1948ffb06120caa2df5 — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-36d4cd83012f.