Skip to content
CAI
Software that uses CAICheck a score

microsoft/VibeVoice

47.3

Weak · 14 November 2026

16.5k

lines of production code

Python

primary language

6

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

VibeVoice is an AI system providing high-performance Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) capabilities with a focus on real-time streaming inference. It supports model customization through LoRA fine-tuning and integrates with vLLM for scalable, high-throughput serving of ASR workloads. The system includes comprehensive tooling for deployment, tokenizer preparation, and various demo interfaces to facilitate testing and usage across different hardware configurations.

Features

Add vLLM plugin for high-performance VibeVoice ASR serving

Introduces a new vLLM plugin that enables the VibeVoice ASR model to be loaded and served via vLLM for high-performance inference. The plugin registers the VibeVoice model architecture, configuration, tokenizer, and processor with their respective registries, allowing seamless integration with vLLM's multimodal pipeline. It includes an audio input mapper that handles various audio formats (file paths, bytes, numpy arrays) and validates audio duration to prevent out-of-memory errors, with a configurable limit via the VIBEVOICE\_MAX\_AUDIO\_DURATION environment variable. The implementation patches vLLM's AudioMediaIO to use FFmpeg for consistent audio decoding across formats.

_vllm\plugin · high confidence

Added tokenizer generation tool for VibeVoice ASR support

A new standalone script, generate\_tokenizer\_files.py, has been added to the vLLM plugin tools to prepare tokenizers for high-performance Automatic Speech Recognition (ASR) serving. This tool downloads the base Qwen2.5 tokenizer and patches it with VibeVoice-specific audio tokens and a modified chat template that correctly handles audio inputs, enabling the plugin to serve streaming ASR workloads.

_vllm\plugin/tools · high confidence

Introduce VibeVoice ASR and streaming capabilities with LoRA fine-tuning support

This update adds a new VibeVoice-ASR model variant for automatic speech recognition, including a dedicated LoRA fine-tuning script (\finetuning-asr/lora\finetune.py\) and configuration files for 1.5B and 7B model sizes. It also introduces a modular streaming architecture (\VibeVoiceStreamingConfig\, \VibeVoiceStreamingModel\) designed for real-time inference, featuring a split language model structure (text encoding vs. TTS generation) and compatibility layers for transformers \>= 4.57. Additionally, the package now exposes streaming components and processors via \vibevoice/\\init\\.py\ and \vibevoice/modular/\\init\\_.py\.

vibevoice · high confidence

Introduce VibeVoice-Realtime streaming TTS demo

A new web-based demo application has been added to the \demo/web\ directory, providing a user interface for the VibeVoice-Realtime text-to-speech model. The backend (\app.py\) implements a streaming TTS service that loads the model and voice presets, supporting device-specific optimizations (CUDA, MPS, CPU) and safe loading of voice preset files. The frontend (\index.html\) offers a modern UI for text input, voice selection, and real-time audio preview, enabling users to interact with the streaming inference capabilities directly in a browser.

demo/web · high confidence

New LoRA fine-tuning and inference scripts for VibeVoice ASR

The \finetuning-asr\ directory now includes \lora\_finetune.py\ and \inference\_lora.py\ scripts, enabling users to fine-tune the VibeVoice ASR model using Low-Rank Adaptation (LoRA) and perform inference with the resulting adapters. The entry point \inference\_lora.py\ loads a base model and LoRA weights via the \peft\ library, supporting features like context injection and configurable generation parameters. A \toy\_dataset\ with synthetic audio and JSON labels is provided for demonstration, alongside a \README\ detailing the data format, training hyperparameters, and usage instructions.

finetuning-asr · high confidence

New VibeVoice demos: streaming ASR, realtime TTS, and Colab notebook

The demo area now includes a suite of new demonstration scripts and a Colab notebook for VibeVoice. A new Gradio-based ASR demo (vibevoice\_asr\_gradio\_demo.py) and a FastAPI streaming ASR demo (vibevoice\_asr\_streaming\_fastapi\_demo.py) enable real-time speech-to-text transcription, with the Gradio demo supporting MPS/Apple Silicon and CPU inference. A batch inference script (vibevoice\_asr\_inference\_from\_file.py) and a streaming file-based inference script (vibevoice\_asr\_streaming\_inference\_from\_file.py) are also added for offline and chunked transcription. For text-to-speech, a realtime demo (vibevoice\_realtime\_demo.py) and its corresponding file-based inference script (realtime\_model\_inference\_from\_file.py) are introduced, supporting MPS/Apple Silicon and CPU devices. Additionally, a Colab notebook (vibevoice\_realtime\_colab.ipynb) provides a quickstart for the realtime TTS model, and a shell script (download\_experimental\_voices.sh) automates the download of experimental voice presets.

demo · high confidence

One-click deployment and streaming ASR demos for vLLM plugin

The vllm\_plugin/scripts area now provides a unified one-click deployment script (start\_server.py) that installs dependencies, downloads models, and launches the vLLM server with optional nginx-based data parallel load balancing for improved throughput. Additionally, new demo scripts (fastapi\_asr\_streaming\_demo\_api.py, gradio\_asr\_demo\_api\_video.py, start\_streaming\_server.py) enable users to easily test streaming ASR capabilities via WebSocket or Gradio interfaces, including video support and cloudflared tunneling for remote access.

_vllm\plugin/scripts · high confidence

Behavioural changes

2 commits (0 fixes) modifying demo/voices

A change to existing behaviour in demo/voices — 2 commits, 25 files.

demo/voices · medium confidence · unverified

Test coverage

Added test suite for vLLM ASR plugin features; Added tests for streaming ASR audio input handling.

Dependencies

Initial project setup with Python 3.10+ and core AI dependencies

The project now includes a pyproject.toml manifest defining the package 'vibevoice' with a minimum Python version of 3.10. It establishes core dependencies for AI inference including torch, transformers (constrained to \>=4.51.3,\<5.0.0), accelerate, diffusers, and vLLM plugin support via an entry point. Optional streaming TTS support is available via the 'streamingtts' extra which pins transformers to 4.51.3. Additional dependencies include llvmlite, numba, librosa, gradio, fastapi, and uvicorn for serving and UI capabilities.

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 48 → 47 (-0.4)
  • Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.

Lenses

  • Code Health 60 → 58 (-1.9)
  • Architecture 99 → 98 (-1.4)
  • Maturity 50 → 56 (+6.1)
  • Readiness 41 → 39 (-2.3)
  • Security 41 → 41 (+0.0)
  • Performance 100 → 100 (+0.0)

Resolved (5)

  • Duplicated block (15 lines × 2) (vibevoice/processor/vibevoice_processor.py)
  • Duplicated block (21–22 lines × 2) (vllm_plugin/scripts/start_server.py)
  • Duplicated block (43–47 lines × 2) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
  • No ADRs found
  • Off-boarding risk: anonymized user #1

New (22)

  • Edited copies of one member (3 members, 4+ corresponding lines) (vibevoice/modular/modeling_vibevoice.py)
  • Edited copies of one member (3 members, 55+ corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
  • Edited copy of a member (10 corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
  • Edited copy of a member (116 corresponding lines) (vllm_plugin/scripts/start_server.py)
  • Edited copy of a member (13 corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
  • Edited copy of a member (15 corresponding lines) (demo/vibevoice_asr_gradio_demo.py)
  • Edited copy of a member (19 corresponding lines) (vibevoice/modular/modeling_vibevoice.py)
  • Edited copy of a member (22 corresponding lines) (vllm_plugin/scripts/fastapi_asr_streaming_demo_api.py)
  • Edited copy of a member (24 corresponding lines) (demo/vibevoice_asr_streaming_inference_from_file.py)
  • Edited copy of a member (24 corresponding lines) (vibevoice/modular/configuration_vibevoice.py)
  • Edited copy of a member (29 corresponding lines) (vllm_plugin/model.py)
  • Edited copy of a member (32 corresponding lines) (vllm_plugin/scripts/gradio_asr_demo_api_video.py)
  • Edited copy of a member (4 corresponding lines) (vibevoice/modular/modular_vibevoice_diffusion_head.py)
  • Edited copy of a member (48 corresponding lines) (demo/vibevoice_asr_gradio_demo.py)
  • Edited copy of a member (5 corresponding lines) (vibevoice/processor/vibevoice_processor.py)
  • Edited copy of a member (71 corresponding lines) (demo/vibevoice_asr_gradio_demo.py)
  • Edited copy of a member (8 corresponding lines) (vibevoice/modular/modular_vibevoice_tokenizer.py)
  • Edited copy of a member (8 corresponding lines) (vibevoice/modular/streamer.py)
  • No ADRs found (pyproject.toml)
  • Off-boarding risk: anonymized user #1
  • …and 2 more

Architecture

  • Unchanged — 0 containers · 1 contexts · 0 edges

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

microsoft/VibeVoice was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 14 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 16fb2cb1217c9934a886e1948ffb06120caa2df5 — the exact code this score is about.
  • Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-36d4cd83012f.