OpenBMB/MiniCPM-V

This system is a multimodal AI framework centered on the MiniCPM-V and MiniCPM-o model families, providing infrastructure for inference, evaluation, and fine-tuning. It supports a range of capabilities including text, image, video, and real-time voice interaction, with dedicated tools for benchmarking performance and adapting models via LoRA or full-parameter tuning. The project also includes web-based demos and comprehensive documentation to facilitate development and usage.

The \eval\_mm\ directory now includes a complete evaluation environment for MiniCPM-V 2.6 and MiniCPM-o 2.6, featuring the \vlmevalkit\ package (version 0.2rc1) for benchmarking on OpenCompass datasets (e.g., MMMU, MathVista, MME) and the \vqaeval\ suite for VQA tasks (TextVQA, DocVQA). This update provides the necessary inference scripts, configuration files, and API wrappers to run these specific model versions against standard multimodal benchmarks.

_eval\mm · high confidence

The web\_demos area now includes a new MiniCPM-o 2.6 demo that adds real-time voice interaction capabilities alongside image and video support. The Python backend (chatbot\_web\_demo\_o2.6.py and model\_server.py) integrates a Voice Activity Detection (VAD) system to handle streaming audio, while the frontend (web\_server) is a new Vue 3 application using Element Plus and Vite, serving as the user interface for the demo.

_web\demos · high confidence

The repository now includes a new HTML logo asset for MiniCPM-O and a comprehensive Markdown file listing the 40+ languages supported by MiniCPM-Llama-V 2.5. Additionally, a star history SVG chart has been added to visualize repository growth, and a temporary placeholder file was created within the MiniCPM-V 2.5 assets directory.

assets · high confidence

The web server for the local chatbot demo has been updated to support MiniCPM-o 2.6. This introduces a new Vue-based frontend interface featuring a call header, a countdown timer for call duration, and a model output area that displays text and audio responses. The demo includes a model configuration panel allowing users to adjust settings such as audio interruption, video quality, VAD threshold, and voice prompts. It also supports voice interaction features like voice cloning, timbre selection, and user feedback (like/dislike) with comments. The backend API endpoints have been updated to handle streaming messages, stopping messages, uploading audio files, and submitting feedback.

_web\_demos/minicpm-o\_2.6/web\server · high confidence

This change introduces a new \finetune\ module providing official scripts and configuration for fine-tuning MiniCPM-V 4.0, MiniCPM-o 2.6, MiniCPM-V 2.6, MiniCPM-Llama3-V 2.5, and MiniCPM-V 2.0. It includes \finetune\_ds.sh\ for full-parameter fine-tuning and \finetune\_lora.sh\ for LoRA-based tuning, both supporting multi-image inputs and DeepSpeed Zero-2/Zero-3 optimization. The module adds a custom \CPMTrainer\ to handle model-specific loss computation and a \SupervisedDataset\ class that manages tokenization, image preprocessing, and conversation formatting for supervised fine-tuning.

finetune · high confidence

The OmniLMM model no longer prints a log message indicating that vision tower weights are being skipped during initialization. This change suppresses console output, resulting in a cleaner execution log for users running the model.

omnilmm · high confidence

The project has transitioned its primary focus from the OmniLMM series to the MiniCPM-V and MiniCPM-o model families. The documentation (README.md and README\_zh.md) has been completely rewritten to highlight MiniCPM-V 4.6 and MiniCPM-o 4.5, replacing previous references to OmniLMM-3B and OmniLMM-12B. Correspondingly, the Python inference script (chat.py) has been refactored: the main controller class is renamed from OmniLMMChat to MiniCPMVChat, and the underlying model classes have been updated to support MiniCPM-V, MiniCPM-Llama3-V 2.5, and MiniCPM-V 2.6, including new logic for multi-GPU inference in version 2.6.

(repo-wide) · high confidence

This change updates core Python dependencies for the main project, upgrading PyTorch to 2.1.2, Transformers to 4.40.0, and Gradio to 4.41.0, while removing the base\_utils dependency. It also introduces three new requirements files: eval\_mm/vlmevalkit/requirements.txt and eval\_mm/vqaeval/requirements.txt for evaluation toolkits, and requirements\_o2.6.txt for the MiniCPM-o 2.6 demo, which pins specific versions for libraries like accelerate, sentencepiece, and gradio to ensure compatibility with the new model version.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 30.

• Code Health 34
• Architecture 99
• Maturity 57
• Readiness 14
• Security 72
• Accessibility 35

• 300 commits — 286 feature/other, 14 fixes

• (root) — 210 commits
• (repo) — 26 commits
• docs/wechat.md — 11 commits
• docs/api.md — 4 commits
• finetune/dataset.py — 4 commits
• finetune/requirements.txt — 3 commits
• web_demos/minicpm-o_2.6 — 3 commits
• assets/MiniCPM-V.jpg — 2 commits
• assets/minicpm-v25.png — 2 commits
• assets/minicpmo4_5 — 2 commits
• assets/minicpmv4.6 — 2 commits
• assets/minicpmv4_5 — 2 commits
• assets/wechat-QR.jpeg — 2 commits
• docs/MiniCPM_V_4_5_Technical_Report.pdf — 2 commits
• docs/llamafactory_train_and_infer.md — 2 commits
• eval_mm/vlmevalkit — 2 commits
• finetune/trainer.py — 2 commits
• .vscode/settings.json — 1 commit
• assets/MiniCPM-V27.jpg — 1 commit
• assets/Minicpm-v 37.jpg — 1 commit

• fix: Fix Simplex mode links in README
• fix: Fix links in README for Simplex modes
• fix: Fix links to LLaVA-UHD v4 in README.md
• fix: Fix speech link
• fix: Fix video links in README with permanent user-attachments URLs
• fix: Merge pull request #1128 from latent-9/docs-fix-llamafactory-link
• fix: Merge pull request #1135 from shoemoney/fix/readme-broken-anchors
• fix: Merge pull request #1139 from Anai-Guo/fix-vad-utils-missing-import-warnings
• fix: fix 2.6 slice placeholder
• fix: fix local web demo no permission for camera/mic
• fix: fix png in readme
• fix: fix(minicpm-o web_demo): add missing import warnings in vad_utils
• fix: fixed missing import
• fix: tiny fix
• change: Add Cookbook
• change: Add Cookbook
• change: Add Cookbook
• change: Add FullDuplexBench v1.0 benchmarking results
• change: Add MiniCPM-o 4.5 Technical Report
• change: Add MiniCPM-o 4.5 technical report links
• …and 280 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

OpenBMB/MiniCPM-V was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.