Skip to content
CAI
Software that uses CAICheck a score

OpenBMB/ChatDev

39.6

Weak · 18 September 2026

24.4k

lines of production code

Python

with JavaScript

1

measurement over time

CAI band scale
CAI lens gauges

What this system is

This system is a workflow orchestration platform that enables users to design, execute, and monitor automated processes through a web-based interface. It supports complex agent behaviors via configurable skills, memory integrations, and loop controls, while allowing batch processing of tasks through CSV or Excel uploads. The platform provides a robust backend for managing workflow definitions, local tools, and session states, complemented by a frontend that features internationalization and real-time execution monitoring.

Features

Adds rich tooltips, i18n support, and auto-hiding navigation

Users can now access detailed help information via rich tooltips on workflow nodes, edges, and form fields, which can be toggled on or off in the Settings modal. The application now supports English and Simplified Chinese, with a language selector added to the Settings modal. Additionally, the sidebar navigation now auto-hides when scrolling down and reappears on scroll up or hover, improving screen real estate on content-heavy pages.

frontend/src/components · high confidence

Agent Skills, Mem0 Memory, and Loop Timer Node

The runtime now supports filesystem-backed Agent Skills, allowing agents to discover, activate, and use external skill definitions (SKILL.md) with specific tool constraints. A new Mem0 memory store is available for cloud-managed semantic search, with independent user/agent scoping and exclusion of assistant output to reduce noise. Additionally, a loop\_timer node has been added to control workflow execution based on time limits, supporting both standard suppression and terminal gate modes. Underlying fixes ensure embedding dimension consistency across OpenAI and local models to prevent mixed-dimension errors in memory retrieval.

runtime · high confidence

Batch Run View and comprehensive UI internationalization

A new Batch Run View page has been added, providing a dashboard and terminal toggle to monitor batch workflow execution, select input files, and manage settings. Additionally, the entire frontend interface (Home, Launch, Tutorial, Workflow, and Workbench views) has been internationalized using vue-i18n, replacing hardcoded English strings with translation keys to support multiple languages.

frontend/src/pages · high confidence

Docker support and improved development workflow

The application now supports running via Docker Compose, simplifying dependency management and providing a consistent environment for local development. A new Makefile has been introduced to streamline common tasks, allowing users to start both backend and frontend servers with a single command or run quality checks. Additionally, the file watching behavior for the development server has been refined to only monitor source code directories, preventing unnecessary restarts triggered by generated agent files in the WareHouse directory.

(repo-wide) · high confidence

Help tooltips enabled by default and batch workflow execution added

Help tooltips are now enabled by default for workflow nodes and edges, with help content structured for internationalization support. The workflow list UI now displays descriptions directly from the API response, showing 'No description' when none is provided. A new batch workflow execution endpoint allows users to run multiple workflows in parallel with configurable limits. The API client has been simplified to use relative paths and updated endpoints for workflow operations.

frontend/src/utils · high confidence

New batch execution, synchronous workflow, and local tool management APIs with workflow metadata endpoints

The server now exposes new endpoints for batch processing (POST /api/workflows/batch) and synchronous workflow execution with streaming logs (POST /api/workflow/run), alongside endpoints to list and create local tools (GET/POST /api/tools/local). Workflow management routes have been updated to include metadata retrieval (GET /api/workflows/{filename}/args and /desc) and the existing CRUD operations now use explicit action suffixes (e.g., /update, /delete, /get) for clarity. Additionally, session zip file cleanup has been moved from atexit handlers to FastAPI background tasks to ensure reliable cleanup, and the WebSocket endpoint now accepts an optional session\_id parameter to support session preservation.

server/routes · high confidence

New batch run page and scroll behavior for tutorial view

The application now includes a new 'Batch Run' page accessible at /batch-run, allowing users to execute batch operations. Additionally, the tutorial view has been updated to automatically scroll to the top when navigated to, improving the user experience by ensuring the content starts at the beginning of the page.

frontend/src/router · high confidence

New development tools for YAML validation and VueGraph synchronization

Added two new Python scripts to the tools directory to streamline workflow configuration management. The \validate\_all\_yamls.py\ script performs strict syntax and schema validation on all YAML files in the \yaml\_instance/\ directory, providing detailed error reporting and exit codes suitable for CI/CD pipelines. The \sync\_vuegraphs.py\ script reads these YAML configurations and uploads them to the VueGraph database via a local API endpoint, ensuring the frontend visualization system reflects the latest workflow definitions.

tools · high confidence

New loop timer, agent skills, and memory configurations with enhanced MCP tooling controls

This update introduces several new configuration capabilities for agents and nodes. Users can now define time-based loop control using the new LoopTimer node, which supports configurable maximum durations (seconds, minutes, or hours), reset behavior, and passthrough modes. Agent configurations now support a new 'skills' section, allowing users to enable filesystem-backed agent skills with an allowlist of discovered skills from the \.agents/skills\ directory. Memory configurations have been extended with a new \Mem0MemoryConfig\ for integrating with the Mem0 managed memory service, requiring an API key and optional scoping IDs. Additionally, MCP tooling configurations (both local and remote) now support \cache\_ttl\ to control how long tool lists are cached and \tool\_sources\ to filter which tools are exposed based on their source metadata. Finally, the base Node configuration now includes a \log\_output\ boolean field, allowing users to disable logging of specific node outputs.

entity · high confidence

Behavioural changes

Batch execution support and improved session resilience

Users can now upload CSV or Excel files to execute multiple workflow tasks in parallel, with progress and results streamed back via WebSocket and saved as CSV/JSON. Session handling is more robust: WebSocket disconnections no longer cancel running workflows, and reconnecting to an existing session replays buffered messages and restores the session state snapshot. Additionally, the server now sanitizes upload filenames to prevent path traversal attacks and fixes a cross-thread crash in WebSocket message sending by using run\_coroutine\_threadsafe instead of spinning up new event loops.

server/services · high confidence

Code executor uses resolved absolute paths for workspace and scripts

The code execution function now resolves the temporary workspace directory and the generated script file to absolute paths before execution. This ensures that the subprocess running the code operates with consistent, fully qualified paths, which can prevent issues related to relative path resolution or working directory changes during script execution.

functions · high confidence

Configurable CORS middleware with environment variable support

The application now uses a dedicated CORS middleware that can be configured via the CORS\_ALLOW\_ORIGINS environment variable. If this variable is set, only the specified origins are allowed; otherwise, it defaults to localhost and 127.0.0.1 on port 5173, while also allowing any port for these hosts via regex matching during development. This change replaces the previous generic middleware addition with a more explicit and configurable CORS setup.

utils · high confidence

Configurable output directory naming and logging behavior

The workflow now allows users to force a fixed output directory name by setting the 'fixed\_output\_dir' metadata flag, preventing the automatic timestamp suffix that is otherwise added to session names. Additionally, node-end logging is now conditional: if a node is configured not to log output, an empty string is recorded instead of the full output text, reducing log noise for nodes that do not require output recording.

workflow · high confidence

Fix missing newline at end of check.py

The check.py file now ends with a newline character, resolving a minor formatting issue where the file previously lacked a trailing newline.

check · high confidence

Test coverage

Added regression tests for file upload sanitization, memory scoping, and server reload behavior

Added test coverage for four distinct areas: (1) attachment upload filename sanitization to prevent path traversal attacks, verifying that directory components are stripped and traversal attempts cannot access or delete files outside the temporary directory; (2) Mem0 memory integration, testing user/agent scoping logic, filter construction (including OR filters when both IDs are present), fallback behavior, and API error handling; (3) memory embedding dimension consistency, ensuring that mixed-dimensional embeddings do not crash retrieval and that OpenAI embedding fallback dimensions are correctly cached and used; (4) server reload configuration, verifying that the reload watcher excludes output directories (WareHouse, logs, data, node\_modules) to prevent unintended restarts, and that appropriate warnings are logged when watchfiles is unavailable.

tests · high confidence

Dependencies

Updated frontend dependencies and added new backend libraries

The frontend now includes vue-i18n for internationalization support and cross-env in development, alongside updates to Babel and esbuild tooling. On the backend, the project adds watchfiles for file watching, openpyxl for Excel handling, and mem0ai for memory integration, while also aligning requirements.txt with pyproject.toml and adding pytest configuration.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Baseline

  • First survey — no prior run to compare against. CAI 40.

Lenses

  • Code Health 37
  • Architecture 100
  • Maturity 74
  • Readiness 27
  • Security 74
  • Accessibility 47

Changes since last survey

  • 205 commits — 145 feature/other, 60 fixes

By area

  • (repo) — 70 commits
  • (root) — 51 commits
  • frontend/src — 26 commits
  • runtime/node — 9 commits
  • .github/workflows — 8 commits
  • entity/configs — 4 commits
  • server/routes — 4 commits
  • server/services — 4 commits
  • docs/user_guide — 3 commits
  • yaml_instance/demo_loop_timer.yaml — 3 commits
  • check/check.py — 2 commits
  • frontend/public — 2 commits
  • openspec/changes — 2 commits
  • yaml_instance/ChatDev_v1.yaml — 2 commits
  • yaml_instance/GameDev_v1.yaml — 2 commits
  • yaml_instance/subgraphs — 2 commits
  • .agents/skills — 1 commit
  • frontend/package-lock.json — 1 commit
  • functions/function_calling — 1 commit
  • server/bootstrap.py — 1 commit

Notable commits

  • fix: Fix nested UI scroll behavior
  • fix: Merge pull request #507 from zxrys/feature/openai-provider-json-fix
  • fix: Merge pull request #515 from MatrixA/fix/readme-reload-hint
  • fix: Merge pull request #530 from rainoeelmae/fix/windows-cross-platform-support
  • fix: Merge pull request #552 from zxrys/feature/fix-demo_human-and-gemini-problem
  • fix: Merge pull request #553 from zxrys/feature/fix-frontend-lag-issue
  • fix: Merge pull request #557 from Br1an67/fix/issue-538-safari-tooltip
  • fix: Merge pull request #583 from LaansDole/fix/toggle-chat-fullscreen-scrollable
  • fix: Merge pull request #600 from voidborne-d/fix/send-message-sync-cross-loop
  • fix: Merge pull request #611 from voidborne-d/fix/reload-exclude-warehouse
  • fix: Merge pull request #617 from benjamin7007/fix/strip-thinking-tokens
  • fix: Merge pull request #618 from zxrys/fix/ui-scrolling-issue-fix
  • fix: Merge pull request #624 from hobostay/fix/temp-file-leak-session-download
  • fix: Merge pull request #641 from swaylq/fix/638-upload-filename-path-traversal
  • fix: Revert "chores: move model name to centralized env variable"
  • fix: [fix] support chat interface
  • fix: [fix] support chat interface
  • fix: [fix] support chat interface
  • fix: [fix] support chat interface
  • fix: add: new contributors upgrade: adjusting contributor ranking fix: remove --reload in Makefile
  • …and 185 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

OpenBMB/ChatDev was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 18 September 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 4fb2db0ea90375ce1059f44fe03ffbd191a7a169 — the exact code this score is about.
  • Scored under rubric-2026.09.15 — the same rubric and the same method as every other entry in this index.
  • Measured by watchdog.canine.dev using codehealth-analyzer preprod-5d04157a340d.