xinnan-tech/xiaozhi-esp32-server

This system is a comprehensive management platform for AI-powered voice agents and connected hardware devices. It provides a Java-based backend API and web/mobile consoles for administering agents, configuring AI models (LLM, TTS, ASR), managing device fleets via OTA updates, and handling user authentication. The system also includes a Python server that handles real-time voice interactions, intent recognition, and tool execution for the connected devices.

108 changes.

This period established the core infrastructure for the Xiaozhi project, introducing a Java-based manager API for system administration, user authentication, and device management alongside a Python server with modular AI provider support. Key developments included the implementation of OAuth2 security, comprehensive device lifecycle handling with OTA updates, and the integration of diverse LLM, ASR, and TTS services. The work also delivered the initial web and mobile admin consoles, Docker deployment scripts, and a unified plugin and tool execution framework.

19 changes.

This period focused on expanding core capabilities by introducing new memory providers, a digital-human testing module, and agent synonym management APIs. Significant effort was also dedicated to establishing comprehensive test coverage across both the backend and frontend, including initial pytest suites for the server and extensive unit and contract tests for the manager applications.

A new memory provider named 'mem\_report\_only' has been added to the system. When this provider is selected, the server will no longer save or summarize conversation history; it simply logs that no memory operations are performed and returns empty results for queries. This allows users to disable memory summarization while retaining the memory interface structure.

_main/xiaozhi-server/core/providers/memory/mem\_report\only · high confidence

Added a new LLM provider implementation for Alibaba Cloud's Bailian (AliBL) service. This provider supports streaming responses, configurable memory integration, and custom base URLs. It includes a fallback mechanism for function calls, as the underlying API does not natively support them, ensuring compatibility by returning plain text tokens instead.

main/xiaozhi-server/core/providers/llm/AliBL · high confidence

The Coze LLM provider implementation has been added to the system, enabling users to connect to the Coze platform using a personal access token and bot configuration. This new provider supports standard text streaming responses as well as function calling (tool use), where tool definitions are injected into the system prompt and tool results are appended to the conversation history. It also includes validation for the API key upon initialization.

main/xiaozhi-server/core/providers/llm/coze, main/xiaozhi-server/core/providers/llm/xinference · high confidence

Introduces a new LLM provider for Dify that supports chat, workflow, and completion message modes. The implementation handles streaming responses, manages conversation IDs for session continuity, filters redundant 'message\_replace' events, and includes specific logic to support function calling by injecting system prompts and processing tool results.

main/xiaozhi-server/core/providers/llm/dify · high confidence

A new FastGPT LLM provider has been added to the system, enabling users to connect to FastGPT services via a configurable API key and base URL. The provider supports streaming responses, filtering out specific thought markers (\\\\\\) from the output, and validates the API key upon initialization. However, function calling (tool use) is not yet implemented; attempting to use functions will result in an error message advising users to employ alternative intent recognition methods.

main/xiaozhi-server/core/providers/llm/fastgpt · high confidence

A new Gemini LLM provider has been added to the system, enabling users to interact with Google's Gemini models (defaulting to gemini-2.0-flash). This implementation supports streaming responses for real-time dialogue and includes function calling capabilities, allowing the model to execute external tools. It also features robust proxy configuration, automatically testing and setting up HTTP/HTTPS proxies to ensure connectivity, and handles conversation history by mapping roles appropriately for the Gemini API.

main/xiaozhi-server/core/providers/llm/gemini · high confidence

Users can now integrate Home Assistant as an LLM provider. The new \homeassistant.py\ module forwards user voice input to the Home Assistant conversation API, using the session ID as the conversation ID and returning the plain speech response. Function calling is explicitly unsupported and will log an error, directing users to use other intent recognition methods.

main/xiaozhi-server/core/providers/llm/homeassistant · high confidence

Introduces a new MCP (Model Context Protocol) endpoint provider that allows the server to connect to external MCP services via WebSocket. This adds client-side logic to initialize connections, fetch available tools, and execute remote tool calls, exposing these remote capabilities as local tools for the system to use.

_main/xiaozhi-server/core/providers/tools/mcp\endpoint · high confidence

Introduces a new PowerMem memory provider that supports both standard asynchronous memory storage and a user profiling mode (UserMemory) requiring OceanBase. The provider allows configuration of database backends (SQLite, OceanBase, PostgreSQL), LLM providers, and embedding models, and includes logic to parse JSON-formatted message content for accurate memory retrieval.

main/xiaozhi-server/core/providers/memory/powermem · high confidence

A new TimbreDetailsVO class has been introduced to structure voice profile details returned by the API. This data transfer object exposes fields for voice identification, language, name, remarks, sorting order, and specific audio assets including reference audio paths, reference text, and demo playback URLs. It also includes identifiers linking the voice to a specific TTS model and voice encoding, enabling the frontend to display comprehensive voice information and select voices for local TTS synthesis.

main/manager-api/src/main/java/xiaozhi/modules/timbre/vo · high confidence

Introduces a new VLLM provider implementation that communicates with vLLM inference servers via the OpenAI API. The provider supports configuration of model name, API key, base URL, and generation parameters (max\_tokens, temperature, top\_p). It handles both text and image inputs (via base64 encoding) and appends a Chinese language instruction to user queries to ensure responses are in Chinese.

main/xiaozhi-server/core/providers/vllm · high confidence

The manager API now exposes a new REST endpoint group at /correct-word to manage synonym replacement files for AI agents. Users can create, update, list, download, and delete synonym files (stored in the ai\_agent\_correct\_word\_file and ai\_agent\_correct\_word\_item tables), as well as batch-delete files. The API also supports associating synonym files with specific agents via a mapping table, allowing agents to retrieve their linked synonym lists.

main/manager-api/src/main/java/xiaozhi/modules/correctword · high confidence

The manager-api module now includes data entity classes for core system administration features. Users can manage system parameters with typed values (string, number, boolean, array) via SysParamsEntity, organize data dictionaries with types and items via SysDictTypeEntity and SysDictDataEntity, and manage system user accounts with status and admin privileges via SysUserEntity. These entities map to the underlying database tables and support automatic tracking of update timestamps and users.

main/manager-api/src/main/java/xiaozhi/modules/sys/entity · high confidence

Introduces the TimbreEntity class to represent voice profiles in the AI TTS system, mapping to the ai\_tts\_voice database table. This entity includes fields for voice identification, language support, naming, and metadata, along with specific attributes for reference audio paths and text used for local TTS voice selection, as well as links to TTS models and demo audio URLs.

main/manager-api/src/main/java/xiaozhi/modules/timbre/entity · high confidence

The manager-api now exposes a new REST controller at /ttsVoice for managing voice timbres. Users with superAdmin permissions can perform paginated searches, create, update, and delete timbre records via the new TimbreController, backed by the TimbreDao persistence layer.

main/manager-api/src/main/java/xiaozhi/modules/timbre/controller · high confidence

Added a new static asset generator for customizing voice box themes, allowing users to configure wake words, fonts, emojis, and chat backgrounds to export an assets.bin file. This includes a new README documenting the tool's features (chip/screen config, wake word, font, emoji, and background settings) and the associated build artifacts (JS/CSS) and static resources (fonts, emoji sets, wake models).

main/manager-web/public/generator · high confidence

The system now supports sending verification code SMS messages via Alibaba Cloud. This change introduces a new SmsService interface and an ALiYunSmsService implementation that retrieves configuration parameters (such as access keys, sign names, and template codes) from the system settings and uses the Alibaba Cloud SDK to deliver messages. The implementation includes error handling that rolls back the daily SMS send count in Redis if a transmission fails, ensuring accurate usage tracking.

main/manager-api/src/main/java/xiaozhi/modules/sms · high confidence

The web manager now supports three additional languages: German (de), Vietnamese (vi), and Brazilian Portuguese (pt\_BR). This change introduces the corresponding translation files and updates the i18n configuration to include these locales alongside the existing English, Simplified Chinese, and Traditional Chinese options, allowing users to switch their interface language to these new variants.

main/manager-web/src/i18n · high confidence

The manager API now includes data models to support device management and Over-The-Air (OTA) updates. New entity classes have been added for device information (DeviceEntity), firmware details (OtaEntity), and a device address book (DeviceAddressBookEntity), enabling the backend to store and manage device records, firmware versions, and inter-device communication contacts.

main/manager-api/src/main/java/xiaozhi/modules/device/entity · high confidence

The admin console now includes static HTML pages for the User Agreement and Privacy Policy in both Chinese and English, ensuring users can review terms of service and data handling practices. Additionally, an offline.html page has been added to provide a user-friendly experience when the network connection is lost, allowing users to retry the connection or view cached content.

main/manager-web/public · high confidence

The models directory now includes the configuration, demo scripts, and source code for the SenseVoiceSmall speech recognition model and the snakers4 Silero Voice Activity Detector (VAD). This change bundles the necessary model definitions and utility libraries locally, enabling the server to perform speech recognition and voice activity detection without relying on external runtime downloads for these specific components.

_main/xiaozhi-server/models, main/xiaozhi-server/models/snakers4\silero-vad · high confidence

A new Shiro filter (ServerSecretFilter) and token class (ServerSecretToken) have been introduced to enforce server-level secret authentication. This filter validates incoming requests by checking for a Bearer token in the Authorization header against a configured server secret, returning a 401 Unauthorized response if the token is missing or invalid, while allowing OPTIONS requests to pass through.

main/manager-api/src/main/java/xiaozhi/modules/security/secret · high confidence

The agent service layer now includes comprehensive implementations for chat history management, agent snapshots, and MCP tool access. Chat history is handled via new services that support session grouping, retrieval of the latest 50 user messages (excluding those without audio), and batch deletion of audio and text records. Agent snapshots provide versioned configuration backups with restore capabilities, including a background redaction process to clean up legacy sensitive data. Additionally, the system now supports fetching agent-specific MCP access points and tool lists via a dedicated service that manages WebSocket connections and JSON-RPC communication.

main/manager-api/src/main/java/xiaozhi/modules/agent/service/impl · high confidence

The application now uses a centralized Vuex store to manage authentication state and public configuration. User login and logout actions are handled through this store, which persists the authentication token and user information to local storage to maintain session state across page refreshes. Additionally, the store retrieves and caches public configuration settings, including ICP and GA filing numbers and the SM2 public key, making these values available throughout the application.

main/manager-web/src/store · high confidence

The manager-api now exposes a dedicated ConfigService that consolidates server, agent, and device configuration retrieval. Users benefit from a unified interface that fetches server settings with optional caching, retrieves agent-specific model configurations (including voice, language, and memory settings) based on device MAC address, and provides agent replacement words. This service also handles logic for managing device output limits and chat history policies, streamlining how configuration data is served to clients and the management console.

main/manager-api/src/main/java/xiaozhi/modules/config/service · high confidence

The manager-web console now supports offloading frontend dependencies to a CDN (hosted on unpkg and cdnjs) to reduce bundle size and improve load times. This behavior is controlled by the VUE\_APP\_USE\_CDN environment variable; when enabled, libraries such as Vue, Vuex, Vue Router, Element UI, Axios, and Opus Decoder are loaded externally via webpack externals. Additionally, a Service Worker is generated in production builds to cache application assets, with the CDN mode status passed to the worker for consistent resource handling.

main/manager-web · high confidence

The manager-api now supports configurable chat history reporting for agents. Administrators can choose to record only text or both text and audio via the agent's chat history configuration. When audio recording is enabled, incoming chat reports include Base64-encoded audio data that is decoded and persisted, linked to the corresponding text record. The system also updates the device's last conversation and connection timestamps upon successful report ingestion.

main/manager-api/src/main/java/xiaozhi/modules/agent/service/biz · high confidence

The speech-to-text system now supports a significantly wider range of providers, including new streaming implementations for Alibaba Cloud (aliyun\_stream) and Alibaba Bailian (aliyunbl\_stream), as well as streaming support for Doubao (doubao\_stream). Additional providers have been added for Baidu (baidu), local FunASR execution (fun\_local), and remote FunASR servers (fun\_server). The update also introduces support for Qwen3-ASR-Flash (qwen3\_asr\_flash), OpenAI-compatible APIs (openai), and local Sherpa-ONNX models (sherpa\_onnx\_local). A new DTO module (dto/dto.py) defines interface types (STREAM, NON\_STREAM, LOCAL) to standardize provider integration.

main/xiaozhi-server/core/providers/asr · high confidence

The manager API now includes a complete set of Liquibase migration scripts that establish the core database schema and populate it with default configuration data. New tables have been added to manage system users, parameters, and dictionaries, as well as the AI-specific entities: model providers, model configurations, TTS voices, agent templates, agents, devices, voiceprints, and chat history. The initial run also seeds the database with default model provider definitions, pre-configured model settings for various ASR, LLM, TTS, memory, and intent services, several default agent templates, and system parameters for server settings, logging, and plugins.

main/manager-api/src/main/resources/db · high confidence

Added a new memory provider implementation for Mem0AI, enabling the system to save and query user memories via the Mem0 API. This change introduces the \mem0ai.py\ module which handles connection initialization, asynchronous saving of conversation messages (with JSON content extraction), and querying memories with user-specific filters and timestamp formatting.

main/xiaozhi-server/core/providers/memory/mem0ai · high confidence

Introduces the \manager-mobile\ project, a cross-platform mobile management interface built on uni-app v3, Vue 3, and Vite. This application supports deployment to H5, iOS, Android, and WeChat Mini Programs, providing users with mobile access to agent management, device configuration, chat history, and voice print features. The release includes the complete source code, environment configuration templates, and build scripts required to run and distribute the mobile app.

main/manager-mobile · high confidence

The application now initializes its routing via a new \src/router/index.js\ file that defines the core navigation structure, including public pages (login, register, retrieve password) and a comprehensive set of management views (user, device, model, OTA, voice, and feature management). Additionally, the router implementation includes a global override for navigation duplication: clicking a navigation link that is already active will now trigger a full page reload instead of silently failing or throwing an error, ensuring the view state refreshes correctly.

main/manager-web/src/router · high confidence

The xiaozhi-server is now available as a standalone component, introducing a new \agent-base-prompt.txt\ file that defines the AI's conversational behavior, tone, and tool usage rules, which can be customized via the \prompt\_template\ configuration. The server application (\app.py\) now features a robust exit mechanism that properly handles Ctrl-C and SIGTERM signals on both Windows and Unix systems to prevent process blocking, alongside a global garbage collection manager to optimize resource usage during operation.

main/xiaozhi-server · high confidence

Added a new server-side MCP (Model Context Protocol) tool provider located in \core/providers/tools/server\_mcp\. This feature introduces a \ServerMCPManager\ that loads MCP server configurations from \data/.mcp\_server\_settings.json\ and initializes them concurrently with a 10-second timeout and lock protection to prevent race conditions. The underlying \ServerMCPClient\ now supports multiple transport protocols, including Stdio, SSE, and the newly added Streamable HTTP. A \ServerMCPExecutor\ integrates these tools into the system's function list, handling tool discovery, execution with retry logic, and automatic reconnection on failure.

_main/xiaozhi-server/core/providers/tools/server\mcp · high confidence

Added a new provider module for device-side Model Context Protocol (MCP) tools, including an MCP client for managing tool state and caching, a handler for sending and processing MCP messages (initialization, tool list requests, and call responses), and an executor that bridges MCP tools to the server's action system. This enables the server to discover, list, and execute tools provided by connected devices via the MCP protocol.

_main/xiaozhi-server/core/providers/tools/device\mcp · high confidence

Added a new local short-term memory provider that maintains a dynamic, structured memory profile for each user role. The provider uses an LLM to summarize conversation history into a JSON-formatted memory state (including identity, events, and relationships) and persists it to a local YAML file. It supports loading existing memory, handling JSON-parsed message content, and includes logic to fall back to a remote API for summary generation when file-based saving is disabled.

_main/xiaozhi-server/core/providers/memory/mem\_local\short · high confidence

Added a new \ServerPluginExecutor\ module that enables the system to dynamically load, configure, and execute server-side plugin tools. This component handles the registration of available tools, expands module-level plugin names into specific functions, retrieves localized descriptions, and manages the execution of both synchronous and asynchronous tool functions with appropriate connection context.

_main/xiaozhi-server/core/providers/tools/server\plugins · high confidence

Added TimbreDataDTO and TimbrePageDTO to the manager-api module to support the new timbre management feature. TimbreDataDTO defines the structure for timbre information, including language, name, reference audio path, reference text, associated TTS model ID, voice code, and demo audio URL, with validation constraints on required fields. TimbrePageDTO provides parameters for paginated queries of timbre records, allowing filtering by TTS model ID and name.

main/manager-api/src/main/java/xiaozhi/modules/timbre/dto · high confidence

The manager-api now exposes a new TimbreService that handles the lifecycle of voice timbres (TTS voices). This service implements paginated listing, retrieval, creation, and deletion of timbre records, with read operations cached in Redis to improve performance. It also integrates user voice clones by automatically appending successful clone entries to the standard timbre list and provides utility methods to resolve default languages and timbre names by ID.

main/manager-api/src/main/java/xiaozhi/modules/timbre/service · high confidence

A new utility module \alibl\endpoint.py\ has been added to validate and resolve WebSocket URLs for Alibaba Cloud Bailian (DashScope) inference endpoints. It enforces that the URL uses the WSS scheme, targets official DashScope hostnames (\dashscope.aliyuncs.com\ or \dashscope-intl.aliyuncs.com\) or Alibaba Cloud MAAS workspaces (\\.maas.aliyuncs.com\), and uses the standard \/api-ws/v1/inference\ path on port 443, ensuring that only valid, secure endpoints are accepted for LLM connections.

main/xiaozhi-server/core/utils · high confidence

The server now supports optional HMAC-SHA256 token authentication for WebSocket connections. A new \AuthManager\ generates and verifies time-limited tokens signed with a secret key, ensuring that only authorized devices can connect. This authentication is integrated into the \WebSocketServer\ handshake process, where valid \Device-ID\, \Client-ID\, and \Authorization\ headers are required if the feature is enabled in the configuration.

main/xiaozhi-server/core · high confidence

The LLM provider layer now includes a new abstract base class (LLMProviderBase) that standardizes response handling and provides a default implementation for streaming function calls, allowing providers to override behavior when tool use is supported. Additionally, a dedicated system prompt module has been added to enforce strict JSON-based formatting for tool invocations, ensuring consistent and parseable function call outputs from the LLM.

main/xiaozhi-server/core/providers/llm · high confidence

The manager-api now includes an XSS filter to protect against cross-site scripting attacks and implements a new RAGFlow adapter for knowledge base operations. This adapter introduces a dedicated HTTP client and adapter layer to manage communication with the RAGFlow service, handling document listing, retrieval, and configuration validation, thereby isolating business logic from external API calls.

xiaozhi-esp32-api · high confidence

Users can now deploy the server and web management modules using a new multi-stage Docker build process (Dockerfile-server, Dockerfile-server-base, Dockerfile-web) and a comprehensive one-click installation script (docker-setup.sh). The server base image is optimized with pip fallbacks to handle metadata issues, while the web image consolidates the Vue frontend and Java backend into a single Nginx-serving container. A new Makefile provides a unified entry point for running Python, Java, and Web unit tests, and the .gitignore has been updated to exclude new build artifacts and local runtime caches.

(repo-wide) · high confidence

A new intent recognition system has been added to the server, introducing a base class \IntentProviderBase\ and a default \NoIntentProvider\ implementation. This framework allows the system to detect user intent from dialogue history and text, with the default behavior set to always return a 'continue chat' function call, establishing the foundation for future intent-based interactions.

main/xiaozhi-server/core/providers/intent · high confidence

A new Spring Boot application entry point, AdminApplication, has been added to the xiaozhi package. This class serves as the startup point for the manager-api service, initializing the application context and printing the Swagger documentation URL (http://localhost:8002/xiaozhi/doc.html) to the console upon successful launch.

main/manager-api/src/main/java/xiaozhi · high confidence

The system now includes dedicated entity classes for managing AI model infrastructure: \ModelConfigEntity\ represents individual model configurations (such as LLM, TTS, or ASR settings) including JSON configuration data, while \ModelProviderEntity\ defines the underlying service providers (like OpenAI) with their specific field requirements. These entities form the backend data structure for the new model management capabilities, enabling users to define and store detailed model settings and provider metadata.

main/manager-api/src/main/java/xiaozhi/modules/model/entity · high confidence

The manager-api module adds foundational components for user authentication and system configuration. This includes a new security package featuring a BCrypt-based password encoder and a \SysUserTokenDao\ for managing user session tokens, alongside a \SysParamsRedis\ component to handle caching of system parameters in Redis.

(repo-wide) · high confidence

This change introduces a set of new service interfaces within the system module to support core administrative and authentication functionalities. Specifically, it adds \SysUserService\ to handle user lifecycle operations including pagination, password management (reset, direct change, and validation), status updates, and registration settings. It also introduces \SysDictTypeService\ and \SysDictDataService\ to enable CRUD operations and retrieval for system data dictionaries. Additionally, \SysParamsService\ is added to manage system configuration parameters, including caching and menu configuration updates. To resolve circular dependencies between user and device modules, \SysUserUtilService\ is introduced to provide a utility for assigning usernames. Finally, \TokenService\ is added to handle token generation for authentication.

main/manager-api/src/main/java/xiaozhi/modules/sys/service · high confidence

The knowledge base module has been restructured to introduce a \KnowledgeBaseAdapter\ pattern, which centralizes all RAGFlow API interactions and isolates business logic from external HTTP calls. This change adds new REST endpoints for managing knowledge bases and documents (including creation, updates, deletion, and file uploads) and introduces a local shadow database (\ai\_rag\_knowledge\_document\) to track remote document states. The update also includes new configuration classes for the adapter and scheduled tasks, along with comprehensive DTOs for datasets, documents, chunks, and retrieval testing.

main/manager-api/src/main/java/xiaozhi/modules/knowledge · high confidence

The manager web application now includes a dedicated API module layer under \src/apis/module\ that exposes frontend interfaces for core management capabilities. This includes endpoints for agent lifecycle and configuration snapshots (\agent.js\), device binding and status (\device.js\), user administration and parameter management (\admin.js\), model configuration and provider lists (\model.js\), OTA firmware management (\ota.js\), voice/timbre settings (\timbre.js\), knowledge base operations (\knowledgeBase.js\), dictionary data (\dict.js\), replacement words (\correctWord.js\), and device address books (\addressBook.js\). A regression test suite (\agentSnapshotApi.test.mjs\) has been added to verify the snapshot restore flow, ensuring that restore previews use the correct state token, that destructive restores require explicit confirmation, and that in-flight requests are properly guarded.

main/manager-web/src/apis/module · high confidence

Added three new API handler modules to the server core: \base\_handler.py\ provides a common base with CORS support; \ota\_handler.py\ implements Over-The-Air update logic, including device authentication, firmware version comparison, and binary file serving from the local data directory; and \vision\_handler.py\ exposes an endpoint for image-based visual analysis via VLLM models, handling multipart form data uploads and returning JSON responses. These changes introduce new HTTP endpoints for device firmware management and visual processing capabilities.

main/xiaozhi-server/core/api · high confidence

The API now exposes dedicated view objects to structure response data for two distinct areas. For the admin user management interface, AdminPageUserVO defines the fields returned when paginating users, including user ID, mobile, status, registration date, and the count of associated devices. For the digital dictionary feature, three new models (SysDictTypeVO, SysDictDataVO, SysDictDataItem) define the structure for dictionary types, their data items, and individual key-value pairs, enabling the frontend to display and manage dictionary configurations.

main/manager-api/src/main/java/xiaozhi/modules/sys/vo · high confidence

The manager-api now includes dedicated data-access interfaces for three device-management capabilities. DeviceAddressBookDao adds methods to insert, list, update aliases and permissions, and batch-delete address-book records for devices, enabling device-to-device communication features. OtaDao provides the MyBatis mapper for OTA firmware management, supporting the new OTA upload/download and update-strategy flows. DeviceDao adds getAllLastConnectedAtByAgentId to retrieve the most recent connection time across all devices for a given agent, allowing the system to surface up-to-date connectivity status without fetching full device records.

main/manager-api/src/main/java/xiaozhi/modules/device/dao · high confidence

Added AgentModelsDTO and CorrectWordsDTO to the manager-api configuration module. AgentModelsDTO allows clients to submit their device MAC address, client ID, and a map of currently instantiated models, while CorrectWordsDTO accepts a device MAC address to retrieve synonym/word-correction settings. These DTOs enable the API to handle agent-specific model selection and vocabulary correction requests.

main/manager-api/src/main/java/xiaozhi/modules/config/dto · high confidence

The intent\_llm module introduces a new intent provider that uses an LLM to analyze user input and determine the appropriate action. It dynamically generates system prompts based on available functions (including MCP tools) and context (such as music file names and Home Assistant device lists). The provider handles specific intents like time/date queries, exit commands, and multi-command inputs, returning structured JSON for function calls or context-based responses. It also includes caching for intent detection results and uses async threading to prevent blocking the event loop during LLM calls.

_main/xiaozhi-server/core/providers/intent/intent\llm · high confidence

Added two Lua scripts to the manager-api module to support new Redis utility methods. The new getKeyOrCreate.lua script retrieves a Redis key's value and, if it is empty, sets it to a specified default value with an optional expiration time. The new emptyAll.lua script executes the FLUSHDB command to clear the current database.

main/manager-api/src/main/resources/lua · high confidence

A new OpenAI LLM provider implementation has been added to the system. It introduces automatic disabling of 'thinking' modes for specific domains (such as DeepSeek and Aliyun) to ensure consistent response formats. The provider also supports granular HTTP timeout configurations (connect, read, write, pool) while maintaining backward compatibility with simple timeout values. Additionally, it includes robust handling for streaming responses, including empty choices and tool calls, and normalizes dialogue inputs to prevent errors from missing content fields.

main/xiaozhi-server/core/providers/llm/openai · high confidence

Added three new utility classes in the sys/utils package: WebSocketClientManager, which provides a builder-based client with configurable buffer sizes (defaulting to 1MB) and message queues to prevent disconnections due to large data volumes; WebSocketValidator, which validates WebSocket URL formats and tests connectivity; and WebSocketTestHandler, a helper handler used by the validator to test connections. These changes address issues where small buffer sizes caused disconnections during high-data scenarios and provide tools for validating WebSocket endpoints.

main/manager-api/src/main/java/xiaozhi/modules/sys/utils · high confidence

The system now exposes dedicated admin controllers for managing core resources. Administrators can paginate and filter users and devices, reset user passwords, change user statuses, and delete users (which now cascades to their associated agents and devices). A new parameter management interface allows saving, updating, and deleting system parameters with specific validation for WebSocket, OTA, MCP, and voice-print URLs, as well as MQTT secret complexity. Additionally, a server-side management controller enables super-admins to retrieve WebSocket server lists and emit configuration update or restart actions to Python backends via WebSocket. Finally, a complete CRUD interface for dictionary types and data has been added, allowing administrators to manage system lookup values.

main/manager-api/src/main/java/xiaozhi/modules/sys/controller · high confidence

The manager-api now exposes a comprehensive set of data transfer objects to support new agent capabilities. Users can now manage agent snapshots (create, list, and restore) and update agent configurations including ASR, TTS, LLM, and memory models. Chat history is now configurable (text-only, audio, or disabled) and includes session summaries. The system also supports agent tagging, context providers, and voice print management (save, update, and identification).

main/manager-api/src/main/java/xiaozhi/modules/agent/dto · high confidence

The API now exposes several new value objects to support richer agent management features. Users can view individual user chat records via AgentChatHistoryUserVO, which includes message content and associated audio IDs. Agent configuration management is enhanced with AgentSnapshotVO to track versioned configuration snapshots, including details on restoration sources and state fingerprints. Voiceprint capabilities are introduced through AgentVoicePrintVO, allowing retrieval of voice profiles linked to specific audio sources. Additionally, AgentInfoVO extends the base agent entity to return plugin mappings, context provider configurations, replacement word file IDs, and the current configuration version number, while AgentTemplateVO adds TTS and LLM model names to template data.

main/manager-api/src/main/java/xiaozhi/modules/agent/vo · high confidence

The manager-api now exposes dedicated REST endpoints for managing agent lifecycle and data. Users can download chat history (current session or with previous 20 sessions) via temporary UUID links, view and restore agent snapshots, and manage agent templates (CRUD with pagination and batch removal). Voice print management (create, update, delete, list) is available for user authentication, and MCP access point addresses and tool lists can be retrieved per agent. Chat summary generation is now asynchronous to avoid blocking responses. All endpoints enforce agent-level permission checks.

main/manager-api/src/main/java/xiaozhi/modules/agent/controller · high confidence

The manager-api backend now exposes a comprehensive set of service interfaces for managing agent capabilities. Users can now interact with agent chat history, including retrieving recent user messages with audio, fetching specific audio content, and generating session summaries or titles. Voice print management is introduced, allowing owners to add, update, and delete voice profiles for their agents. Additionally, the system supports agent tagging, MCP access point configuration, plugin mapping, and configuration snapshotting (create, restore, delete). These services form the backend foundation for the new chat, voice, and configuration features.

main/manager-api/src/main/java/xiaozhi/modules/agent/service · high confidence

The manager-api now exposes a set of new and refined DTOs that define the input and output contracts for several admin and system features. Administrators can now use the new pagination endpoint for user management (AdminPageUserDTO) and manage system parameters with strict type validation (SysParamsDTO). Authentication flows have been extended with dedicated models for password changes (PasswordDTO) and password retrieval via SMS verification (RetrievePasswordDTO). Additionally, the API introduces models for managing dictionary data (SysDictTypeDTO, SysDictDataDTO) and for sending server-side actions via WebSocket (EmitSeverActionDTO, ServerActionPayloadDTO, ServerActionResponseDTO), enabling remote configuration and restart capabilities for the backend services.

main/manager-api/src/main/java/xiaozhi/modules/sys/dto · high confidence

A new ConfigController has been added to expose three POST endpoints under the /config path: /server-base retrieves general server configuration, /agent-models fetches available models for a specific agent based on MAC address and selected module, and /correct-words retrieves replacement/correction words associated with a device's MAC address. These endpoints allow clients to dynamically fetch configuration and customization data from the manager-api service.

main/manager-api/src/main/java/xiaozhi/modules/config/controller · high confidence

The API now exposes structured data transfer objects to support the new model management capabilities. Users can interact with model configurations via ModelConfigDTO and ModelConfigBodyDTO, which handle details like model type, provider code, and JSON configuration. Model providers are managed through ModelProviderDTO, allowing for CRUD operations on provider definitions. Additionally, VoiceDTO introduces support for voice information including demo audio URLs, language types, and cloning status, while LlmModelBasicInfoDTO provides a simplified view of LLM model metadata.

main/manager-api/src/main/java/xiaozhi/modules/model/dto · high confidence

Added DeviceOtaVO to structure OTA update responses (including activation, MQTT, server time, and firmware details) and UserShowDeviceListVO to define the fields returned in the user-facing device list, such as device type, alias, auto-update status, and connection timestamps.

main/manager-api/src/main/java/xiaozhi/modules/device/vo · high confidence

The device service layer now supports a broader set of device lifecycle and management operations. Users can manually add devices, view paginated device lists with user-specific filtering, and see the latest connection time for devices associated with an agent. Device binding and unbinding logic has been refined to require specific verification codes. Additionally, Over-The-Air (OTA) firmware management is introduced via a new service interface for handling firmware updates, and a new address book service enables device-to-device communication features such as calling by nickname and managing permissions.

main/manager-api/src/main/java/xiaozhi/modules/device/service · high confidence

The device management API now exposes endpoints for binding, registering, and unbinding devices, as well as manual device addition and address book management. A new OTA subsystem allows devices to check for and receive firmware updates via the OTAController, while the OTAMagController provides administrative interfaces for managing firmware versions, uploading files, and controlling download access.

main/manager-api/src/main/java/xiaozhi/modules/device/controller · high confidence

The manager-api module introduces a suite of new Data Transfer Objects to support expanded device capabilities. These include DeviceReportReqDTO and DeviceReportRespDTO for handling device firmware reporting and OTA activation responses (containing WebSocket and MQTT configuration details), DeviceAddressBookAliasDTO and DeviceAddressBookPermissionDTO for device-to-device address book and calling permissions, DeviceManualAddDTO for manual device registration, DevicePageUserDTO for paginated device listing, DeviceUnBindDTO for unbinding devices, and DeviceUpdateDTO for updating device aliases and auto-update settings. Additionally, DeviceToolsCallReqDTO supports tool invocation requests, and DeviceRegisterDTO handles device registration headers.

main/manager-api/src/main/java/xiaozhi/modules/device/dto · high confidence

This change introduces several new backend capabilities for device management. It adds a device address book service allowing users to define aliases and call permissions between devices, with caching for performance. It implements Over-The-Air (OTA) firmware management, including activation via codes, version checking, and ensuring only one firmware per type is retained. Additionally, it introduces a new MQTT gateway authorization mechanism that generates daily SHA-256 tokens for secure communication, and enhances device service logic to handle connection updates and online status reporting.

main/manager-api/src/main/java/xiaozhi/modules/device/service/impl · high confidence

A new IoT tool provider has been added to the server, enabling the system to interact with IoT devices via tool definitions. This change introduces a module that parses device descriptors to register specific tools for querying device properties (e.g., \get\_devicename\_property\) and executing device methods (e.g., \devicename\_method\). It includes an executor that handles these tool calls by sending JSON commands over WebSocket and updating local device state, along with handlers to manage descriptor registration and status updates from connected devices.

_main/xiaozhi-server/core/providers/tools/device\iot · high confidence

A new standalone digital-human test module has been added to main/digital-human, providing a local testing page and frontend interaction resources to help integrate the full digital-human interaction chain. The module includes a Python startup script (start.py) that serves the interface on port 8006, featuring a Live2D model display, a chat message stream, and camera controls. It supports local wakeword runtime via a WebSocket bridge and includes a settings panel for configuring device details, wakeword lists, and MCP tools. The default configuration disables emotional expressions (emojiEnabled=false) to reduce resource usage during testing.

main/digital-human · high confidence

The manager-api now includes a comprehensive set of new entity classes to support expanded agent capabilities. These include AgentEntity and AgentTemplateEntity for managing agent configurations (such as ASR, TTS, LLM, and SLM model selections), AgentSnapshotEntity for configuration versioning and restore, and AgentChatHistoryEntity, AgentChatAudioEntity, and AgentChatTitleEntity to store chat logs, audio data, and session titles. Additionally, AgentVoicePrintEntity supports voiceprint recognition, AgentContextProviderEntity enables context source configuration, AgentPluginMapping handles plugin associations, and AgentTagEntity/AgentTagRelationEntity provide tagging functionality.

main/manager-api/src/main/java/xiaozhi/modules/agent/entity · high confidence

This change introduces three new enum and constant classes in the agent module to support snapshot management, chat history categorization, and MCP communication. AgentSnapshotField defines the specific agent properties (such as model IDs, prompts, and configuration settings) that can be captured in a snapshot and restored, enabling granular snapshot and restore operations. AgentChatHistoryType adds an enumeration to distinguish between user and agent messages in chat history records. XiaoZhiMcpJsonRpcJson provides pre-serialized JSON-RPC 2.0 request strings for MCP initialization, notifications, and tool listing, optimizing performance by reusing these constant payloads instead of reconstructing them for every request.

main/manager-api/src/main/java/xiaozhi/modules/agent/Enums · high confidence

This update introduces several new function plugins to the server's plugin system. Users can now initiate voice calls between devices using the new \call\_device\ tool, which supports both outgoing calls and answering incoming calls via the manager API. The assistant's personality can be dynamically switched to preset roles (English Teacher,机车 Girlfriend, Curious Boy) using the \change\_role\ plugin. Additionally, news retrieval capabilities have been expanded with new plugins for fetching news from China News (\get\_news\_from\_chinanews\) and NewsNow (\get\_news\_from\_newsnow\), allowing users to request news by category or source. Other new tools include \get\_lunar\ for querying lunar calendar and almanac information, and \handle\_exit\_intent\ for managing conversation termination.

_main/xiaozhi-server/plugins\func/functions · high confidence

The manager-web interface now includes dedicated views for managing the device address book (with per-device permission controls and alias editing), agent templates (create, edit, and batch delete), device management (binding, unbinding, alias editing, and OTA switch), dictionary management (types and data CRUD), feature management (toggling capabilities like voiceprint, knowledge base, and VAD/ASR), and knowledge base management (document upload, parsing, and retrieval testing). These views provide the frontend UI for the corresponding backend APIs, completing the management capabilities for these areas.

main/manager-web/src/views · high confidence

The manager-web interface now includes dedicated dialog components to streamline configuration workflows: AddDeviceDialog allows users to bind devices via a 6-digit verification code; AddModelDialog enables adding model configurations with dynamic, provider-specific credential fields; AddWisdomBodyDialog simplifies creating new agents by requiring only a name; and AddressBookDialog provides a validated form for managing contact entries. Additionally, the UI is enhanced with a reusable CustomButton component supporting distinct visual styles for default, confirm, delete, and add actions, and a new AudioPlayer component offering custom controls for playback, seeking, and volume management.

main/manager-web/src/components · high confidence

The manager-api now exposes dedicated endpoints for managing model configurations and providers. Users can create, edit, and delete model settings (LLM, TTS, etc.) with validation logic that prevents deleting models referenced by agents or intent configurations, and ensures LLM providers are restricted to OpenAI or Ollama types. Additionally, a new provider management interface allows listing, adding, editing, and removing model providers, including the ability to register user-specific knowledge bases as RAG providers.

main/manager-api/src/main/java/xiaozhi/modules/model/controller, main/manager-api/src/main/java/xiaozhi/modules/model/service · high confidence

The \performance\_tester\ directory now includes dedicated scripts for benchmarking specific AI components: \performance\_tester\_asr.py\ and \performance\_tester\_stream\_asr.py\ measure speech-to-text latency (including first-word delay for streaming providers like Doubao and Qwen); \performance\_tester\_llm.py\ evaluates large language model response times and first-token latency using configurable system prompts; \performance\_tester\_tts.py\ and \performance\_tester\_stream\_tts.py\ benchmark text-to-speech synthesis speed (including Alibaba Cloud Bailian endpoints); and \performance\_tester\_vllm.py\ assesses vision model performance. These tools provide structured, tabulated reports on average latency, stability, and error rates for each configured provider.

_main/xiaozhi-server/performance\tester · high confidence

The manager-api now includes three new enumeration classes to support server management and role configuration: ServerActionEnum defines actions such as 'restart' and 'update\_config' for WebSocket server control; ServerActionResponseEnum standardizes response statuses ('success', 'fail') with strict validation; and SuperAdminEnum defines boolean-like values (1/0) for super-administrator status. These enums provide structured, JSON-serializable constants for server-side operations and administrative role checks.

main/manager-api/src/main/java/xiaozhi/modules/sys/enums · high confidence

The manager API now includes new data access object interfaces for system configuration and user management. Specifically, SysDictDataDao and SysDictTypeDao provide methods to query dictionary data by type and retrieve type codes, supporting the dictionary feature. SysParamsDao adds capabilities to get, update, and list parameter values by code, enabling dynamic parameter management. SysUserDao is introduced as a base interface for system user operations.

main/manager-api/src/main/java/xiaozhi/modules/sys/dao · high confidence

This change introduces several new service implementations in the system module: dictionary data and type management (SysDictDataServiceImpl, SysDictTypeServiceImpl) with Redis caching and creator/updater name resolution; parameter management (SysParamsServiceImpl) with strict value-type validation (string, array, number, boolean, JSON) and Redis synchronization; and a user utility service (SysUserUtilServiceImpl) that caches usernames in Redis to avoid repeated database lookups. It also includes updated user service logic for password changes, resets, and admin pagination, as well as a basic token service implementation.

main/manager-api/src/main/java/xiaozhi/modules/sys/service/impl · high confidence

The LoginController now exposes endpoints for user registration, login, password changes, and password retrieval. Registration supports mobile phone verification when enabled via configuration, and both login and registration utilize SM2 asymmetric encryption for password transmission along with captcha validation. A new endpoint allows authenticated users to change their password, while the retrieve-password flow enables users to reset their password via SMS verification if mobile registration is enabled.

main/manager-api/src/main/java/xiaozhi/modules/security/controller · high confidence

The manager-web application now includes several new utility modules in the src/utils directory to support enhanced functionality. cacheViewer.js provides tools to inspect, check, and clear Service Worker caches, specifically targeting CDN resources (CSS/JS) to aid in debugging and cache management. featureManager.js introduces a centralized system for managing feature flags (such as voiceprint recognition, knowledge base, and address book), allowing these capabilities to be toggled via backend configuration or default settings. deviceTime.mjs adds robust utilities for parsing, formatting, and comparing timestamps, addressing issues with cross-timezone device time synchronization. Additionally, constant.js defines application-wide configuration keys (storage, routes, languages), while date.js and format.js offer standardized helpers for date formatting and file size display.

main/manager-web/src/utils · high confidence

The manager-api now exposes a new voice cloning module under /voiceClone and /voiceResource. Users can upload audio files (.mp3, .wav, max 10MB) to initiate voice cloning, update clone names, retrieve audio data via temporary UUIDs, and trigger the cloning training process. The system validates file types and sizes, checks for duplicate voice IDs within a model, and integrates with the model configuration service to manage TTS platform settings. Admins can manage voice resources (create, delete, view) and view TTS platform lists.

main/manager-api/src/main/java/xiaozhi/modules/voiceclone · high confidence

The digital-human module introduces a DetectorAssetsBuilder that validates required ONNX model files and generates a keywords.txt file from configured wake words using pypinyin. The server's ASR layer adds a base provider that manages a priority thread for audio processing, handles voice-stop detection, and runs ASR and voiceprint identification concurrently. A new tool\_types module defines the ToolType enum and ToolDefinition dataclass to standardize tool registration. Additionally, a FishSpeech TTS provider is added, supporting reference audio/text, various audio formats, and configurable generation parameters like seed and temperature.

python · high confidence

The security service layer now supports sending and validating SMS verification codes, introducing rate limiting (60-second interval and daily count limits) to prevent abuse. Additionally, a password change feature has been implemented, which updates the user's password and immediately invalidates existing authentication tokens, requiring the user to log in again.

main/manager-api/src/main/java/xiaozhi/modules/security/service/impl · high confidence

The web application now supports Service Worker-based caching and a CDN delivery mode. When enabled via the VUE\_APP\_USE\_CDN environment variable, the app registers a Service Worker that caches specific CDN resources (Element UI, Vue, Vuex, Vue Router, Axios, and opus-decoder) using a CacheFirst strategy with a one-year expiration, while local static assets are cached with StaleWhileRevalidate and HTML pages with NetworkFirst. A CacheViewer component and Alt+C keyboard shortcut allow users to inspect CDN cache status in the browser console, and a notification prompts users to refresh when a new Service Worker version is detected. Mobile users are automatically redirected to the H5 version if the VUE\_APP\_H5\_URL is configured.

main/manager-web/src · high confidence

The TTS provider system has been reorganized to support a unified streaming interface model. A new Data Transfer Object (DTO) layer (dto.py) defines message types for sentence stages (FIRST, MIDDLE, LAST), content types (TEXT, FILE), and interface types (DUAL\_STREAM, SINGLE\_STREAM, NON\_STREAM). The base provider (base.py) was refactored to handle text processing, markdown cleaning, and correct-word replacement centrally, while delegating audio generation to specific providers. New provider implementations include alibl\_stream.py (Alibaba Cloud CosyVoice dual-stream), aliyun\_stream.py (Alibaba Cloud dual-stream with WebSocket connection reuse), and huoshan\_double\_stream.py (Volcano Engine bidirectional streaming with custom protocol handling). Existing providers like edge.py, doubao.py, and cozecn.py were also updated or added to fit the new architecture, enabling more robust streaming audio delivery and configuration management.

main/xiaozhi-server/core/providers/tts · high confidence

The server now uses a new UnifiedToolHandler and ToolManager to centralize tool execution, supporting server plugins, server-side MCP, device IoT, device MCP, and MCP endpoints. This change enables the configuration and initialization of MCP endpoints via the 'mcp\_endpoint' config key and displays active tool calls on the device interface during execution.

main/xiaozhi-server/core/providers/tools · high confidence

The manager-api now includes a shared common module that standardizes cross-cutting concerns. It adds a MyBatis-Plus data filter interceptor to enforce row-level security based on user/department context, configures a dedicated async thread pool for background tasks, and sets up a Redis caching layer with JSON serialization and an aspect to toggle caching behavior. Additionally, it provides a comprehensive Swagger/OpenAPI configuration that groups endpoints by domain (device, agent, OTA, knowledge, etc.) and establishes a unified exception handling and error code system for consistent API responses.

main/manager-api/src/main/java/xiaozhi/common · high confidence

A change to existing behaviour in main/xiaozhi-server/core/providers/llm/ollama — 10 commits (2 fixs), 1 file.

main/xiaozhi-server/core/providers/llm/ollama · medium confidence · unverified

The plugin system now automatically discovers and loads modules within the \plugins\_func\ package, eliminating the need for manual registration. A new structured registry (\FunctionRegistry\) manages available functions, supported by enums for \ToolType\ (including IOT and MCP support) and \Action\ (introducing a \RECORD\ action to log tool calls without invoking the LLM). This change also introduces a mapping mechanism to expand module-level plugin names into specific function names, ensuring accurate function resolution and description updates.

_main/xiaozhi-server/plugins\func · high confidence

The API layer in manager-web has been refactored to use a centralized request service (httpRequest.js) built on the Flyio library. This change introduces automatic inclusion of the user's authentication token in request headers and supports server-side internationalization by dynamically setting the Accept-Language header based on the current locale. It also standardizes error handling, including automatic redirection to the login page on 401 responses and consistent display of error messages.

main/manager-web/src/apis · high confidence

The agent module's DAO layer has been significantly expanded to support new and refined capabilities. New mappers (AgentChatTitleDao, AgentContextProviderDao, AgentTemplateDao, AgentVoicePrintDao, AiAgentChatAudioDao, AiAgentChatHistoryDao) provide direct persistence access for chat titles, context providers, templates, voice prints, and chat audio/history records. The AgentDao now includes methods to retrieve device counts, fetch default agents by MAC address, load agent info with plugins, lock records for concurrent configuration writes, and perform precise snapshot field updates. AgentSnapshotDao adds versioning, history navigation, and batch redaction support, while AgentTagDao and AgentTagRelationDao enable tag-based organization. AgentPluginMappingMapper supports plugin associations, and AgentCorrectWordMappingDao handles batch insertions and deletions for correction mappings. These changes collectively enhance how agent configurations, histories, and metadata are stored and retrieved.

main/manager-api/src/main/java/xiaozhi/modules/agent/dao · high confidence

The backend API now provides localized error messages for a broader range of features, including voice cloning, RAG configuration, OTA updates, and device management. This update adds new translations for German (de\_DE) and Brazilian Portuguese (pt\_BR), and introduces a new Vietnamese (vi\_VN) locale. Users will see clearer, language-specific feedback for errors related to voiceprints, model configurations, and API validation.

main/manager-api/src/main/resources/i18n · high confidence

A new global stylesheet (global.scss) has been added to standardize the visual appearance of the application. It overrides browser autofill highlighting to remove background colors, applies a consistent centered layout and semi-transparent backdrop to dialog wrappers, and defines a reusable mixin for custom scrollbar styling. Additionally, it enforces specific dimensions for the footer, video icons, and light-themed popovers, and sets a maximum width variable for table-based dialogs.

main/manager-web/src/styles · high confidence

The VAD provider now uses the Silero ONNX model to detect speech activity, implementing a dual-threshold logic (high and low thresholds) to reduce false positives and negatives. It maintains per-connection state for context and silence duration, supporting both manual and automatic listening modes to improve recognition accuracy and reduce interruptions.

main/xiaozhi-server/core/providers/vad · high confidence

The manager-api now implements a dedicated OAuth2 authentication flow using Apache Shiro. This change introduces a new filter (Oauth2Filter) that extracts Bearer tokens from the Authorization header, a realm (Oauth2Realm) that validates these tokens against the database to retrieve user details and permissions, and supporting classes for token generation and user context retrieval. For users, this establishes the underlying mechanism for securing API endpoints with access tokens, replacing any previous authentication method.

main/manager-api/src/main/java/xiaozhi/modules/security/oauth2 · high confidence

The LLM service layer has been refactored to perform conversation summarization and session title generation directly on the Java server side. This change introduces a new LLMService interface and an OpenAI-style implementation that calls external model APIs to generate summaries (including merging with historical memory) and concise titles, while automatically disabling the 'thinking' mode for specific provider domains like DeepSeek and Aliyun to ensure consistent response formats.

main/manager-api/src/main/java/xiaozhi/modules/llm · high confidence

The API now exposes dedicated data transfer objects for authentication flows: LoginDTO accepts a username (mobile number), password, mobile captcha, and a captcha ID, while SmsVerificationDTO handles SMS code requests with a phone number, captcha code, and captcha ID. These structures define the input schema for the login and SMS verification endpoints.

main/manager-api/src/main/java/xiaozhi/modules/security/dto · high confidence

The server now uses a new configuration loading mechanism that merges a default \config.yaml\ with a user-defined \data/.config.yaml\, allowing users to override settings without modifying the default file. It supports fetching configuration dynamically from a remote management API (Java backend) when a \manager-api\ URL is provided, enabling remote device management. The logging system has been upgraded to use \loguru\ with automatic file rotation (10 MB per file) and retention (30 days), including version and module selection tags in log entries. Additionally, the \ManageApiClient\ has been refactored to fix race conditions during shutdown and connection pool management, ensuring stable operation when connecting to the management API.

main/xiaozhi-server/config · high confidence

The server's message handling logic has been restructured to decouple text and audio processing. A new registry-based system (TextMessageProcessor) now routes text messages (hello, abort, listen, iot, mcp, server, ping) to specific handlers, replacing the previous monolithic approach. Audio reception is now handled by a dedicated module that manages VAD, AEC, and intent analysis, while audio sending is managed by a rate-controlled module to ensure smooth playback. This refactoring improves code maintainability and allows for more precise control over device interactions and audio flow.

main/xiaozhi-server/core/handle · high confidence

The text message handling logic has been decoupled into dedicated handler classes (Abort, Hello, IoT, Listen, MCP, Ping, and Server) to improve code structure and maintainability. This change introduces a WebSocket heartbeat mechanism via a new PingMessageHandler, allowing clients to maintain connection stability when enabled. Additionally, the Listen handler now supports specific device call instructions and refined wake-word processing, while the Server handler enables secure, secret-validated dynamic configuration updates and server restarts over the WebSocket connection.

main/xiaozhi-server/core/handle/textHandler · high confidence

The security service layer has been restructured with new interfaces for captcha handling, Shiro token management, and user token operations. A key behavioral addition is the introduction of SMS verification codes, allowing users to send and validate one-time passwords via phone number in addition to existing image captchas. The token service now explicitly supports password changes and user retrieval by token, providing the backend foundation for the new password change API and improved login token validation.

main/manager-api/src/main/java/xiaozhi/modules/security/service · high confidence

The manager-api now registers a Shiro-based security filter chain that enforces OAuth2 authentication for most endpoints while allowing anonymous access to public routes (login, registration, OTA, docs) and applying a server-secret filter to internal management paths (config, device address-book, chat history/report/summary/title). It also configures the Spring MVC layer to handle CORS broadly, serialize Java 8 date/time types and Long IDs as strings via Jackson, and resolve the user's preferred language from the Accept-Language header, supporting Simplified Chinese, Traditional Chinese, US English, German, and Vietnamese.

main/manager-api/src/main/java/xiaozhi/modules/security/config · high confidence

The system startup process now explicitly refreshes the device address book cache upon initialization. This ensures that the cached address book data is up-to-date when the application starts, supporting the new device-to-device communication and address book features.

main/manager-api/src/main/java/xiaozhi/modules/config/init · high confidence

A new MyBatis type handler (ContextProviderListTypeHandler) was added to correctly serialize and deserialize lists of context providers to and from JSON in the database. The implementation extends MyBatis's BaseTypeHandler rather than MyBatis-Plus's AbstractJsonTypeHandler to ensure compatibility across MyBatis-Plus versions (specifically avoiding issues introduced in 3.5.6).

main/manager-api/src/main/java/xiaozhi/modules/agent/typehandler · high confidence

Added device module tests for user validation and Redis device info simulation; Added i18n duplicate key validation script; Added regression tests for device service logic; Added smoke test for plugin package import; Added tests for ASR language tag filtering and TTS quote-safe sentence segmentation; Added tests for Shiro configuration bean initialization; Added tests for agent snapshot and context provider handling; Added tests for config merging logic; Added unit and contract tests for utility modules and frontend contracts; Added unit tests for CorrectWordFileService; Added unit tests for KnowledgeBaseAdapterFactory; Added unit tests for TimbreServiceImpl; Added unit tests for core utility modules; Added unit tests for login, dictionary, and system parameter services; Added unit tests for manager-api utilities and services; Initial pytest test suite for server components.

This change introduces the dependency configuration files for the project's core components. The Python server (xiaozhi-server) now uses pyproject.toml and requirements.txt, pinning key libraries such as torch 2.2.2, torchaudio 2.2.2, and numpy 1.26.4, while allowing upgrades for others like openai 2.8.1 and mem0ai 1.0.0. The Java-based manager API (manager-api) is initialized with a pom.xml defining Spring Boot 3.4.3, MyBatis-Plus 3.5.17, and Shiro 2.0.2. Additionally, the mobile admin client (manager-mobile) and web admin client (manager-web) are added with their respective package.json and lockfiles, establishing the frontend dependency trees for the Uni-app mobile interface and the Vue 2 web interface.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 41.

• Code Health 34
• Architecture 73
• Maturity 59
• Readiness 41
• Security 57
• Accessibility 41
• Performance 60

• 300 commits — 195 feature/other, 105 fixes

• (repo) — 103 commits
• main/manager-web — 78 commits
• main/xiaozhi-server — 61 commits
• main/manager-api — 40 commits
• main/manager-mobile — 5 commits
• (root) — 4 commits
• .github/dependabot.yml — 2 commits
• docs/images — 2 commits
• .github/workflows — 1 commit
• docs/Deployment.md — 1 commit
• docs/FAQ.md — 1 commit
• docs/mcp-vision-integration.md — 1 commit
• docs/web-search-integration.md — 1 commit

• fix: #3212 fix： 1.优化知识库连接RAGFlow异常提示，修改前端卡片提示样式
• fix: #3212 fix： 1.修复RAGFlow连接异常导致级联删除问题 2.修复RAGFlow获取数量逻辑问题 3.远程失败不再删除本地影子记录，避免产生RAGFlow端孤儿文档 4.优化前端提示语
• fix: #3212 fix： 1.修改知识库前端卡片提示样式，优化警示效果
• fix: Fix agent snapshot history issues
• fix: Merge branch 'main' into fix-model-name
• fix: Merge branch 'refs/heads/main' into fix-knowledge-base-ragflow-bug
• fix: Merge branch 'refs/heads/main' into fix-voiceprint-dialogue-bug
• fix: Merge pull request #3204 from xinnan-tech/py-fix-calling
• fix: Merge pull request #3210 from xinnan-tech/fix-knowledge-base-web-bug
• fix: Merge pull request #3213 from xinnan-tech/fix-knowledge-base-ragflow-bug
• fix: Merge pull request #3224 from xinnan-tech/py-fix-bug
• fix: Merge pull request #3235 from xinnan-tech/py-fix-tts
• fix: Merge pull request #3267 from xinnan-tech/fix-agent-idor
• fix: Merge pull request #3270 from xinnan-tech/fix-voiceprint-dialogue-bug
• fix: Merge pull request #3271 from xinnan-tech/py-fix-bug
• fix: Merge pull request #3278 from xinnan-tech/agent-snapshot-history-fixes
• fix: Merge pull request #3279 from xinnan-tech/fix/issue-3276-context-provider
• fix: Merge pull request #3283 from xinnan-tech/fix/device-timezone-consistency
• fix: Merge pull request #3292 from xinnan-tech/fix/manager-api-test-i18n
• fix: Merge pull request #3298 from xinnan-tech/py-fix-intent
• …and 280 more

• 0 containers · 2 bounded contexts · 0 dependency edges (baseline)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

xinnan-tech/xiaozhi-esp32-server was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.